Python Support
Detailed Description
As of v0.86, the python bindings moved from a pyste based generator to a pyplusplus based generator. This has meant a significant improvement in bindings quality as pyste required substantial patching to get it to generate suitable output for TnFOX, but also that pyplusplus generates better quality output anyway. Nevertheless, it took till v0.87 to get both the bindings and pyplusplus itself up to a reasonable state - my thanks to Roman Yakovenko for the tireless work he has put into pyplusplus, most especially that he has spent so much time on getting the TnFOX bindings up to scratch.
One of the big advantages of pyplusplus over pyste is that you can programmatically set call policies and patch in code. The many days of manually writing policies that pyste required were effectively wasted time when viewed from this context - however, it also means that the pyplusplus bindings are far more maintainable than pyste ones. And the pyste ones were already far more maintainable than any SWIG based solution.
Current issues with the python bindings which will hopefully get fixed soon:
- pyplusplus generates wrong code for FX::FXWinShellLink (misinterprets unions). Currently the entire class is excluded.
TnFOX's Python bindings are almost entirely automatically generated and you can find the necessary files in the Python directory. You will also need Python v2.4 or later and the Boost library v1.34 or later (or else patch in my
void * support patch into an earlier version).
You'll also need a modern compiler such as MSVC7.1 or later or GCC v3.22 or later (GCC v3.4 or later is far faster, so you should use that instead. GCC v4.0 or later comes with my symbol visibility patch which very substantially improves load times)
You'll also need a fast computer. Anything using this much compile-time introspection takes an age but you can rest easy knowing that for every two years hence compilation speed should double. In only a few years, it'll take no more than five minutes or so. Until then, distributed compilation is very useful as are multi-processor machines - and you can tell scons to use as many processors as you have using the -j switch. You should always run one more build than there are processors in your system eg; two for a uniprocessor system, three for a dual core system etc.
You also need a minimum of 1Gb RAM. This is for linking the bindings, which takes a huge amount of RAM. Anything less swaps to hell and takes forever.
Place Boost into a directory called "boost" next to the TnFOX directory. Overwrite the files in Boost.Python with the contents of BoostPatches.zip. Get a bjam from somewhere (see boost docs). Run bjam "-sBUILD=release" in boost/libs/python/build (use debug if you're building a debug version). This should build a TnFOX customised boost.python library - no compile errors should occur, but it should refuse to link (this is deliberate).
Install gccxml, pygccxml and pyplusplus.
Open a command window and go into the TnFOX/Python directory. If you have changed the header files of TnFOX (eg; through upgrading the version), make VERY sure you delete the fx.xml file first - this is the cache of parsing the TnFOX headers - and also delete the generated directory as pyplusplus uses the existing output to speed up its operation.
Run 'python create_tnfox.py'. After a few minutes it should return. If there is a file called patch.diff in the Python directory then apply this using:
cd generated
patch -p1 < ../patch.diff
As pyplusplus improves, the requirement to do this patch will disappear.
Run scons. This part takes between an hour and six hours depending on how fast your machine is.
- Note:
- If using GCC, you really want to use a
-fvisibility enabled version (>=v4.x) of GCC if possible as it will reduce the size of the TnFOX shared object by around 20Mb and you will reduce load times for anything linked against TnFOX.so from six minutes to four seconds :). You will also need v1.3.2 or later of Boost.
Afterwards, it's as simple as
... to begin writing powerful portable GUI based apps from Python!
In fact it's easier to list what doesn't work rather than what does which is the overriding quantity - but off the top of my head:
- Apart from the exceptions below, all classes & all enums and all members (nested classes, enums, data etc) therein including a full duplication of the class inheritance structure, except for those retired for reasons of deprecation or where it doesn't make sense to include them. See declarations_to_exclude.py and files_to_exclude.py. Most of these are sensible.
- Apart from one virtual method in FXApp, every virtual thing can be overriden by python code.
- All method & function overloads are available.
- Default parameters work as expected, and you can name your parameters
- Complex number support
- Provision for simple invocation of python code
- Multiple python interpreters, plus multiple threads therein.
- Via BPL, extremely easy manipulation of python objects from C++ plus embedding of python mini-scripts and python expression evaluations.
Furthermore I've implemented a number of automatic conversions where TnFOX classes transparently become Python ones and vice versa:
- FX::FXString with python strings. On v1.6 FOX based builds, FX::FXString always converts to a python unicode object. String and Unicode objects convert to FX::FXString. On v1.4 FOX based builds, python unicode object conversion is not supported.
- FX::FXuchar, FX::FXchar, FX::FXushort, FX::FXshort, FX::FXuint and possibly FX::FXint & FX::FXulong (depending on host long integer size) all automatically translate to/from a python integer. FX::FXulong and FX::FXlong on 32 bit machines automatically translate to/from a python long integer.
- A python list of strings becomes
const char ** and vice versa. Be VERY careful with this as I don't copy the strings so ensure the list lives forever. This was intended purely for use for argv situations and nothing more.
- FXException and anything deriving off it when thrown appear as a python exception of e.report(). Even the most appropriate type of python exception is chosen where possible.
- Python exceptions automatically become a FX::FXPythonException.
- C style arrays become full python lists which can be sliced, altered etc. - the only restriction is that they cannot be extended or shrunk. You can even alter FXImage's bitmap data directly through altering the sequence returned by getData()! Note however that FXGLViewer::lasso and FXGLViewer::select's returns are very simple container emulations and only offer read-only access. However, in all cases length is correctly observed and maintained.
- The QTL (currently only QValueList & QMemArray but QPtrList is pending) maps onto a fully-featured python emulation. Items returned from QPtrList are expected to be managed entirely by
new - if this isn't the case, do not modify the list - use the QPtrList methods directly. If you take references from a QPtrList, their lifetime is set to expire with the list reference.
- The same method for managing FXWindow message dispatching as FXPy was chosen for aiding compatibility. For each instance of the window class, in its constructor you should call the member functions FXMAPFUNC(), FXMAPFUNCS(), FXMAPTYPE() or FXMAPTYPES() as appropriate. These have the same spec as their macro counterparts in FXObject.h.
If you look inside the Python directory, there is a "FXPy examples" directory containing some of the same examples that come with FXPy except altered to use TnFOX. The changes in most cases were trivial, but you should note that these python bindings are automatically generated rather than by hand, so useful behaviour like FXPy had is not possible:
- If an enum is named, it is always accessed in TnFOX's python bindings by prefixing the constant with its enum name eg;
FXSelType.SEL_COMMAND rather than just SEL_COMMAND. I know this isn't like the C++, but I agree with BPL's designers that more disambiguation is better than less.
- Another difference is that python object instances and their C++ counterpart are tied together strongly in the direction of python upwards. In other words, TnFOX can delete the C++ side of a python object but the python object shall continue to exist, except that it becomes a zombie. Similarly to this, if you delete the python side of an object, it always deletes its C++ side. Typical code in C++ such as:
new MenuCommand(parentmenu, ...)
self.something to it), python will delete it. This is a major difference from FXPy and code will need to be adjusted to support it - see imageviewer.py ported from FXPy to see what I mean.
Not everything can be supported. Python and C++ are two very different languages which is why they complement each other so well. Things that are not supported and never will be supported:
- Setting or retrieving pointers to functions. There is a workaround for FOX's sort functions which lets you set arbitrary python code as the sort function. The code which enables this is public (FX::FXCodeToPythonCode) so if you have a similar problem, you can use that. Remember that C++ can't generate new C++ code like python code can, so this will always be a design constraint.
- Anything taking ellipsis parameters (...)
- Python doesn't have the concept of something being unsigned. Furthermore there are no chars or shorts to speak of - just a plain signed integer which can be far bigger than a 64 bit integer.
- The FOX portion of TnFOX is not exception safe. BPL throws an exception when something displeases it and if there's any GUI code up the call stack, it can leave FOX in an indeterminate state (ie; royally screwed sometimes where the only realistic recourse is an immediate fatal exit).
- Python code cannot be cancelled via a POSIX thread cancellation (it would leave reference counts dangling) and so thread cancellation is automatically disabled during calls into python code. This means you must take care to call QThread::checkForTerminate() which if it returns true, raise a SystemExit exception or something (SystemExit is ignored by the default python run() call code).
And then also there's a long list of things which will be added or fixed with time:
- None of the TnFOX docs are reflected into python doc strings even though Boost.Python supports this. This is one of the future features which should be added to pyplusplus in time.
- BPL gets confused where overloads have a static member function and a normal member function. It will default to the static version - therefore you'll need to pass the instance as the first parameter to the non-static version. This may get fixed in some future version of BPL.
Finally, there is the issue of lifetime management. Boost.Python allows you to tie the lifetime of things together to prevent dangling references eg; you get the address() from a QBlkSocket and delete the QBlkSocket - now the address is clearly no longer valid unless you made a copy of it which is exactly what has been enforced.
As it happens, actually in this case as a QHostAddress is copyable, a copy is made silently for you. But the point remains valid - wherever I have had a choice, I erred on the side of caution and chose anything returned (all reference and pointer types - values are copied as are const references when the object has a copy constructor) to live only as long as the thing you got it from. Hence things like FXApp::getWindowAt() where the window returned would almost certainly outlive the reference you temporarily got from FXApp::instance() will in fact become invalid when your FXApp reference is killed. If you think I was being over-zealous, please post to the newsgroup below stating why with a good example of why alternative behaviour would be better and still safe!
If you want support for these bindings, please join the tnfox-discussion
at
lists.sourceforge.net mailing list and ask there.
