Various ways to integrate Python and C (C++)

Koichi Tamura's blog: Various ways to integrate Python and C (C++)

Various ways to integrate Python and C (C++)

This is originally what I wrote in a mail I sent to a friend of mine. I modified it a little bit to post it here.

There are many tools for that.

- boost::python
- swig
- Py++
- pyrex/cython
- ctypes
- pyInline, scipy::weave, etc

boost::python, swig, pyrex, Py++ can be used for extending, which means writing a Python extension module (.pyd file on windows and .so file on Linux and Mac) in C (C++). That lets a Python script access to functions written in C (C++) , passing arguments to c and getting results from C (C++).

For embedding, you can use boost::python and pyrex/cython. boost::python will be better because pyrex/cython is mainly for extending. Embedding means starting a Python interpreter in your C (C++) program and running Python scripts inside the program. Extending is said to be better than embedding but I don't really care when embedding seems to be better for my specific problem.

boost::python is easy to use for a C++ programmer but it needs some knowledge of C++ template, and it's a little bit troublesome until you get the right compiler options. It is good if you don't have any code yet and you start writing a Python extension from a scratch. boost::python is sometimes good because all you need to write is c++ code and there's no additional script, interface definition file, etc.

If you already have a big amount of C++ code (usually library) and you want to make a Python wrapper for the library, swig is the one to choose. It is what Autodesk has used to make Maya's Python API wrapper. What I think is not good about swig is that essentially it cannot make an extension module which contains a Python class. It can only create functions in an extension. It still can generate c++ wrapper but to wrap a c++ class, it creates an extension that contains methods (i.e. functions), and it creates a module written in Python that has a class which methods are redirected to the functions in the extension module. So two files (.pyd and .py) are generated to wrap a single class. Still it's a defacto standard for wrapping a C (C++) library in my opinion.
To wrap it you need to write an interface definition file. For a very simple program, C (C++) header file can be used as an interface definition file.

Py++ is another option to wrap C (C++) library. it generates C++ source code which uses boost::python, and when you compile it, you'll get a Python extension that wraps the C (C++) library. It depends on several libraries . I don't recommend it for beginners since it takes some work to get it working. I haven't used it a lot.

If you already have Python scripts and want to make them faster withough rewriting it in C (C++), pyrex/cython is good for it. It needs some experience to use it (two/three weeks?) but if you get used to it you can convert a Python module to an extension module quickly, so in the long run it may worth the effort. Apart from that, pyrex/cython is Pythonic! I would use cython because it lets me use list comprehension (I love it!) but first I would compare them some more to see if cython has any big downside.

ctypes is part of Python standard library. You can use it from Python2.5. It calls shared libraries in a platform dependent way. I'd rather not use it because it's platform dependent. It's good if some functionality is given as .dll or .so, and you want to use it from a Python script.

pyInline, scipy::weave, etc is somewhat quite different. You can embed c code inside a Python script. I wrote an article about it on my blog.
http://koichitamura.blogspot.com/2008/05/embedding-c-code-in-python.html
If you are mainly making something in Python and want to convert just a little piece of the script in c for performance reason, I think it's a good idea to use it. I think It's good for Python plug-ins (though I haven't done it myself). Some of them are unix only.

July 23 added.
skuda gave me an information on SIP.
http://www.riverbankcomputing.co.uk/news
http://www.controlvideo.de/sip/out/index.html
Looks it covers what swig lacks.