Advanced C++ / Python integration with Boost.Python (Part 2)


Last time we looked at how to use Boost.Python to wrap a very simple piece of C++ code. This time we’re going to take that one step further along, and do the same thing for a more complex C++ example – which includes a C++ Class.

For the purposes of this example – let us assume that we have a “legacy” C++ class (i.e. one that we’re not going to change): which looks something like this:

This (deliberately, slightly contrived) class has three methods (and a constructor). We can set the three 8-bit unsigned integers (uint8_t) directly using the set_data() method; we can print the data to the screen; or we can either import from or export to an unsigned char*.

In order to be able to use the importer & exporter methods with Python (via Boost.Python) we’ll need to further wrap these – as Boost.Python won’t let us use unsigned char* directly.

In order to use the exporter() method; we need a wrapper for the unsigned char* so that in Python it will become a Python bytes object.

If we want to be able to extract the internal state of the object (for example to be able to copy it to a new object to create a copy of the internal state of the original one) we need to define a way for Python to understand the data it’s going to receive. In C++ we’d just create a void* or unsigned char* and use memcpy to copy the arbitrary memory; but we can’t do that for Python. Instead, here we make use of MemoryView a type of PyObject object (a Python built-in type) to contain the data. But in order to get the data into that object we need a little additional work.

However since we’ve already said we don’t want to change our class directly – we could put all of the wrapper code into a separate library.cpp file…

The wrapper code required is here:

We create a PyObject* to store the data; and a regular C++ unsigned char* to contain the returned data from the exporter() method. The clever bit comes when we call PyMemoryView_FromMemory the intuition for which is essentially doing the same as our memcpy… It takes three bytes (as specified) from export_data, and makes them available as read-only data to Python.

From Python we can then turn it into a bytes object using .tobytes(). We can then interact with the data as with any other Python data…

In fact we can go one better – and obviate the need to do the bytes conversion in Python if we use one additional line:

This way the return value when we call the exporter from Python is just a regular Python bytes object.

What about going back the other way? How can we do this import from Python bytes?

Again we need a wrapper function…

This code is based on an answer in Stack Overflow (see here). As described there: “…the auxiliary C++ function would need to populate a continuous block of memory with the elements of from the bytes [object]. The boost::python::stl_input_iterator can provide a convenient way to construct C++ containers, such as std::vector<char>, from a Python object, such as … bytes …”

The original example there (which was designed to be robust with Python 2 code too) – constructs an iterator from the py_buffer object (because, as the answer explains, Python 2 doesn’t have bytes and the str object use instead isn’t iterable). Since it’s nearly 2018, no-one should be using Python 2 any more; so we can skip that additional complexity.

We can then consume this from Python with something like this:

The only remaining thing to do is to actually link all of this up – so that Boost.Python knows to call these functions: rather than calling the original class methods directly.

Here we see that for the print & set methods we just provide the method directly to the Boost.Python macro; but for the other two (would-be) Python-side methods: we provide the functions discussed above.

The full code to support this example can be found here: https://github.com/Auctoris/boost_python_impex