midhunadarvin

November 27, 2023

Embed Python in C++

Why Embed Python in C++?

Before we dive into the technical details, let’s discuss the motivations behind embedding Python in C++. There are several compelling reasons to do so:

1. Scripting Capabilities: Python is excellent for scripting and automation, making it easy to create dynamic behavior within a C++ application. By embedding Python, you can expose your C++ code to Python scripts, allowing you to build customizable and extendable applications.
2. Access to Python Libraries: Python has a rich ecosystem of libraries and frameworks for various tasks, such as machine learning, data analysis, and web development. Embedding Python in C++ allows you to leverage these libraries from your C++ application.
3. Rapid Prototyping: Python’s concise syntax and interactive nature make it an ideal choice for rapid prototyping. You can quickly experiment with ideas and then seamlessly integrate them into your C++ codebase.
4. GUI Development: Python has powerful GUI libraries like PyQt and Tkinter. By embedding Python in a C++ application, you can create user-friendly interfaces easily, while the heavy lifting remains in C++.
5. Extensibility: Embedding Python can make your application more extensible, allowing users to create custom plugins and extensions. This fosters a vibrant ecosystem of third-party contributions.

Prerequisites

• Python : https://www.python.org/about/gettingstarted/
• python-dev : ( header files you need to build Python extensions )
• C++ Compiler ( GNU / Clang / Microsoft Visual C++ )

The Process

Python Module

Create a python module named multiply.py

def multiply(a,b):
	print ("Will compute", a, "times", b)
	c = 0
	for i in range(0, a):
        	c = c + b
	return c

This function takes in 2 arguments ( a, b ) and computes the value of the expression a multiplied by b

We will next create a c++ program to load this python module, call the multiply method and retrieve the result.

C++ Program

Create a c++ source file called main.cpp . We will include the Python header file to use the Python/C API methods and objects. These are required to setup, initialize and call the python script from our C++ program.

#include <Python.h>

int main(int argc, char *argv[])
{
	return 0;
}

In an application embedding Python, the Py_Initialize() function must be called before using any other Python/C API functions; with the exception of a few functions and the global configuration variables. This function initializes the python interpreter that is used to interpret the python script. We will add it in the main method of our c++ program.

Py_Initialize();

We will import the python script created earlier as a python module. PyImport_Import will help us to import it. It returns a pointer reference to a PyObject ( which is a python object structure part of the c++ python API )

PyObject *pythonModule = PyImport_Import("multiply.py");

From the python module, we need to get the reference to the multiply function, so that it can be invoked from the c++ code.

if (pythonModule != NULL) {
        PyObject *multiplyFunc = PyObject_GetAttrString(pModule, "multiply");
				...
}

We will need to add checks if the function exists and it is callable.

if (multiplyFunc && PyCallable_Check(multiplyFunc)) {
	...
}

Now since we have the reference to the multiply function we can call it, but it takes 2 arguments, a and b , so we need to initialize the arguments before calling the function.

PyObject *pArgs = PyTuple_New(2);

// We are setting the arguments as 2 and 3. Instead of hard coding
// we could take the commandline arguments. Hard coding the arguments
// here for the ease of understanding.
// The arguments are passed as PyTuple object structure.
PyTuple_SetItem(pArgs, 0, PyLong_FromLong(2));
PyTuple_SetItem(pArgs, 1, PyLong_FromLong(3));

Call the python function using PyObject_CallObject method, the result of which would be pointer reference to a PyObject. For converting the result into a long int variable in C++, we use the PyLong_AsLong method.

PyObject *pValue = PyObject_CallObject(multiplyFunc, pArgs);
if (pValue != NULL) {
    printf("Result of call: %ld\\n", PyLong_AsLong(pValue));
}

Clean up

We would have to clean the memory references to avoid memory leak. For dereferencing the PyObject pointers we can use the Py_DECREF function. We can call Py_Finalize to cleaning up the python interpreter at the end of the program.

Py_DECREF(pValue);
Py_DECREF(pArgs);
Py_DECREF(multiplyFunc);
Py_DECREF(pythonModule);

Py_Finalize();