Skip to content

Instantly share code, notes, and snippets.

@Xonxt
Last active February 24, 2023 02:03
Show Gist options
  • Save Xonxt/26d2a9ac6c56505d0896822ede99a646 to your computer and use it in GitHub Desktop.
Save Xonxt/26d2a9ac6c56505d0896822ede99a646 to your computer and use it in GitHub Desktop.
Sending a cv::Mat from C++ to a Python script
/*
This requires adding the "include" directory of your Python installation to the include diretories
of your project, e.g., in Visual Studio you'd add `C:\Program Files\Python36\include`.
You also need to add the 'include' directory of your NumPy package, e.g.
`C:\Program Files\PythonXX\Lib\site-packages\numpy\core\include`.
Additionally, you need to link your "python3#.lib" library, e.g. `C:\Program Files\Python3X\libs\python3X.lib`.
*/
// python bindings
#include "Python.h"
#include "numpy/arrayobject.h"
#include <iostream>
#include "opencv2/opencv.hpp"
// references to all the functions
PyObject *m_PyDict, *m_PyFooBar;
// reference to the Pyhton module
PyObject* m_PyModule;
int main(int argc, char** argv) {
// initialize Python embedding
Py_Initialize();
// set the command line arguments (can be crucial for some python-packages, like tensorflow)
PySys_SetArgv(argc, (wchar_t**) argv);
// add the current folder to the Python's PATH
PyObject *sys_path = PySys_GetObject("path");
PyList_Append(sys_path, PyUnicode_FromString("."));
// this macro is defined by NumPy and must be included
import_array1(-1);
// load our python script (see the gist at the bottom)
// use the script's filename, sans the extension
m_PyModule = PyImport_ImportModule("pythonScript");
if (m_PyModule != NULL)
{
// get dictionary of all available entities in the module
m_PyDict = PyModule_GetDict(m_PyModule);
// grab the functions we are interested in (using its name)
m_PyFooBar = PyDict_GetItemString(m_PyDict, "foo_bar");
// execute the function
if (m_PyFooBar != NULL)
{
// take a cv::Mat object from somewhere (we'll just create an empty one)
cv::Mat img = cv::Mat::zeros(480, 640, CV_8U);
// total number of elements (here it's a greyscale 640x480)
const unsigned int nElem = 640 * 480;
// create an array of apropriate datatype
uchar* m = new uchar[nElem];
// copy the data from the cv::Mat object into the array
std::memcpy(m, img.data, nElem * sizeof(uchar));
// the dimensions of the matrix
npy_intp mdim[] = { 480, 640 };
// convert the cv::Mat to numpy.array
PyObject* mat = PyArray_SimpleNewFromData(2, mdim, NPY_UINT8, (void*) m);
// create a Python-tuple of arguments for the function call
// "()" means "tuple". "O" means "object"
PyObject* args = Py_BuildValue("(O)", mat);
// if we want several arguments, we can write ("i" means "integer"):
// PyObject* args = Py_BuildValue("(OOi)", mat, mat, 123);
// to send two images and an integer, equivalent to Python's (mat, mat, 123) tuple
// see detailed explanation here: https://docs.python.org/2.0/ext/buildValue.html
// execute the function
PyObject* result = PyEval_CallObject(m_PyFooBar, args);
// process the result
// ...
// decrement the object references
Py_XDECREF(mat);
Py_XDECREF(result);
Py_XDECREF(args);
delete[] m;
}
}
else
{
std::cerr << "Failed to load the Python module!" << std::endl;
PyErr_Print();
}
return 0;
}
// same example, but for a 3-channel RGB image.
// python bindings
#include "Python.h"
#include "numpy/arrayobject.h"
#include <iostream>
#include "opencv2/opencv.hpp"
// for the references to all the functions
PyObject *m_PyDict, *m_PyFooBar;
// for the reference to the Pyhton module
PyObject* m_PyModule;
int main(int argc, char** argv) {
// initialize everything (see gist above for details)
Py_Initialize();
PySys_SetArgv(argc, (wchar_t**)argv);
PyObject *sys_path = PySys_GetObject("path");
PyList_Append(sys_path, PyUnicode_FromString("."));
import_array1(-1);
m_PyModule = PyImport_ImportModule("pythonScript");
if (m_PyModule != NULL)
{
// get dictionary of available items in the module
m_PyDict = PyModule_GetDict(m_PyModule);
// grab the functions we are interested in
m_PyFooBar = PyDict_GetItemString(m_PyDict, "foo_bar");
// execute the function
if (m_PyFooBar != NULL)
{
// take a cv::Mat object from somewhere (we'll just create one)
cv::Mat img = cv::Mat::zeros(480, 640, CV_8UC3);
// total number of elements (here it's an RGB image of size 640x480)
const unsigned int nElem = 640 * 480 * 3;
// create an array of apropriate datatype
uchar* m = new uchar[nElem];
// copy the data from the cv::Mat object into the array
std::memcpy(m, img.data, nElem * sizeof(uchar));
// the dimensions of the matrix
npy_intp mdim[] = { 480, 640, 3 };
// convert the cv::Mat to numpy.array
PyObject* mat = PyArray_SimpleNewFromData(3, mdim, NPY_UINT8, (void*) m);
// create a Python-tuple of arguments for the function call
// "()" means "tuple". "O" means "object"
PyObject* args = Py_BuildValue("(O)", mat);
// execute the function
PyObject* result = PyEval_CallObject(m_PyFooBar, args);
// process the result
// ...
// decrement the object references
Py_XDECREF(mat);
Py_XDECREF(result);
Py_XDECREF(args);
delete[] m;
}
}
else
{
std::cerr << "Failed to load the Python module!" << std::endl;
PyErr_Print();
}
return 0;
}
import cv2
import numpy as np
# define the function that we will call from the C++ code
def foo_bar(img=None):
if img is not None:
cv2.imshow("image in python", img)
@omeryasar
Copy link

Hi, First of all thanks for sharing this code. But I don't understand with which arguments we should call this code. And PyObject *sys_path = PySys_GetObject("path"); how this part is working. Can you please explain a little bit. Thanks in advance

@Xonxt
Copy link
Author

Xonxt commented Dec 10, 2020

Hi, First of all thanks for sharing this code. But I don't understand with which arguments we should call this code. And PyObject *sys_path = PySys_GetObject("path"); how this part is working. Can you please explain a little bit. Thanks in advance

Hi, so in this case this line: PySys_GetObject("path") basically returns a path to the C++ file that is currently running, and from which you're running the python code.

For example, you have a main.cpp in D:\Projects\PythonApp\code and you have some python code in D:\Projects\PythonApp\code\pycode. Then in your main.cpp you need to do:

PyObject *sys_path = PySys_GetObject("path");
PyList_Append(sys_path, PyUnicode_FromString("./pycode"));  

Apart from that, you also need to add the C:\Program Files\Python3X\include and C:\Program Files\PythonXX\Lib\site-packages\numpy\core\include to your include directories in your project, and also link your C:\Program Files\Python3X\libs\python3X.lib. Here, "X" is your Python version, in my case, "Python36".

Hope that helps.

@omeryasar
Copy link

Hi, First of all thanks for sharing this code. But I don't understand with which arguments we should call this code. And PyObject *sys_path = PySys_GetObject("path"); how this part is working. Can you please explain a little bit. Thanks in advance

Hi, so in this case this line: PySys_GetObject("path") basically returns a path to the C++ file that is currently running, and from which you're running the python code.

For example, you have a main.cpp in D:\Projects\PythonApp\code and you have some python code in D:\Projects\PythonApp\code\pycode. Then in your main.cpp you need to do:

PyObject *sys_path = PySys_GetObject("path");
PyList_Append(sys_path, PyUnicode_FromString("./pycode"));  

Apart from that, you also need to add the C:\Program Files\Python3X\include and C:\Program Files\PythonXX\Lib\site-packages\numpy\core\include to your include directories in your project, and also link your C:\Program Files\Python3X\libs\python3X.lib. Here, "X" is your Python version, in my case, "Python36".

Hope that helps.

Thanks for reply, finally may i ask PySys_SetArgv(argc, (wchar_t**)argv); this part of code.

@Xonxt
Copy link
Author

Xonxt commented Dec 10, 2020

Thanks for reply, finally may i ask PySys_SetArgv(argc, (wchar_t**)argv); this part of code.

oh yeah, that's just passing the command line arguments from here int main(int argc, char** argv) into your Python interpreter.

@omeryasar
Copy link

Thanks for reply, finally may i ask PySys_SetArgv(argc, (wchar_t**)argv); this part of code.

oh yeah, that's just passing the command line arguments from here int main(int argc, char** argv) into your Python interpreter.

Thanks a lot again. Your solution make my work easier.

@omeryasar
Copy link

I have working on your solution for a while to integrate c++ project with a python project for object detection. I need to call python project with certain string parameters but I can't manage to do that. Any reccomendation ?

@Xonxt
Copy link
Author

Xonxt commented Dec 15, 2020

I have working on your solution for a while to integrate c++ project with a python project for object detection. I need to call python project with certain string parameters but I can't manage to do that. Any reccomendation ?

Do you mean, pass string parameters through the command line? Or actually call a function with string-type arguments?

@omeryasar
Copy link

omeryasar commented Dec 15, 2020

Do you mean, pass string parameters through the command line? Or actually call a function with string-type arguments?

Both works actually but I need to send an image as parameter so I think I need call a function with both with string and mat object.

@Xonxt
Copy link
Author

Xonxt commented Dec 15, 2020

Both works actually but I need to send an image as parameter so I think I need call a function with both with string and mat object.

Then please look at the first gist, lines 73-75. I'm giving an example with an integer, but you just need to replace the i with a s for a string-type. There's also a link for more explanations for different types.

Basically, you'll do something like:

PyObject* args = Py_BuildValue("(Os)", image, "your_string_here");
      
// call your function
PyObject* result = PyEval_CallObject(m_PyFooBar, args);

@omeryasar
Copy link

This is a bit irrelative to your gist but I wonder is it possible to call more than one python function and suppose that one of the python functions update some global variable in python code and the one that we called later use the updated global variable. I search it but actually I can't find much about it. Thanks in advance

@Xonxt
Copy link
Author

Xonxt commented Dec 16, 2020

This is a bit irrelative to your gist but I wonder is it possible to call more than one python function and suppose that one of the python functions update some global variable in python code and the one that we called later use the updated global variable. I search it but actually I can't find much about it. Thanks in advance

Calling multiple functions is of course possible.

The PyObject *m_PyDict object in the code actually gathers the full list of all objects from the python code through PyModule_GetDict('MODULE') (similar to calling from MODULE import * in Python).

Then you just add more function objects, e.g.:

PyObject *m_PyFoo1, *m_PyFoo2, *m_PyFoo3, *m_PyFoo4;

And initialise them using PyDict_GetItemString(m_PyDict, "function_name"), e.g.:

m_PyFoo1 = PyDict_GetItemString(m_PyDict, "function_name_1");
m_PyFoo2 = PyDict_GetItemString(m_PyDict, "function_name_2");
...

As for the global variable.. I'm not sure, but I assume since you've loaded the python module using PyImport_ImportModule anyway, global variables should work too, I believe.
Just try it: create two functions in Python, both using a global variable, and try to call them from C++ and see if it works,

@poreocook
Copy link

Good day. Thank you for this post/codes, I understood it well and planning to use this as a guide. If you have time, I would like to ask if how can you display a returning value from python function that we called. for example I sent an image to python then processed it then return it into c++, how would i display it? Thank you again for this.

@Xonxt
Copy link
Author

Xonxt commented Feb 4, 2021

Good day. Thank you for this post/codes, I understood it well and planning to use this as a guide. If you have time, I would like to ask if how can you display a returning value from python function that we called. for example I sent an image to python then processed it then return it into c++, how would i display it? Thank you again for this.

Hi, thank you for your interest. I haven't used Python in C in a long time, so I'm not sure, but I guess you have two options:

  1. Either you add the outputImage as another argument for your Python function, and save the result into that.

  2. Or, you do something similar to what I've done in one of my applications.
    Here I have a python function, that returns the following:

return {"Right": np.array(...), "Left": np.array(...)}

Then in my C application I read it like this:

result = PyEval_CallObject( m_PyFunction, args );

// this contains the keys: "Right" and "Left"
PyObject* handDictKeys = PyDict_Keys( result );

// this is the number of keys:
Py_ssize_t numKeys = PyList_Size( handDictKeys );

// temporary container for the value in dict, should be an array
PyObject *obj = NULL;

// temporary container for the key value (strings "Right" and "Left")
char* key = NULL;

for ( Py_ssize_t k = 0; k < numKeys; k++ )
{
  // entry in the dict
  obj = PyList_GetItem( handDictKeys, k );

  // get the KEY of the entry
  key = PyUnicode_AsUTF8( PyObject_Repr( obj ) );

  // we'll be saving the hand-points into this vector:
  std::vector<cv::Point3d> handPoints;

  // get a pointer to the array
  PyObject* handPointArrayObj = PyDict_GetItem( result, obj );
  PyArrayObject* handPointArray = reinterpret_cast<PyArrayObject*>(handPointArrayObj);

  // array size:
  npy_intp* arrDims = PyArray_SHAPE( handPointArray );
  int nDims = PyArray_NDIM( handPointArray );

  double* arrPtr = reinterpret_cast<double*>(PyArray_DATA( handPointArray ));
}

And at the end, the arrPtr is essentially a returned NumPy array. I'm sure you can try to adapt this to return an image from Python, which would also probably be a Numpy array.

@poreocook
Copy link

Good day. Thank you for this post/codes, I understood it well and planning to use this as a guide. If you have time, I would like to ask if how can you display a returning value from python function that we called. for example I sent an image to python then processed it then return it into c++, how would i display it? Thank you again for this.

Hi, thank you for your interest. I haven't used Python in C in a long time, so I'm not sure, but I guess you have two options:

  1. Either you add the outputImage as another argument for your Python function, and save the result into that.
  2. Or, you do something similar to what I've done in one of my applications.
    Here I have a python function, that returns the following:
return {"Right": np.array(...), "Left": np.array(...)}

Then in my C application I read it like this:

result = PyEval_CallObject( m_PyFunction, args );

// this contains the keys: "Right" and "Left"
PyObject* handDictKeys = PyDict_Keys( result );

// this is the number of keys:
Py_ssize_t numKeys = PyList_Size( handDictKeys );

// temporary container for the value in dict, should be an array
PyObject *obj = NULL;

// temporary container for the key value (strings "Right" and "Left")
char* key = NULL;

for ( Py_ssize_t k = 0; k < numKeys; k++ )
{
  // entry in the dict
  obj = PyList_GetItem( handDictKeys, k );

  // get the KEY of the entry
  key = PyUnicode_AsUTF8( PyObject_Repr( obj ) );

  // we'll be saving the hand-points into this vector:
  std::vector<cv::Point3d> handPoints;

  // get a pointer to the array
  PyObject* handPointArrayObj = PyDict_GetItem( result, obj );
  PyArrayObject* handPointArray = reinterpret_cast<PyArrayObject*>(handPointArrayObj);

  // array size:
  npy_intp* arrDims = PyArray_SHAPE( handPointArray );
  int nDims = PyArray_NDIM( handPointArray );

  double* arrPtr = reinterpret_cast<double*>(PyArray_DATA( handPointArray ));
}

And at the end, the arrPtr is essentially a returned NumPy array. I'm sure you can try to adapt this to return an image from Python, which would also probably be a Numpy array.

Thank you for this response. I will try this and see if i get it right.

If i get it right, I'm planning on focusing more on this part.

  // execute the function
  PyObject* result = PyEval_CallObject(m_PyFooBar, args);
  
  // process the result
  // ...

the variable result holds the returning value from the python function m_PyFoobar given the arguments args. So what ever the returning value type of python will be the case for the result variable. Meaning, if i retuned something like an ndarray from python i should get an ndarray in c++? btw Im still a student and new to python embedding in c++ and want to know more about this topic. and i thank you because you just added a ton of knowledge. Thank you!

@Xonxt
Copy link
Author

Xonxt commented Feb 4, 2021

the variable result holds the returning value from the python function m_PyFoobar given the arguments args. So what ever the returning value type of python will be the case for the result variable. Meaning, if i retuned something like an ndarray from python i should get an ndarray in c++? btw Im still a student and new to python embedding in c++ and want to know more about this topic. and i thank you because you just added a ton of knowledge. Thank you!

yeah, most likely you'll just need to take the last few lines from my latest example, something like:

// cast the generic Python object (returned from the function) as a Python ARRAY object
PyArrayObject* your_numpy_array = reinterpret_cast<PyArrayObject*>( result );

// get array size (dimensions):
npy_intp* arrDims = PyArray_SHAPE( your_numpy_array );
int nDims = PyArray_NDIM( handPointArray ); // number of dimensions

// convert the PyArrayObject into a C-style array pointer (replace 'double' with your type)
double* array_pointer = reinterpret_cast<double*>( PyArray_DATA( your_numpy_array ) );

@SabraHashemi
Copy link

@Xonxt
i embeded my script function in c++, and pass cv::mat to it ang get result back
but
my script (without embedding) just take 30 ms
but when i use it in embedding mode and send cv::mat from c++ to python it takes about 80-130 ms

why and how can i solve this problem?

@SabraHashemi
Copy link

for more details:
PyObject_CallObject(pFunc, pArgs); takes 130 ms
but when i check time of my function in python (that im called) it is about 30 ms

@Xonxt
Copy link
Author

Xonxt commented Apr 28, 2021

for more details:
PyObject_CallObject(pFunc, pArgs); takes 130 ms
but when i check time of my function in python (that im called) it is about 30 ms

Hi. I'm not sure, since I haven't worked with C++/Python-communication since I'd first posted this gist, but I suspect this is an unavoidable "overhead" that you get when trying to communicate between those two.
An obvious solution would be to just switch to working purely in Python (which is what I did), or purely in C++, or maybe put your C++ code into a library and write a proper Python-wrapper for it.

@IricsDo
Copy link

IricsDo commented Oct 24, 2021

Thanks Guys Xonxt for this script. It's very helpfull. I run this code on Visual Studio 2017 version 15.9.40 and i don't meet any problem.

But I meet the problem: ModuleNotFoundError: No module named 'numpy'. This problem occur when i using Visual Studio 2019 version 16.11.5. The line code import_array1(-1) is the reason for that.
Can you help me? I don't know why MSVC 2019 imcompatible with numpy API of Python.
thanks for your reading !

@Xonxt
Copy link
Author

Xonxt commented Oct 24, 2021

Thanks Guys Xonxt for this script. It's very helpfull. I run this code on Visual Studio 2017 version 15.9.40 and i don't meet any problem.

But I meet the problem: ModuleNotFoundError: No module named 'numpy'. This problem occur when i using Visual Studio 2019 version 16.11.5. The line code import_array1(-1) is the reason for that. Can you help me? I don't know why MSVC 2019 imcompatible with numpy API of Python. thanks for your reading !

Did you install the 'numpy' module in your Python and import the C:\Program Files\PythonXX\Lib\site-packages\numpy\core\include into your MSVC project?
If that doesn't work, then I don't know, sorry. I haven't worked with this stuff for a while. Ever since I posted this to be exact :D.

@IricsDo
Copy link

IricsDo commented Oct 24, 2021

Yes. I already installed numpy package in python. That why i can run your code in VS 2017. Now i want run your code in VS 2019 but i have the problem.
Thanks you response for me.

@IricsDo
Copy link

IricsDo commented Oct 25, 2021

Yes. I already installed numpy package in python. That why i can run your code in VS 2017. Now i want run your code in VS 2019 but i have the problem. Thanks you response for me.

I have more information, now i return VS 2017 and this code have the same problem :)) . I install two Visual Studio together 2017 and 2019. First 2017 and second 2019.

@SabraHashemi
Copy link

SabraHashemi commented Oct 25, 2021

https://gist.github.com/Xonxt/26d2a9ac6c56505d0896822ede99a646#gistcomment-3723333

I tested it again and I think it didn't have any overhead anymore. thanks for the reply.
@Xonxt

@IricsDo
Copy link

IricsDo commented Mar 18, 2022

Hi guys, long time no reply :)
I have a new question for you. How i can choose enviromnent for python? It's mean when i call python from c++ script? How to choose env to run python script.
Example: I using ubuntu 18.04 OS . I have 2 virtual environment ( anaconda and venv) and 1 real environment python3.6.9 default of ubuntu (python default). How to i know what env i run python code when call from c++ and how to change it? is it based on python.h (python.a in linux). I think this is similar on window with venv and anaconda and python when install from offcial web site of python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment