Ask Your Question

how to return a numpy array to python from c++ (shared memory)

asked 2018-05-01 10:52:26 -0500

wlei gravatar image

Is there a way to return a numpy array from c++ to python (shared memory)? This is for sharing image data between a c++ app and python. I am using SWIG currently.

I was able to shared the memory between c++ and python (for modifying image data in both end). Please see

Now, instead of letting user to retrieve the shared data from memory and construct the right shape for the numpy array, I would like to package that so that it will just return the numpy array (shared memory) and let user manipulates it in python?

numpy_array = np.frombuffer(imgPtr, dtype=np.uint8) //for one byte integer image numpy_array.shape = (rows1, cols1)

So in summary, I would like to package the above two lines of code in my c++ app and provide a function call in python to retrieve the numpy array. This is more user friendly since the user that uses python app are not professional sw developer, so asking them to get the data from shared memory and reshape the numpy array before they can use the numpy array to visualize it in opencv is a little bit too much for them.

Thanks in advamce/

edit retag flag offensive close merge delete



That's bull**. Anyone can learn to code C++, because it's natural.

sjhalayka gravatar imagesjhalayka ( 2018-05-01 11:09:35 -0500 )edit

Unfortunately, my users do not have interest or time to learn c++. I wish you are my target user :-). Anyway, thanks for your comment.

wlei gravatar imagewlei ( 2018-05-01 12:20:48 -0500 )edit

Is this not where you need a function like DLLEXPORT char *alloc_img(const char *file_name, int *width, int *height, size_t *image_id)?

.. and also a function like... DLLEXPORT int free_img(size_t *image_id)

... and also the Python code:

width = ct.pointer(ct.c_int())
height = ct.pointer(ct.c_int())
img_id = ct.pointer(ct.c_size_t())
s = "dove.png"
lib = ct.CDLL("void_dll.dll")
alloc_img = lib.alloc_img
free_img = lib.free_img
alloc_img.restype = ct.c_char_p

bytes_array = alloc_img(ct.c_char_p(s.encode("utf-8")), width, height, img_id)
numpy_array = np.frombuffer(bytes_array, dtype=np.uint8)
sjhalayka gravatar imagesjhalayka ( 2018-05-01 12:27:51 -0500 )edit

I think you must have misunderstood my question.

Instead of letting user retrieve the shared memory pointer, create a numpy array and shape it to the correct dimemsion, I would like to let the user achieve that by a single function call from C++ app to retrieve the numpy array (i.e. without performing the following code line 2 to 3):

Original solution:

import myLib;
imgPtr = myLib.GetImgBuffer(img1, cols1, rows1) 
numpy_array = np.frombuffer(imgPtr, dtype=np.uint8)
numpy_array.shape = (rows1, cols1)


New Solution:

import myLib
retreiveNPArray = myLib.GetNumpyArray();

--> this retrieves the numpy array created in c++ with the correct shape (and is using shared memory with Python, so any python related updated on this array will be reflected in c++ automatically)

wlei gravatar imagewlei ( 2018-05-01 12:57:23 -0500 )edit

The original solution requires user to understand the programming basics of retrieving the buffer pointer, create a numpy array from buffer pointer and then shape the numpy array accordingly. For my users, they will ask me "Why do I have to do all that? This is too complicated. Why can't you just give me the numpy array in Python for the displayed image on C++ app?"

wlei gravatar imagewlei ( 2018-05-01 13:01:03 -0500 )edit

So, you've figured it out? That's good!

Sorry, yes I do believe that I have misunderstood you.

sjhalayka gravatar imagesjhalayka ( 2018-05-01 13:28:43 -0500 )edit

no, I am still looking for a solution...I am still researching how to implement GetNumpyArray() on the c++ side...

wlei gravatar imagewlei ( 2018-05-01 15:46:38 -0500 )edit

I think I can use PyArray_SimpleNewFromData()... but I got access violation error with the following code:

assuming it is a 512x512 int8 image and buf is the:

 void pointer* (image data)
            PyObject* Py_Image::GetNumArray()

                npy_intp m[1] = { 512 };
                PyObject* pNdArray = PyArray_SimpleNewFromData(512, m, NPY_INT, buf); <----access violation error
                return pNdArray;

Am I not calling PyArray_SimpleNewFromData() properly?

wlei gravatar imagewlei ( 2018-05-01 17:01:28 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2018-05-03 17:30:39 -0500

wlei gravatar image

... import_array(); npy_intp size[2] = { height, width }; int type = GetNPEnumTypeForImage(); PyObject* pNdArray = PyArray_SimpleNewFromData(2, size, type, buf); ...

edit flag offensive delete link more


Can you contribute a GitHub repository full of the C++ / Python code needed to allocate and use the shared memory?

sjhalayka gravatar imagesjhalayka ( 2018-05-03 19:27:14 -0500 )edit

Question Tools

1 follower


Asked: 2018-05-01 10:52:26 -0500

Seen: 2,600 times

Last updated: May 03 '18