Ask Your Question
1

PYTHON C++ SWIG share image pixel data in memory

asked 2018-04-19 17:36:18 -0500

wlei gravatar image

updated 2018-04-25 15:32:40 -0500

Hello,

I am working on passing image pixel data from c++ app to python and vice versa. The use case is user creates an image in the c++ application and then retrieve the image (preferably from memory so that we do not need to copy back and forth) and do some image process in python (using python image processing library such as PIL, opencv etc). Once image processing is done in python, we would want to pass the image back to the c++ application (preferably from memory as well, so that we do not need to copy the image data).

So, my question is how to get the image pixel data from python? I can pass the void* that points to the pixel data as well as the width and height information from the C++ application (with SWIG).

And how do we pass the image pixel data from python back to the c++ app (again prefer to just refer it to the pixel data in memory)?

My application is written in c++ and I am using SWIG to interface with python.

Thanks.

edit retag flag offensive close merge delete

Comments

Is your C++ going into a DLL? Have you tried it just to see if it works, before you get into SWIG?

sjhalayka gravatar imagesjhalayka ( 2018-04-20 12:08:31 -0500 )edit
1

Yes, the c++ is a dll. In fact it is now a .pyd since I am using SWIG. I can pass the void* (image data is unsigned char in memory) to python. But I cannot seem to find a library to construct the data from memory (in terms of void* address) to create an image (given that from python, I know the rows and column of the image). I am still exploring how to do it using numpy/opencv/PILLOW.

Do you know how to do it without swig?

wlei gravatar imagewlei ( 2018-04-20 12:24:22 -0500 )edit
1

This code sets up a new Mat from a void* pointer (called ptr):

Mat frame_content = Mat(480, 640, CV_16UC1, ptr).clone();

Does that help?

sjhalayka gravatar imagesjhalayka ( 2018-04-20 15:32:58 -0500 )edit

Thanks, but Mat is not available in cv2, right? I think I need a function in python (openCV or numpy or which ever image processing library that can create the image back from the block of memory (void * from c++).

wlei gravatar imagewlei ( 2018-04-20 15:39:50 -0500 )edit

I'm not sure what OpenCV 2.x's capabilities are. Sorry about that.

sjhalayka gravatar imagesjhalayka ( 2018-04-20 15:42:27 -0500 )edit
1

... so you're looking to pass the image data from C++ to Python, right?

sjhalayka gravatar imagesjhalayka ( 2018-04-20 15:53:10 -0500 )edit

Thanks. Yes, both ways from C++ to python (get image from c++ app, use the image in python to do some image processing and let the C++ app pick up the latest image data) and then from python back to C++.

So, if we use the Mat mechanism, how do we expose it to python to pick it up to do processing?

wlei gravatar imagewlei ( 2018-04-20 16:40:27 -0500 )edit

That's a good question, and I'm going to look into it. Do you have any source code that I can work from? I use GitHub for sharing code.

sjhalayka gravatar imagesjhalayka ( 2018-04-20 16:52:45 -0500 )edit
1

I am using python 3.6 for 64 bit. I haven't tried to build c++ dll with OpenCV yet. Actually, right now, my cpp code does not use opencv yet. I am still finding way to expose the data to python. Looks like there is a python buffer protocol that I can make use of... I still need to explore it... have you use it?

So far, I am using opencv on the python side to try to pick up the data from memory address... no luck so far...

sorry, my source code is linking all over the places...too much dependencies that will require a lot of time to set up... thanks though!

wlei gravatar imagewlei ( 2018-04-20 17:26:31 -0500 )edit

Do you have a working example of a void** parameter function?

sjhalayka gravatar imagesjhalayka ( 2018-04-21 15:36:47 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
2

answered 2018-04-22 21:04:43 -0500

sjhalayka gravatar image

updated 2018-04-23 10:18:19 -0500

I exported a DLL function that returns a char* array. The Python code calls the function and converts the bytes into a 2D numpy array, and back again to bytes:

Input image:

image description

Windows code:

#include <opencv2/opencv.hpp>
using namespace cv;
#pragma comment(lib, "opencv_world331.lib")

#include <iostream>
#include <iomanip>
#include <string>
#include <map>
#include <mutex>
using namespace std;


#define DLLEXPORT extern "C" __declspec(dllexport)


map<size_t, char*> ids;
mutex rw_ids;

DLLEXPORT int get_img_from_char_array(char *c, int *width, int *height)
{
    Mat frame_content(*height, *width, CV_8UC1, c);

    return 0;
}


DLLEXPORT char *alloc_img(const char *file_name, int *width, int *height, size_t *image_id)
{
    rw_ids.lock();

    Mat img = imread(file_name, IMREAD_GRAYSCALE);

    if (img.empty())
    {
        *width = 0;
        *height = 0;

        rw_ids.unlock();

        return nullptr;
    }

    *width = img.cols;
    *height = img.rows;

    size_t num_pixels = (*width)*(*height);

    char *p = new char[num_pixels];
    memcpy(p, img.data, num_pixels*sizeof(char));

    ids[ids.size()] = p;
    *image_id = ids.size() - 1;

    cout << "Created img id " << *image_id << endl;

    rw_ids.unlock();

    return p;
}

DLLEXPORT int free_img(size_t *image_id)
{
    rw_ids.lock();

    for (map<size_t, char*>::const_iterator ci = ids.begin(); ci != ids.end(); ci++)
    {
        if (ci->first == *image_id)
        {
            cout << "Deleted img id " << *image_id << endl;

            if (nullptr != ci->second)
                delete[] ci->second;

            ids.erase(ci);

            rw_ids.unlock();

            return 0;
        }
    }

    rw_ids.unlock();

    return 1;
}

Python code:

import ctypes as ct
import numpy as np
import cv2

ptr = ct.c_char_p()
s = "dove.png"
width = ct.pointer(ct.c_int())
height = ct.pointer(ct.c_int())
img_id = ct.pointer(ct.c_size_t())
lib = ct.CDLL("void_dll.dll")
alloc_img = lib.alloc_img
free_img = lib.free_img
get_img_from_char_array = lib.get_img_from_char_array

alloc_img.restype = ct.c_char_p

bytes_array = alloc_img(ct.c_char_p(s.encode("utf-8")), width, height, img_id)
numpy_array = np.frombuffer(bytes_array, dtype=np.uint8)
numpy_array.shape = (width.contents.value, height.contents.value)

cppbytes = numpy_array.tobytes()
get_img_from_char_array(cppbytes, width, height);

free_img(img_id)

I also have some extra code handy, which uses a void** pointer to allocate memory:

int func(void **ptr)
{
    float *p = new float[50];
    p[0] = 123.456;
    *ptr = p;

    return 0;
}

int main(void)
{
    float *p = 0;
    func(reinterpret_cast<void**>(&p));

    cout << p[0] << endl;

    delete[] p;

    return 0;
}
edit flag offensive delete link more

Comments

2

Thanks for the detailed information.

I was able to get it working with Python buffer and memory view from the C++ side. In case you are interested. Here is the code:

PyObject* :GetBufferPtr(Image img, long width, long height) { ..... Py_buffer pybuffer; //where buf points to a void* for image pixel int res = PyBuffer_FillInfo(&pybuffer, 0, buf, val, false, PyBUF_CONTIG); .. return PyMemoryView_FromBuffer(&pybuffer); }

On the Python side, I was able to retrieve the image pixel using numpy.frombuffer() function: np_array = np.frombuffer(imgPtr, dtype=np.uint8) np_array.shape = (rows, cols) //where rows and cols are the width and height of the image

The above function can be used with or without SWIG...

wlei gravatar imagewlei ( 2018-04-23 15:05:47 -0500 )edit
1

Right on. Glad to hear you got it working.

If you like my answer, then please mark it as correct.

sjhalayka gravatar imagesjhalayka ( 2018-04-23 15:21:32 -0500 )edit
1

Thank you.

Now, the other part is how to write to the shared memory from python side. I am able retrieve the numpy array and creates/displays the image. Now, if I do some processing on the image (say flip the image 90 degree) and I want to save the new image data to the share memory, how do I do that in python?

sorry, I am pretty new to this forum, I will have to figure out how to mark the answer as correct...

wlei gravatar imagewlei ( 2018-04-24 09:59:50 -0500 )edit
1

I also put my code up at https://github.com/sjhalayka/python_c...

In the C++ file void_dll.cpp, there is a function get_img_from_char_array() that takes a Python byte array and converts it into a Mat object. The void_dll.py file calls the function from the DLL.

Perhaps this may be inspirational to you.

sjhalayka gravatar imagesjhalayka ( 2018-04-24 10:58:45 -0500 )edit
1

There is a check mark on the left of the top of my answer. There are also an up arrow and down arrow, so you could vote me up that way too, if you like.

And thank you for the inspiration to write some code.

sjhalayka gravatar imagesjhalayka ( 2018-04-24 11:04:14 -0500 )edit

Which version of OpenCV are you using?

sjhalayka gravatar imagesjhalayka ( 2018-04-24 11:38:48 -0500 )edit
1

I am using the latest opencv (3.4.1)

The get_img_from_char_array does not help as I was looking at the code on python side to update the memory directly (that was shared by the c++ app) so that the c++ app just need to do a refresh to pick up the change from the python (i.e. c++ app should not need any new code to achieve this as only the memory content is changed so it should still be able to update data pointed by the void* that was shared with the python buffer).

I was able to update the buffer on the python side and display the image using the latest change in the buffer. But, for some unknown reason, when c++ pick up the change, only part of the memory is updated.

Here is the code on the python side to update the buffer (pls. see next comment):

wlei gravatar imagewlei ( 2018-04-24 13:34:17 -0500 )edit
1

import myLib import cv2 import numpy as np imgPtr = myLib.GetImgBuffer(img1, cols1, rows1) numpy_array = np.frombuffer(imgPtr, dtype=np.uint8) numpy_array.shape = (rows1, cols1)

//assume: rows1 = cols1 = 1024
//change the image in python
dst = np.ones((1024,1024), dtype=np.uint8)

//update data in the buffer
np.copyto(numpy_array, dst) //IS THIS CORRECT FOR UPDATING THE MEMORY IN THE BUFFER??
numpy_array.shape = dst.shape

 //create another numpy array from the updated buffer
numpy_array2 = np.frombuffer(imgPtr, dtype=np.uint8)
numpy_array2.shape = numpy_array.shape // this array display the correct change but when we pick up 
//the change in c++, only ~ quarter of the image is changed)
cv2.imshow('img - retrieve image from updated buffer & display in Python', numpy_array2)
cv2.waitKey(0)
wlei gravatar imagewlei ( 2018-04-24 13:44:05 -0500 )edit
1

Can you please paste your code into a Gist on Github?

sjhalayka gravatar imagesjhalayka ( 2018-04-24 14:05:18 -0500 )edit
1
wlei gravatar imagewlei ( 2018-04-24 14:57:10 -0500 )edit
0

answered 2018-04-25 13:21:01 -0500

wlei gravatar image

updated 2018-04-25 14:13:37 -0500

sjhalayka gravatar image

To expose the shared python buffer from C++ side:

PyObject* :GetBufferPtr(Image img, long width, long height) 
{ .....
  Py_buffer pybuffer; //where buf points to a void* for image pixel 
  int res = PyBuffer_FillInfo(&pybuffer, 0, buf, val, false, PyBUF_CONTIG); 
  .. 
  return PyMemoryView_FromBuffer(&pybuffer); 
}

To retrieve image buffer from Python's side and update the buffer from Python side:

import myLib
import cv2 
import numpy as np 
imgPtr = myLib.GetImgBuffer(img1, cols1, rows1) 
numpy_array = np.frombuffer(imgPtr, dtype=np.uint8)  //for one byte integer image
numpy_array.shape = (rows1, cols1)
//change the image in python
dst = np.ones((rows1,cols1), dtype=np.uint8)

//update data in the buffer
np.copyto(numpy_array, dst) //update the buffer
numpy_array.shape = dst.shape

**Now, if you refresh your display in the C++ app, you will see the updated change from Python

Have fun coding!

edit flag offensive delete link more

Comments

I edited your answer to codify the code. I also rated you up!

sjhalayka gravatar imagesjhalayka ( 2018-04-25 14:14:02 -0500 )edit

So what does SWIG do???

sjhalayka gravatar imagesjhalayka ( 2018-04-25 14:23:12 -0500 )edit
1

In this case, we don't really need SWIG to do the interfacing since we already return a PyObject*. But, if one is already using SWIG, it will still work with the .i (interface) file.

Thanks for your rating!

wlei gravatar imagewlei ( 2018-04-25 15:08:54 -0500 )edit
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2018-04-19 17:36:18 -0500

Seen: 1,275 times

Last updated: Apr 25 '18