OpenCV Error [Segmentation fault]: Converting data from OpenGL to OpenCV Matrix.
Hello, I stumbled across this API which captures images directly and converts the camera feed (raw YUV data) to RGBA and does this on the GPU. This is beneficial because I'm using my RPi and it doesn't have a lot of processing power, so I want a live camera feed which can be directly seen on screen for instance, and then the CPU will continuously process the data in the background.
The API lacks documentation and is very unclear on how to use with OpenCV, it is simple and efficient and is just want I need, so it really is quiet helpful if I can get it working. So this is the code I've tried this to convert the camera feed to a cv::Mat object with no success.
const void* frame_data; int frame_sz;
if(cam->BeginReadFrame(1,frame_data,frame_sz))
{
//if doing argb conversion the frame data will be exactly the right size so just set directly
textures[1].SetPixels(frame_data);
cam->EndReadFrame(1);
}
//convert the frame data to a cv::Mat object.
cv::Mat TempMat = cv::Mat(MAIN_TEXTURE_HEIGHT, MAIN_TEXTURE_WIDTH, CV_8UC1, &frame_data,frame_sz);
imshow("Camera Feed",TempMat);
Of course, this is just a small segment from the code, but it contains a pointer (cam) which refers to a class which contains these functions:
bool BeginReadFrame(int level, const void* &out_buffer, int& out_buffer_size);
void EndReadFrame(int level);
And the BeginReadFrame function basically make a call to some MMAL routines and reads from the camera output and converts it into RGBA format and the other function releases the data buffer back to the pool from whence it came.
So the API is pretty much a wrapper for the relatively complex library MMAL to make stuff a lot easier.
On the other hand, converting this frame data to a OpenCV Image is not shown how. I believe it is quiet simple, but I'm not very experienced in neither OpenCV or OpenGL.
I encourage you to check out the API to get a better idea of what it does, and how it works.
The Segmentation fault error is just printed in the Console and no other information is provided, it then terminates the program.
Thank you very much in advance!
using CV_8UC1 assumes, your image is 1 channel, 8bit grayscale. is it so ?
(you mention RGBA in another place, CV_8UC4 would be appropriate, then, CV_8UC1 does not alloc enough memory for your image)
also, your frame_data pointer will go out of scope, thus you need to clone() the Mat to get a 'deep' copy of the pixels.
A few suggestions... Did you read the docs on cv::Mat constructor? Should it not be frame_data instead of &frame_data on the cv::Mat constructor line. also the last argument frame_sz is wrong because it is the size of the image not of the data step size. You can leave out the last step argument if the data is consecutive.
@boaz001 Thank you, I did read that but found it a bit hard to understand, the problem is when I try to use frame_data instead of &frame_data on the cv:Mat constructor line, it gives me the error: invalid conversion from ´const void´ to ´void´ [-fpermissive]. Any ideas?
@berak Thank you, I now changed the third parameter to CV_8UC4, it is in fact ARGB from what I can understand and what I've read on here. It doesn't make much difference though, I'm not sure if you want me to just simply put
TempMat.clone()after the Mat constructor call or something else, it gives me the same error without any other information (i.e Segmentation fault). Is it correct to use &frame_data? To refer to it as a pointer, it gives me a compiling error if I don't (see my above comment to @boaz001).It must be a pointer and cv::Mat does not guarantee it's const-ness. If you dereference a void pointer what data type of what size do you think you will get? Anyway, a solution could be to copy the data from the frame_data to (a non-const data type).
char* frame_data_copy = new[frame_sz]; // create a buffer of same size in bytes
std::memcpy(frame_data_copy, frame_data, frame_sz); // copy data from frame_data to frame_data_copy
cv::Mat TempMat(MAIN_TEXTURE_HEIGHT, MAIN_TEXTURE_WIDTH, CV_8UC4, frame_data_copy); //create cv::Mat
delete [] frame_data_copy; //free allocated memory for copy, this also invalidates the data of TempMat so that's why @berak suggests to take a clone. (It would also be invalidated if frame_data runs out of scope).
also, you should take a close look, at that api, if frame_data is even valid after cam->EndReadFrame(1);
@berak didn't see that one coming ;)
@berak Okay, I have now placed the code within the IF statement so cam->EndReadFrame(1); Is called afterwards. @boaz001 Thanks it didn't give me any compile errors, but still the segmentation fault remains, maybe there is something else about my code that is wrong? You can take a look here, and if it doesn't make sense you may want to look at the source code of the API, particularly graphics.cpp and camera.cpp.