Ask Your Question
0

(u->refcount == 0) in deallocate

asked 2018-12-11 11:05:39 -0500

EZenderink gravatar image

updated 2018-12-11 11:19:46 -0500

Hi,

I am having an issue that only occurs on Linux (arm) on my Raspberry Pi 3b. It works fine on windows.

I am trying to read a video file using this loop:

while (videoCapture.isOpened()) {

    DebugWrite("Start Grabbing Frame from VIDEO stream.", "ImageGrabber::RunGrabber");
    _shouldStop.mx.lock();
    if (_shouldStop.stop) {
        break;
    }
    _shouldStop.mx.unlock();

    DebugWrite("Initiating MAT", "ImageGrabber::RunGrabber");
    cv::Mat frame;

    DebugWrite("Finished Initiating MAT, reading frame!", "ImageGrabber::RunGrabber");
    if (videoCapture.read(frame)) {
        DebugWrite("Frame written.", "ImageGrabber::RunGrabber");
        if (cv::waitKey(30) >= 0 || frame.empty()) break;

        DebugWrite("Copying frame to UMAT.", "ImageGrabber::RunGrabber");
        _frameBuffer.mx.lock();
        frame.copyTo(*_frameBuffer.writeFrame);
        _frameBuffer.notempty = true;
        DebugWrite("Frame copied to UMAT.", "ImageGrabber::RunGrabber");
        _frameBuffer.mx.unlock();

        DebugWrite("Videocapture (" + _imageSource.source + ").", "ImageGrabber::RunGrabber", frame);
    }
    else {
        DebugWrite("Could not read video.", "ImageGrabber::RunGrabber");
        break;
    }


}

It read the first frame fine. But when I want to read the second frame from the video file it throws this error:

SmartPause::DetectionThread | Not Stopping Detection Thread
SmartPause::DetectionThread | Retreiving Frame
SmartPause::DetectionThread | Not Stopping Detection Thread
ImageGrabber::RunGrabber | Copying frame to UMAT.SmartPause::DetectionThread | Retreiving Frame

SmartPause::DetectionThread | Not Stopping Detection Thread
SmartPause::DetectionThread | Retreiving Frame
[ INFO:0] Initialize OpenCL runtime...
ImageGrabber::RunGrabber | Frame copied to UMAT.
ImageGrabber::RunGrabber | Videocapture (/home/pi/test.mp4).
SmartPause::DetectionThread | Reading frame from buf0
SmartPause::DetectionThread | Image To Process
ImagePreProcessor::PerformPreProcessing | Start Image PreProcessing
ImageGrabber::RunGrabber | Start Grabbing Frame from VIDEO stream.
ImageGrabber::RunGrabber | Initiating MAT
ImageGrabber::RunGrabber | Finished Initiating MAT, reading frame!
OpenCV(4.0.0-dev) Error: Assertion failed (u->refcount == 0) in deallocate, file /home/pi/opencv/modules/core/src/matrix.cpp, line 169
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.0.0-dev) /home/pi/opencv/modules/core/src/matrix.cpp:169: error: (-215:Assertion failed) u->refcount == 0 in function 'deallocate'

Aborted

As can be seen from the debug messages, the error happens on the line:

if (videoCapture.read(frame))

Since it happens here, it feels like something that is out of my control.

This code runs fine on my windows machine. Another peculiar thing is the fact that it tries to open the "OpenCL runtime"? As far as I know, opencl is not supported on the raspberry pi. As can be seen, this runs in a seperate thread and the frame is accessed in another thread, but protected by mutexes. In the other thread a pointer swap is done to prevent the necessity to copy the frame.

I have no clue if I am doing something wrong or that something underwater (with opencv) is going wrong. If someone knows of a similar issue or can give me pointers (no pun intended ;) ) to a solution, I would be very glad.

Thanks in advance!

edit retag flag offensive close merge delete

Comments

be so nice, and replace the screenshot of your errors with a text version, thank you !

berak gravatar imageberak ( 2018-12-11 11:08:50 -0500 )edit

uhmm, what is _framebuffer, and why do you neeed to lock it ?

can it be you're using multiple threads here (without telling us about it) ?

berak gravatar imageberak ( 2018-12-11 11:12:17 -0500 )edit
1

@berak Ok, no problem. Also something funky is going on, when running in gdb it shows that it's grabbing atleast a few more frames until it's aborted.

I mentioned that it's running accross multiple threads (so you probably didn't read the full post but thats fine). _framebuffer is a struct as follows:

struct ConcurrentFrameBuffer {
    cv::UMat frameBuf0; 
    cv::UMat frameBuf1;
    cv::UMat *writeFrame = &frameBuf0;
    bool notempty = false;
    std::mutex mx;  //to lock this frame/struct
};
EZenderink gravatar imageEZenderink ( 2018-12-11 11:18:32 -0500 )edit

yea, obviously missed that part, but there exactly might be your problem ;)

also this is terrible:

cv::UMat *writeFrame

DON'T EVER use pointers to cv::Mat or UMAt, since you defeat the internal refcounting with aliased ponters

berak gravatar imageberak ( 2018-12-11 11:23:56 -0500 )edit

Oh that sucks. I didn't know that. So it isn't possible to do manual pointer swapping to improve performance? (I need it to be multithreaded tho). I didn't feel like copying the data to a new instance of UMat to squeeze out that little bit of extra performance. Then again, why is this working without issues on windows?

EZenderink gravatar imageEZenderink ( 2018-12-11 11:28:59 -0500 )edit

a pointer swap

no, swap the actual mat using cv::swap(), NOT pointers to it

again, Mat's (and UMat's too) are refcounted objects. you only copy the Mat header( ~60bytes, not the pixel data). no reason to be afraid of it.

berak gravatar imageberak ( 2018-12-11 11:29:21 -0500 )edit

why is this working without issues on windows?

lookup "undefined behaviour" (that's what you made here.) it may or may not work, or even run for weeks w/o problems in a different env, and then crash in an entirely different part of your program.

berak gravatar imageberak ( 2018-12-11 11:34:40 -0500 )edit

again, to boil it down -- using pointers to UMat defeats the internal refcounting, do you understand that part ?

berak gravatar imageberak ( 2018-12-11 11:38:20 -0500 )edit
1

Sorry for the late reply. Yes, the UMat uses an internal reference counter much like shared_ptr to keep track if it's still in scope or not and by using pointers (cross scopes) it messes up this internal reference counter. Thats the gist of it right?

EZenderink gravatar imageEZenderink ( 2018-12-11 14:25:31 -0500 )edit

@berak, Sorry for bothering you with this again, I changed the pointer swapping to cv::swap. Now I write only to frameBuf0 in the videocapture thread. In the other thread, once a second, i lock the mutex, swap the buffers between frameBuf0 and 1 using cv::swap and then release the lock. After that, in the same thread I perform some processing on frameBuf1. But... no dice. Still the same assert failure occurs. Could this be because I am copying the local Mat to the UMat (using copyTo)(frameBuf0) that resides within ConcurrentFrameBuffer struct? (Also the fact is that the assertion failure happens on videoCapture.read() instead of the point where I read the swapped UMat is odd).

EZenderink gravatar imageEZenderink ( 2018-12-12 02:45:50 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
1

answered 2018-12-12 05:28:01 -0500

EZenderink gravatar image

Edit2: Found the problem, and found the solution: I had a processing function that accepted a reference to the umat. it performed resizing. Instead of putting the result into a new instance of UMat i made it overwrite using the umat from the referencepointer:

cv::resize(imagereference, imagereference, cv::Size, factor, factor)

It failed the refcount assertion here. (Since it happens at deallocation, internally it tries to deallocate it or something and then assertion is thrown).

I changed it to this:

cv::UMat output;
cv::resize(imagereference, output, cv::Size, factor, factor)

And returned output. (Since smartpointers are used it does not copy the whole image data to a new instance. This i didn't know previously.

edit flag offensive delete link more

Comments

i hope you don't mind, making this an answer ;)

berak gravatar imageberak ( 2018-12-12 05:29:31 -0500 )edit
1

I don't mind at all. Hopefully this answer will help someone else down the line!

EZenderink gravatar imageEZenderink ( 2018-12-12 09:04:18 -0500 )edit
-2

answered 2018-12-11 20:26:05 -0500

supra56 gravatar image

Change this:

if (videoCapture.read(frame)) {
to
if (videoCapture.read()) {
edit flag offensive delete link more

Comments

By doing that I won't get to write the frame to an actual mat right? It just checks if it can read frames ;). I believe @berak nailed the issue down. But thanks for your answer.

EZenderink gravatar imageEZenderink ( 2018-12-12 02:05:55 -0500 )edit

ignore this answer ...

berak gravatar imageberak ( 2018-12-12 02:07:06 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-12-11 11:05:39 -0500

Seen: 481 times

Last updated: Dec 11 '18