Ask Your Question
0

Opencv fails to allocate memory

asked 2013-02-01 09:54:35 -0600

manmedia gravatar image

updated 2013-02-03 11:00:35 -0600

Hello,

In part of my code I am doing this!

// LONG_HIST, etc. are pre-defined
// getHistoryLong() etc. return IplImage*
myMasks.clear(); // Clear so that we only have one set at a time.
myMasks.push_back(std::make_pair<int, IplImage>(LONG_HIST, *cvCloneImage(repmo.getHistoryLong())));
myMasks.push_back(std::make_pair<int, IplImage>(LONG_HIST_NEW, *cvCloneImage(repmo.getHistoryLong())));
myMasks.push_back(std::make_pair<int, IplImage>(SHORT_HIST, *cvCloneImage(repmo.getHistoryLong())));

My image processing application is such that when I detect a certain event, I store the mask information of my current scene using a method called updateMasks() which, in essence, does the above. However, I only update the images only once. I have also in place a flag called "savedMasks" and this is what i do when I save the masks to my current scene object:

if(currentSceneObject->getSaveMasks == true) {
    repMo.updateMasks(); // Update mask image vector
    currentSceneObject->saveMasks(repMo.getMaskImages()); // save them against current scene
    currentSceneObject->setSaveMasks == false; // clear the flag
}

If my understand is correct, It will only save it once, but not multiple times.

__WHY AM I CLONING OR DEEP COPYING THE IplImage*(s)__

The reason I am cloning the image is that if I don't and simply dereference it, after stack call is returned, the information is gone! I mean partially. Because it turns out that when I am trying to fetch the saved values back into the main application It is having access violation errors.

This solves the problem except the fact that now I am getting memory allocation failures as:

OpenCV Error: Insufficient Memory (failed to allocate xyz bytes) in unknown function, file 
..\..\opecv\modules\core\src\alloc.cpp line 52.

from OpenCV. I believe that using cvCloneImage() I am making too many calls to malloc but not freeing the memory correctlyl. However, I might be wrong. Shall I store it as OpenCV and load it as IplImage* pointer to be safe?

edit retag flag offensive close merge delete

3 answers

Sort by ยป oldest newest most voted
1

answered 2013-02-02 11:36:18 -0600

sammy gravatar image

You are storing in memory more vectors of images. No wonder your computer tells you there is no more memory. The solution is not to store the images.

Please keep in mind that a decompressed image/video is way larger than a compressed one. A usual 720p video one minute long needs 4.7GB of memory when decompressed.

So change your algorithm to use only a few frames at once, then release them correctly.

edit flag offensive delete link more

Comments

@sammy thanks for your answer but this is unfortunately not a solution. I need to store those images and I have a routine that makes sure that images are stored only ocassionally, not all the time. For example, i have a flag called "ImageSaved" and it only stores those images when the flag is true. Once I save them, I don't store them anymore. Additionally, every time a set of images are stored in the vector, previous images are cleared from it beforehand so that this problem can be avoided. However, it seems that I am running out of memory anyway. Unfortunately, the design specification is to store those images anyway. I hope you understand why I am suffering now.

manmedia gravatar imagemanmedia ( 2013-02-03 10:50:24 -0600 )edit
  • First, make sure you do not have leaks. Using the C++ interface, (with cv::Mat instead of IplImage*) is a great help on this.

  • Next, you should know that in 99.99% of the cases, there is a smarter algorithm to tackle the problem, instead of overloading RAM. Just look at the bigger picture.

  • And if you still insist in keeping the images in memory, buy a server with way mode RAM (the one I am programming for right now has 256GB RAM).

  • Finally, if you stick to your solution, you need to know exactly how many images you expect, how many images fit your hardware memory, and then build a limitation in the app, so you won't receive out-of memory exceptions. An app that crashes is not doing its job so your algorithm isn't of any good.

sammy gravatar imagesammy ( 2013-02-04 01:57:38 -0600 )edit

@sammy I agree with your comment where you mentioned about keeping it for C++ rather than C and C++ mixed. However, buying a server will probably not be in my control (or I might get fired!). I am struggling because the legacy code cannot be changed! And even after clearing all the images, it still gives me errors...aaargh....don't know!!

manmedia gravatar imagemanmedia ( 2013-02-04 04:54:25 -0600 )edit
0

answered 2013-02-04 11:47:49 -0600

manmedia gravatar image

updated 2013-02-05 13:10:07 -0600

__Solved this problem (temporarily) by doing the following:__

I was saving 4 IplImage* images each of size 288h x 352w. But what I wasn't doing before is releasing (clearing) the images in my currentSceneObject once I save them. The trick was to save and load - but not to __HOLD__ it for long. However, that only helps me in short run when I am saving my history images under Long, Short, and Not-Allowed cateogories whereas there are total 6 categories (Long, Long_New, Short, Short_New, Not_Allowed, Not_Allowed_New). However, I think if I manage to convince my tech. lead to stop using pointer references for masks and use values in, e.g. cv::Mat types, it will be better and more efficient.

One key thing is NOT to mix up C and C++ codes in OpenCV. Some stuff are better buried under and unearthing them simply makes life painful. And if possible, minimise usage of pointers in your designs and use __ONLY__ if you have no choice.

edit flag offensive delete link more

Comments

please do NOT use the answer form for discution - it's only reserved for answers. Update your answer, or use a comment to the question/answer.

sammy gravatar imagesammy ( 2013-02-05 06:34:24 -0600 )edit

I am certain that the OpenCV uses its own malloc wrapper to allocate memory and probably uses a wrapper for free (alloc.cpp file in modules)...isn't there any way to ask OpenCV allocate more memory :(

manmedia gravatar imagemanmedia ( 2013-02-17 02:41:58 -0600 )edit

You did not understand the basic issue: If you have a 10 liters bucket, there is NO way to carry 20 liters at once with it. You are trying to allocate more memory than it is available there. On 32bit systems 2GB is the absolute limit - nothing will give you more. On 64bit systems, the theoretical limit is higher, the practical limit is around a few GB on a PC. What you have to do is to limit the memory usage of your application. So, the OpenCV wrapper is of no use here.

sammy gravatar imagesammy ( 2013-02-18 01:29:46 -0600 )edit

I suggest you yo open up another question describing your algorithm and to ask for ways to improve memory usage. Usually, gains by a smarter algorithm are way bigger than what you can do by buying more RAM

sammy gravatar imagesammy ( 2013-02-18 02:10:14 -0600 )edit
0

answered 2013-03-01 06:46:31 -0600

manmedia gravatar image

Solved this problem now!!!!

The issue is that my algorithm was relying on an image stabiliser that used some offsets to summarise to detect velocity and indicate whether or not the camera has moved. This stabiliser outputs were a bit sluggish and led to this problem (don't want to explain much details).

I solved it by using a flag that would stop allocating any memory when the camera is in motion. Was tricky to find it though!!

edit flag offensive delete link more

Question Tools

Stats

Asked: 2013-02-01 09:54:35 -0600

Seen: 29,647 times

Last updated: Mar 01 '13