Hello all,
I've been trying to debug this for a month, sure it was my bad programming practise, but I think it may be a bug, so I'm asking here first before I report.
Consider the following code:
#include <sys/resource.h> // memory management.
#include <stdio.h>
#include <iostream>
#include <iomanip>
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/video/background_segm.hpp"
using namespace std;
using namespace cv;
// Load frame from disk.
void readFrame(int frameNum, Mat &frame) {
// Construct filenames
Mat image;
stringstream number, filename;
number << setw(7) << setfill('0') << frameNum; // expecting over 1e10 images over the installation period.
filename << "../images/store-" << number.str() << ".jpg"; // assumes jpegs!//
cout << "Loading filename: " << filename.str() << endl;
image = imread( filename.str() );
if (image.empty() or !image.data) {
cout << "Input image empty:\n";
}
frame = image.clone();
}
// Class to hold the perceptual chunks.
class percepUnit {
public:
cv::Mat image; // percept itself
cv::Mat mask; // alpha channel
// constructor method
percepUnit(cv::Mat &ROI, cv::Mat &alpha, int ix, int iy, int iw, int ih, int area) {
image = ROI.clone();
mask = alpha.clone();
}
};
// Segment foreground from background
void segmentForeground(list<percepUnit*> &percepUnitsForeground, Mat &foreground, Mat &frame) {
Mat contourImage = Mat(foreground.rows, foreground.cols, CV_8UC1, Scalar::all(0));
vector<vector<Point>> contours;
int area;
// The following causes strange spikes in memory usage:
// find contours
findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
for (int idx = 0; idx < contours.size(); idx++) {
area = contourArea(contours[idx]);
if (area > 100) {
percepUnit *thisUnit = new percepUnit(frame, contourImage, 0, 0, 100,100, area);
percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
}
}
/* The following does not:
findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
for (int idx = 0; idx < contours.size(); idx++) {
area = contourArea(contours[idx]);
}*/
/* Neither does this:
for (int idx = 0; idx < 10; idx++) {
percepUnit *thisUnit = new percepUnit(frame, contourImage, 0, 0, 100,100, area);
percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
}*/
}
int main(int argc, const char** argv)
{
int frameCount = 78298;
Mat frame, foreground;
BackgroundSubtractorMOG2 MOG2model;
list<percepUnit*> scratchPercepUnitsForeground;
// add rusage stuff
struct rusage usage; // memory usage.
for(int i=0; i<= 75; i++)
{
// run full segmenter here. (background disabled)
readFrame(frameCount, frame); // was frame = readFrame();
// Only process if this frame actually loaded (non empty)
if ( not frame.empty() ) {
MOG2model(frame,foreground); // Update MOG2 model, downscale?
// before we segment again clear scratch
// TODO how to delete the actual memory allocated? Run delete on everything?
for (list<percepUnit*>::iterator percepIter = scratchPercepUnitsForeground.begin();
percepIter != scratchPercepUnitsForeground.end();
percepIter++) {
delete *percepIter; // delete what we point to.
//percepIter = scratchPercepUnitsForeground.erase(percepIter); // remove the pointer itself, and update the iterator.
}
// Added with EDIT1
scratchPercepUnitsForeground.clear();
// Segment the foreground regions and generate boolImage to extract from background.
segmentForeground(scratchPercepUnitsForeground, foreground, frame);
}
frameCount++;
getrusage(RUSAGE_SELF, &usage);
cout << "DEBUG leakTest_bug_report " << i << " " << usage.ru_maxrss/1024.0 << endl;
}
return 0;
}
If you use the images available here, you will find that the program's memory usage increases, and it seems to increase with the number of foreground contours. Since I'm clearing my storage (scratchPercepUnitsForeground) for each frame, I don't see why there should be increasing memory usage. The segmentForeground() function should exit, deallocating all its used memory, for each frame. The memory usage should be constant over time since we only check memory usage after the function has exited. It seems something is being left over that I can't figure out.
If I run just the findContours() part without the percepUnit() constructor, memory usage is constant, as I expect. If I run just the percepUnit() constructor without findContours(), memory usage is constant. Memory usage increases only when I use both. See commented code in segmentForeground() above.
I've confirmed this issue on two of my machines (both AMD64) and running opencv 2.4.6.1 and 2.4.5.
EDIT1
Code above has been changed to include the suggestion below, and yet the problem persists.
Here is what the memory increase looks like:
The red line is the increase of memory (which is correlated with the test images linked above) seen when findContours() and the constructor are both called. The stable lines below are the two cases where we run either findContours() or the constructor.
This should be reproducible! Please make sure you can reproduce it before providing an answer.
EDIT2 (revised code and valgrind output, removed pointer method)
Here I have changed the list from a list of pointers to a list of instances. The memory increase is confirmed.
#include <sys/resource.h> // memory management.
#include <stdio.h>
#include <iostream>
#include <iomanip>
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/video/background_segm.hpp"
using namespace std;
using namespace cv;
// Load frame from disk.
void readFrame(int frameNum, Mat &frame) {
// Construct filenames
Mat image;
stringstream number, filename;
number << setw(7) << setfill('0') << frameNum; // expecting over 1e10 images over the installation period.
filename << "../images/store-" << number.str() << ".jpg"; // assumes jpegs!//
cout << "Loading filename: " << filename.str() << endl;
image = imread( filename.str() );
if (image.empty() or !image.data) {
cout << "Input image empty:\n";
}
frame = image.clone();
}
// Class to hold the perceptual chunks.
class percepUnit {
public:
cv::Mat image; // percept itself
cv::Mat mask; // alpha channel
// constructor method
percepUnit(cv::Mat &ROI, cv::Mat &alpha, int ix, int iy, int iw, int ih, int area) {
image = ROI.clone();
mask = alpha.clone();
}
};
// Segment foreground from background
void segmentForeground(list<percepUnit> &percepUnitsForeground, Mat &foreground, Mat &frame) {
Mat contourImage = Mat(foreground.rows, foreground.cols, CV_8UC1, Scalar::all(0));
vector<vector<Point>> contours;
int area;
// The following causes strange spikes in memory usage:
// find contours
findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
for (int idx = 0; idx < contours.size(); idx++) {
area = contourArea(contours[idx]);
if (area > 100) {
percepUnit thisUnit = percepUnit(frame, contourImage, 0, 0, 100,100, area);
percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
}
}
/* The following does not:
findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
for (int idx = 0; idx < contours.size(); idx++) {
area = contourArea(contours[idx]);
}*/
/* Neither does this:
for (int idx = 0; idx < 10; idx++) {
percepUnit thisUnit = percepUnit(frame, contourImage, 0, 0, 100,100, area);
percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
}*/
}
int main(int argc, const char** argv)
{
int frameCount = 78298;
Mat frame, foreground;
BackgroundSubtractorMOG2 MOG2model;
list<percepUnit> scratchPercepUnitsForeground;
// add rusage stuff
struct rusage usage; // memory usage.
for(int i=0; i<= 75; i++)
{
// run full segmenter here. (background disabled)
readFrame(frameCount, frame); // was frame = readFrame();
// Only process if this frame actually loaded (non empty)
if ( not frame.empty() ) {
MOG2model(frame,foreground); // Update MOG2 model, downscale?
// before we segment again clear scratch
scratchPercepUnitsForeground.clear();
// Segment the foreground regions and generate boolImage to extract from background.
segmentForeground(scratchPercepUnitsForeground, foreground, frame);
}
frameCount++;
getrusage(RUSAGE_SELF, &usage);
cout << "DEBUG leakTest_bug_report " << i << " " << usage.ru_maxrss/1024.0 << endl;
}
return 0;
}
Here is the corresponding valgrind output:
==3562== Memcheck, a memory error detector
==3562== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3562== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3562== Command: ./leakTest
==3562==
==3562==
==3562== HEAP SUMMARY:
==3562== in use at exit: 15,024 bytes in 7 blocks
==3562== total heap usage: 795,556 allocs, 795,549 frees, 29,269,731,785 bytes allocated
==3562==
==3562== 568 bytes in 1 blocks are still reachable in loss record 1 of 7
==3562== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562== by 0x63A720A: __fopen_internal (iofopen.c:76)
==3562== by 0xA8BC050: libjpeg_general_init (in /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2)
==3562== by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562== by 0x400F3DE: _dl_init (dl-init.c:52)
==3562== by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562==
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 2 of 7
==3562== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562== by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x1495E4AE: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x14950888: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562== by 0x400F3DE: _dl_init (dl-init.c:52)
==3562== by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562==
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 3 of 7
==3562== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562== by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x1495E0EF: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x14950890: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562== by 0x400F3DE: _dl_init (dl-init.c:52)
==3562== by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562==
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 4 of 7
==3562== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562== by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x14971A6F: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x14950898: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562== by 0x400F3DE: _dl_init (dl-init.c:52)
==3562== by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562==
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 5 of 7
==3562== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562== by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x1499024F: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x149508A0: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562== by 0x400F3DE: _dl_init (dl-init.c:52)
==3562== by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562==
==3562== 2,072 bytes in 1 blocks are still reachable in loss record 6 of 7
==3562== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562== by 0x1495F675: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x149610EF: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x149253B8: ??? (in /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.24.4)
==3562== by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562== by 0x400F3DE: _dl_init (dl-init.c:52)
==3562== by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562==
==3562== 4,096 bytes in 1 blocks are still reachable in loss record 7 of 7
==3562== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3562== by 0xA8BC067: libjpeg_general_init (in /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2)
==3562== by 0x400F305: call_init.part.0 (dl-init.c:85)
==3562== by 0x400F3DE: _dl_init (dl-init.c:52)
==3562== by 0x40016E9: ??? (in /lib/x86_64-linux-gnu/ld-2.15.so)
==3562==
==3562== LEAK SUMMARY:
==3562== definitely lost: 0 bytes in 0 blocks
==3562== indirectly lost: 0 bytes in 0 blocks
==3562== possibly lost: 0 bytes in 0 blocks
==3562== still reachable: 15,024 bytes in 7 blocks
==3562== suppressed: 0 bytes in 0 blocks
==3562==
==3562== For counts of detected and suppressed errors, rerun with: -v
==3562== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
And yet, memory still increases unexplainably: (the overall increase from the previous plot is due to running this test in valgrind.)
EDIT3
It was suggested below that I read proc rather than using the rusage method, and look at this, memory does not steadily increase: (!)
This does appear similar to the massif output!! So I guess I need to redo all my unit tests. Anyone have a reason I should not give up here and consider the issue rusage?