B. Bogart's profile - activity

pretrained model for gaze / frontal face detection for cv::dnn? Hello all, I'm very happy with the performance and accu

Indeed YAML 1.0 is not supported by current YAML libraries (which require YAML 1.1 --- at the start of documents). I think opencv should switch over to YAML 1.1, I'm filing a bug report to that effect now.

I ended up doing split / merge on the GPU:

void percepUnit::applyAlpha() {

    cv::gpu::GpuMat tmpImage, tmpMask, tmpAlphaImage;
    std::vector<cv::gpu::GpuMat> channels;

    tmpImage.upload(this->image);
    tmpMask.upload(this->mask);

    cv::gpu::split(tmpImage,channels); // break image into channels
    channels.push_back(tmpMask); // append alpha channel
    cv::gpu::merge(channels,tmpAlphaImage); // combine channels

    tmpAlphaImage.download(this->alphaImage);

    tmpAlphaImage.release();
    tmpImage.release();
    tmpMask.release();

    channels[0].release();
    channels[1].release();
    channels[2].release();
}

@Adi is my code useful in being in openframeworks?

@Adi ok. Do you have to render in opencv, or could you render in opengl? I have been using alpha channels in my application, and when I was doing the rendering in opencv, I was doing so with a loop over all pixels... Now I'm rendering in opengl (openframeworks), but applying the alpha channel is not efficient either: http://answers.opencv.org/question/31136/fastest-way-to-apply-alpha-channel-to-image/ OpenCV does need some better alpha handling...

I have a list of instances that contain a cv::Mat image and a cv::Mat alpha/mask channel. These are separate Mats because they are generated according to different processes.

Before I render (in GL) I need to convert the RGB image and mask images into an RGBA image. Here is my current function:

void percepUnit::applyAlpha() {

    int x,y,w,h;
    /*vector<cv::Mat> channels;
    if (image.rows == mask.rows and image.cols == mask.cols) {
        cv::split(image,channels); // break image into channels
        channels.push_back(mask); // append alpha channel

        cv::merge(channels,alphaImage); // combine channels
    }*/

    // Avoid merge
    cv::Mat src[] = {this->image, this->mask};
    int from_to[] = {0,0, 1,1, 2,2, 3,3};
    this->alphaImage = Mat(image.rows, image.cols, CV_8UC4);
    cv::mixChannels(src, 2, &(this->alphaImage), 1, from_to, 4); // &(*alphaImage)?
}

I just loop through my list of instances and applyAlpha() before rendering. I had to increase the resolution of the Mats to work around an opencv bug to 1280x720, and now this function uses a lot of time (about half the cycles of the whole program).

Is there a way I can speed up this operation?

The list contains a few thousand instances, so it seems some parallelism could help; I have 4 cores and a fast GPU (GeForce 780).

If you have opencv_gpu, you could try cv::gpu::alphaComp(), which is likely quite efficient!

How do you determine the "different alpha at different positions in the ROI"?

It was recommended by @Nghia that I try and make the percepts a constant size. Sure enough, if I fix the dimensions and type of the cv::Mat members of percepUnit, then the leak disappears.

So it seems to me this is a bug in OpenCV that effects calling clone() and copyTo() on Mats of different sizes that are class members. So far unable to reproduce in a simple program. The leak does seem small enough that it may be the headers leaking, rather than the underlying image data.

It was recommended by @Nghia that I try and make the percepts a constant size. Sure enough, if I fix the dimensions and type of the cv::Mat members of percepUnit, then the leak disappears.

So it seems to me this is a bug in OpenCV that effects calling clone() and copyTo() on Mats of different sizes that are class members. So far unable to reproduce in a simple program. The leak does seem small enough that it may be the headers leaking, rather than the underlying image data.

Thanks @Nghia. I did try feeding the program the same frame over and over, and there is no leak in that case. So I expect the leak would stop when the loop repeats. Now if you feed it frames from a webcam or other video, that should show the leak better. Note that considering the results you post here, I am convinced you are reproducing my problem. As for fragmentation, is that possible with no explicit new/delete ?

@Nghia bump.

@Nghia In this version the list contains instances (not pointers to instances via new). The clone function just assigns each class member in the existing instance ((distIter->unit)) to the values of the new instance (newUnit). Since cv::Mats are ref counted, as I understand this should work. I also tried ((distIter->unit)) = newUnit;, but memory still increases. The percepUnit destructor does not do anything, since the class does not allocate memory. I tried putting cv::Mat::release() calls in the destructor, but that did not change anything either (which is expected since Mats are ref counted). I also tried calling cv::Mat::release() in percepUnit::clone() for cv::Mat members, but that also did not make any difference.

Ok, good. I put my ms_print output above.

@Nghia Did the crappy ms_print output show a memory increase? As for the graph, I just parsed the samples from the ms_print output and used it to generate a plot for "total" in R. The crappy internal graph should still show a memory increase.

Could you please run massif? (its faster than memcheck) Over 400 frames the increase is small, likely too small to see in top, but you can use the plots above for comparison. Over 100,000 frames the memory increase becomes very significant. (Memory should increase if you feed it any input, like from a web-cam for example). If memory is increasing on your machine (as in the plots above), then it's clearly the code, but I can't figure how to fix it.

Thanks @Nghia. Valgrind does not show any leaks on my end either, but massif does show increasing memory usage (That is the data used to generate the plots above). Can you confirm that massif reports increasing memory usage over time?

@Nghia See the update above.

Hello all.

This is related to an already posted question, but I think I've reduced the problem to some degree. I'm totally at my wit's end with this and no one I have talked to has been able to see what could be wrong.

Here is all the code. I put in both leaking and non-leaking cases to make the problem more clear.

This plot shows the difference in memory usage in both cases. The only difference in the code between the cases is lines 353 and 449 in clustering.cpp where mergePerceps() is called only in the leaking case. If I comment out mergePerceps() no leak, if I comment out line 452 (*(distIter->unit)).clone(newUnit); no leak, if I remove the cv::Mat members from the class, no leak.

I've tried valgrind memcheck without any hints to the problem, due to the total lack of any explicit allocation.

If there is another Linux user out there that could verify this is reproducible on another machine, I would really appreciate it. One only needs a sequence of appropriately named images to test, see segmentation::readFrame() line 82 of segmentation.cpp for details. I'm using OpenCV 2.4.5 on Ubuntu Precise 64bit.

It seems I'm using cv::Mats in some unexpected way, so I would be happy to hear what the proper method of doing this operation is to avoid this leak.

If this is a poor use of this system, I apologize. If there is a better venue for this type of question, I'd be happy to move it. Thanks.

Update 1

Here (~70MB) are some test images. The path needs to be changed on line 16 of segmentationClusteringTest.cpp, but the rest should work as I did not change the filenames.

Update 2

ms_print output for leaking (top) and nonleaking (bottom) cases:

@berak, I was about to ask this question again, to find I already had! I'm trying to replace an instance containing a mat with a new instance (or a copy of all the members), but my replacement causes a leak. using clone() does not solve the leak. Please see this post: http://stackoverflow.com/questions/18772065/how-to-replace-an-instance-with-another-instance-via-pointer

I'm trying to rewrite my very slow naive segmentation using floodFill to something faster. I ruled out meanShiftFiltering a year ago because of the difficulty in labelling the colours and then finding their contours.

The current version of opencv seems to have a fast new function that labels segments using mean shift: gpu::meanShiftSegmentation(). It produces images like the following:

So this looks to me pretty close to being able to generating contours. How can I run findContours to generate segments?

Seems to me, this would be done by extracting the labelled colours from the image, and then testing which pixel values in the image match each label colour to make a boolean image suitable for findContours. This is what I have done in the following (but its a bit slow and strikes me there should be a better way):

Mat image = imread("test.png");

...
// gpu operations on image resulting in gpuOpen
...

// Mean shift
TermCriteria iterations = TermCriteria(CV_TERMCRIT_ITER, 2, 0);
gpu::meanShiftSegmentation(gpuOpen, segments, 10, 20, 300, iterations);

// convert to greyscale (HSV image)
vector<Mat> channels;
split(segments, channels);

// get labels from histogram of image.
int size = 256;
labels = Mat(256, 1, CV_32SC1);
calcHist(&channels.at(2), 1, 0, Mat(), labels, 1, &size, 0);

// Loop through hist bins
for (int i=0; i<256; i++) {
    float count = labels.at<float>(i);

    // Does this bin represent a label in the image?
    if (count > 0) {
        // find areas of the image that match this label and findContours on the result.
        Mat label = Mat(channels.at(2).rows, channels.at(2).cols, CV_8UC1, Scalar::all(i)); // image filled with label colour.
        Mat boolImage = (channels.at(2) == label); // which pixels in labeled image are identical to this label?
        vector<vector<Point>> labelContours;
        findContours(boolImage, labelContours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
        // Loop through contours.
        for (int idx = 0; idx < labelContours.size(); idx++) {
            // get bounds for this contour.
            bounds = boundingRect(labelContours[idx]);

            // create ROI for bounds to extract this region
            Mat patchROI = image(bounds);
            Mat maskROI = boolImage(bounds);
        }
    }
}

Is this the best approach or is there a better way to get the label colours? Seems it would be logical for meanShiftSegmentation to provide this information? (vector of colour values, or vector of masks for each label, etc.)

Thank you.

Could you explain the difference between rss and maxrss? If other processes request more memory, does maxrss shrink while rss stays stable?

How can I convert from pages to bytes?