Ask Your Question

B. Bogart's profile - activity

2020-03-23 03:33:21 -0500 received badge  Popular Question (source)
2019-12-20 12:03:03 -0500 asked a question pretrained model for gaze / frontal face detection for cv::dnn?

pretrained model for gaze / frontal face detection for cv::dnn? Hello all, I'm very happy with the performance and accu

2019-07-24 16:32:29 -0500 received badge  Nice Question (source)
2018-02-07 07:45:44 -0500 received badge  Popular Question (source)
2017-10-22 13:20:48 -0500 received badge  Nice Answer (source)
2017-07-25 10:51:49 -0500 received badge  Famous Question (source)
2016-10-04 05:22:52 -0500 received badge  Student (source)
2016-03-09 04:14:29 -0500 received badge  Notable Question (source)
2015-12-19 01:15:11 -0500 commented question How do I load an OpenCV generated yaml file in python?

Indeed YAML 1.0 is not supported by current YAML libraries (which require YAML 1.1 --- at the start of documents). I think opencv should switch over to YAML 1.1, I'm filing a bug report to that effect now.

2015-08-21 02:42:42 -0500 received badge  Popular Question (source)
2015-07-23 16:38:17 -0500 received badge  Famous Question (source)
2014-10-29 16:50:18 -0500 received badge  Notable Question (source)
2014-06-25 05:36:05 -0500 received badge  Popular Question (source)
2014-04-11 04:11:12 -0500 received badge  Teacher (source)
2014-04-10 12:15:42 -0500 received badge  Self-Learner (source)
2014-04-10 11:50:12 -0500 answered a question Fastest way to apply alpha channel to image

I ended up doing split / merge on the GPU:

void percepUnit::applyAlpha() {

    cv::gpu::GpuMat tmpImage, tmpMask, tmpAlphaImage;
    std::vector<cv::gpu::GpuMat> channels;

    tmpImage.upload(this->image);
    tmpMask.upload(this->mask);

    cv::gpu::split(tmpImage,channels); // break image into channels
    channels.push_back(tmpMask); // append alpha channel
    cv::gpu::merge(channels,tmpAlphaImage); // combine channels

    tmpAlphaImage.download(this->alphaImage);

    tmpAlphaImage.release();
    tmpImage.release();
    tmpMask.release();

    channels[0].release();
    channels[1].release();
    channels[2].release();
}
2014-04-07 10:53:58 -0500 commented answer Efficient Alpha Blending

@Adi is my code useful in being in openframeworks?

2014-04-04 11:32:57 -0500 commented answer Efficient Alpha Blending

@Adi ok. Do you have to render in opencv, or could you render in opengl? I have been using alpha channels in my application, and when I was doing the rendering in opencv, I was doing so with a loop over all pixels... Now I'm rendering in opengl (openframeworks), but applying the alpha channel is not efficient either: http://answers.opencv.org/question/31136/fastest-way-to-apply-alpha-channel-to-image/ OpenCV does need some better alpha handling...

2014-04-03 07:07:40 -0500 received badge  Necromancer (source)
2014-04-02 12:48:35 -0500 asked a question Fastest way to apply alpha channel to image

I have a list of instances that contain a cv::Mat image and a cv::Mat alpha/mask channel. These are separate Mats because they are generated according to different processes.

Before I render (in GL) I need to convert the RGB image and mask images into an RGBA image. Here is my current function:

void percepUnit::applyAlpha() {

    int x,y,w,h;
    /*vector<cv::Mat> channels;
    if (image.rows == mask.rows and image.cols == mask.cols) {
        cv::split(image,channels); // break image into channels
        channels.push_back(mask); // append alpha channel

        cv::merge(channels,alphaImage); // combine channels
    }*/

    // Avoid merge
    cv::Mat src[] = {this->image, this->mask};
    int from_to[] = {0,0, 1,1, 2,2, 3,3};
    this->alphaImage = Mat(image.rows, image.cols, CV_8UC4);
    cv::mixChannels(src, 2, &(this->alphaImage), 1, from_to, 4); // &(*alphaImage)?
}

I just loop through my list of instances and applyAlpha() before rendering. I had to increase the resolution of the Mats to work around an opencv bug to 1280x720, and now this function uses a lot of time (about half the cycles of the whole program).

Is there a way I can speed up this operation?

The list contains a few thousand instances, so it seems some parallelism could help; I have 4 cores and a fast GPU (GeForce 780).

2014-04-02 12:39:28 -0500 answered a question Efficient Alpha Blending

If you have opencv_gpu, you could try cv::gpu::alphaComp(), which is likely quite efficient!

2014-03-12 13:00:59 -0500 commented question Blending two images with masked ROI

How do you determine the "different alpha at different positions in the ROI"?

2014-02-21 11:22:31 -0500 answered a question What is the best practise for passing cv::Mats around

It was recommended by @Nghia that I try and make the percepts a constant size. Sure enough, if I fix the dimensions and type of the cv::Mat members of percepUnit, then the leak disappears.

So it seems to me this is a bug in OpenCV that effects calling clone() and copyTo() on Mats of different sizes that are class members. So far unable to reproduce in a simple program. The leak does seem small enough that it may be the headers leaking, rather than the underlying image data.

2014-02-21 11:20:54 -0500 answered a question Debugging memory leak related to instance with cv::Mat members

It was recommended by @Nghia that I try and make the percepts a constant size. Sure enough, if I fix the dimensions and type of the cv::Mat members of percepUnit, then the leak disappears.

So it seems to me this is a bug in OpenCV that effects calling clone() and copyTo() on Mats of different sizes that are class members. So far unable to reproduce in a simple program. The leak does seem small enough that it may be the headers leaking, rather than the underlying image data.

2014-01-31 10:49:10 -0500 commented answer Debugging memory leak related to instance with cv::Mat members

Thanks @Nghia. I did try feeding the program the same frame over and over, and there is no leak in that case. So I expect the leak would stop when the loop repeats. Now if you feed it frames from a webcam or other video, that should show the leak better. Note that considering the results you post here, I am convinced you are reproducing my problem. As for fragmentation, is that possible with no explicit new/delete ?

2014-01-30 20:48:13 -0500 commented answer Debugging memory leak related to instance with cv::Mat members

@Nghia bump.

2014-01-23 13:52:17 -0500 commented answer Debugging memory leak related to instance with cv::Mat members

@Nghia In this version the list contains instances (not pointers to instances via new). The clone function just assigns each class member in the existing instance ((distIter->unit)) to the values of the new instance (newUnit). Since cv::Mats are ref counted, as I understand this should work. I also tried ((distIter->unit)) = newUnit;, but memory still increases. The percepUnit destructor does not do anything, since the class does not allocate memory. I tried putting cv::Mat::release() calls in the destructor, but that did not change anything either (which is expected since Mats are ref counted). I also tried calling cv::Mat::release() in percepUnit::clone() for cv::Mat members, but that also did not make any difference.

2014-01-23 13:33:49 -0500 commented answer Debugging memory leak related to instance with cv::Mat members

Ok, good. I put my ms_print output above.

2014-01-22 18:46:17 -0500 commented answer Debugging memory leak related to instance with cv::Mat members

@Nghia Did the crappy ms_print output show a memory increase? As for the graph, I just parsed the samples from the ms_print output and used it to generate a plot for "total" in R. The crappy internal graph should still show a memory increase.

2014-01-21 12:14:31 -0500 commented answer Debugging memory leak related to instance with cv::Mat members

Could you please run massif? (its faster than memcheck) Over 400 frames the increase is small, likely too small to see in top, but you can use the plots above for comparison. Over 100,000 frames the memory increase becomes very significant. (Memory should increase if you feed it any input, like from a web-cam for example). If memory is increasing on your machine (as in the plots above), then it's clearly the code, but I can't figure how to fix it.

2014-01-20 18:54:46 -0500 commented answer Debugging memory leak related to instance with cv::Mat members

Thanks @Nghia. Valgrind does not show any leaks on my end either, but massif does show increasing memory usage (That is the data used to generate the plots above). Can you confirm that massif reports increasing memory usage over time?

2014-01-16 13:01:31 -0500 commented question Debugging memory leak related to instance with cv::Mat members

@Nghia See the update above.

2014-01-15 19:19:06 -0500 asked a question Debugging memory leak related to instance with cv::Mat members

Hello all.

This is related to an already posted question, but I think I've reduced the problem to some degree. I'm totally at my wit's end with this and no one I have talked to has been able to see what could be wrong.

Here is all the code. I put in both leaking and non-leaking cases to make the problem more clear.

This plot shows the difference in memory usage in both cases. The only difference in the code between the cases is lines 353 and 449 in clustering.cpp where mergePerceps() is called only in the leaking case. If I comment out mergePerceps() no leak, if I comment out line 452 (*(distIter->unit)).clone(newUnit); no leak, if I remove the cv::Mat members from the class, no leak.

I've tried valgrind memcheck without any hints to the problem, due to the total lack of any explicit allocation.

If there is another Linux user out there that could verify this is reproducible on another machine, I would really appreciate it. One only needs a sequence of appropriately named images to test, see segmentation::readFrame() line 82 of segmentation.cpp for details. I'm using OpenCV 2.4.5 on Ubuntu Precise 64bit.

It seems I'm using cv::Mats in some unexpected way, so I would be happy to hear what the proper method of doing this operation is to avoid this leak.

If this is a poor use of this system, I apologize. If there is a better venue for this type of question, I'd be happy to move it. Thanks.

Update 1

Here (~70MB) are some test images. The path needs to be changed on line 16 of segmentationClusteringTest.cpp, but the rest should work as I did not change the filenames.

Update 2

ms_print output for leaking (top) and nonleaking (bottom) cases:

"Leaking Case"

"Nonleaking Case"

2013-09-16 18:13:46 -0500 asked a question How to replace an instance with cv::Mat members with another instance via pointer?

I'm doing online destructive clustering (clusters replace clustered objects) on a list of class instances (stl::list<percepunit>).

Background

My list of current percepUnits is: stl::list<percepUnit> units; and for each iteration I get a new list of input percepUnits stl::list<percepUnit> scratch; that need to be clustered with the units.

I want to maintain a fixed number of percepUnits (so units.size() is constant), so for each new scratch percepUnit I need to merge it with the nearest percepUnit in units. Following is a code snippet that builds a list (dists) of structures (percepUnitDist) that contain pointers to each pair of items in scratch and units percepDist.scratchUnit = &(*scratchUnit); and percepDist.unit = &(*unit); and their distance. Additionally, for each item in scratch I keep track of which item in units has the least distance minDists.

// For every scratch percepUnit:
for (scratchUnit = scratch.begin(); scratchUnit != scratch.end(); scratchUnit++) { 
    float minDist=2025.1172; // This is the max possible distance in unnormalized CIELuv, and much larger than the normalized dist.
    // For every percepUnit:
    for (unit = units.begin(); unit != units.end(); unit++) { 

        // compare pairs
        float dist = featureDist(*scratchUnit, *unit, FGBG);
        //cout << "distance: " << dist << endl;

        // Put pairs in a structure that caches their distances
        percepUnitDist percepDist;
        percepDist.scratchUnit = &(*scratchUnit); // address of where scratchUnit points to.
        percepDist.unit = &(*unit);
        percepDist.dist = dist;

        // Figure out the percepUnit that is closest to this scratchUnit.
        if (dist < minDist)
            minDist = dist;

        dists.push_back(percepDist); // append dist struct
    }
    minDists.push_back(minDist); // append the min distance to the nearest percepUnit for this particular scratchUnit.
}

So now I just need to loop through the percepUnitDist items in dists and match the distances with the minimum distances to figure out which percepUnit in scratch should be merged with which percepUnit in units. The merging process mergePerceps() creates a new percepUnit which is a weighted average of the "parent" percepUnits in scratch and units.

My understanding is that the following code should do the job just fine, but for some reason memory usage increases with each of these "merge" operations. Even if I provide a method that calls clone() for cv::Mat members, I still get an increase in memory. If I don't copy the cv::Mats at all there is no unexpected memory increase. If I don't "replace" the old instance with the newly merged one, I also don't see memory increase. This is related to this question

// Loop through dists and merge all the closest pairs.
// Loop through all dists
for (distIter = dists.begin(); distIter != dists.end(); distIter++) {
    // Loop through all minDists for each scratchUnit.
    for (minDistsIter = minDists.begin(); minDistsIter != minDists.end(); minDistsIter++) {
        // if this is the closest cluster, and the closest cluster has not already been merged, and the scratch has not already been merged.
        if (*minDistsIter == distIter->dist and not distIter->scratchUnit->remove) {

            percepUnit newUnit;
            mergePerceps(*(distIter->scratchUnit), *(distIter->unit), newUnit, FGBG);
            *(distIter->unit) = newUnit; // replace the cluster with the new merged version.

            distIter->scratchUnit->remove = true;
        }
    }
}

For completeness, here is the whole percepUnit ... (more)

2013-09-13 16:03:27 -0500 commented answer Deep copy of list of instances with cv::Mat members

@berak, I was about to ask this question again, to find I already had! I'm trying to replace an instance containing a mat with a new instance (or a copy of all the members), but my replacement causes a leak. using clone() does not solve the leak. Please see this post: http://stackoverflow.com/questions/18772065/how-to-replace-an-instance-with-another-instance-via-pointer

2013-08-29 19:23:32 -0500 asked a question How to run findContours() on meanShiftSegmentation() output?

I'm trying to rewrite my very slow naive segmentation using floodFill to something faster. I ruled out meanShiftFiltering a year ago because of the difficulty in labelling the colours and then finding their contours.

The current version of opencv seems to have a fast new function that labels segments using mean shift: gpu::meanShiftSegmentation(). It produces images like the following:

meanShiftSegmentation() output

So this looks to me pretty close to being able to generating contours. How can I run findContours to generate segments?

Seems to me, this would be done by extracting the labelled colours from the image, and then testing which pixel values in the image match each label colour to make a boolean image suitable for findContours. This is what I have done in the following (but its a bit slow and strikes me there should be a better way):

Mat image = imread("test.png");

...
// gpu operations on image resulting in gpuOpen
...

// Mean shift
TermCriteria iterations = TermCriteria(CV_TERMCRIT_ITER, 2, 0);
gpu::meanShiftSegmentation(gpuOpen, segments, 10, 20, 300, iterations);

// convert to greyscale (HSV image)
vector<Mat> channels;
split(segments, channels);

// get labels from histogram of image.
int size = 256;
labels = Mat(256, 1, CV_32SC1);
calcHist(&channels.at(2), 1, 0, Mat(), labels, 1, &size, 0);

// Loop through hist bins
for (int i=0; i<256; i++) {
    float count = labels.at<float>(i);

    // Does this bin represent a label in the image?
    if (count > 0) {
        // find areas of the image that match this label and findContours on the result.
        Mat label = Mat(channels.at(2).rows, channels.at(2).cols, CV_8UC1, Scalar::all(i)); // image filled with label colour.
        Mat boolImage = (channels.at(2) == label); // which pixels in labeled image are identical to this label?
        vector<vector<Point>> labelContours;
        findContours(boolImage, labelContours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
        // Loop through contours.
        for (int idx = 0; idx < labelContours.size(); idx++) {
            // get bounds for this contour.
            bounds = boundingRect(labelContours[idx]);

            // create ROI for bounds to extract this region
            Mat patchROI = image(bounds);
            Mat maskROI = boolImage(bounds);
        }
    }
}

Is this the best approach or is there a better way to get the label colours? Seems it would be logical for meanShiftSegmentation to provide this information? (vector of colour values, or vector of masks for each label, etc.)

Thank you.

2013-08-22 10:51:39 -0500 commented answer Bug? Increasing memory usage per iteration when using findContours()

Could you explain the difference between rss and maxrss? If other processes request more memory, does maxrss shrink while rss stays stable?

How can I convert from pages to bytes?

2013-08-21 20:03:32 -0500 marked best answer Bug? Increasing memory usage per iteration when using findContours()

Hello all,

I've been trying to debug this for a month, sure it was my bad programming practise, but I think it may be a bug, so I'm asking here first before I report.

Consider the following code:

#include <sys/resource.h> // memory management.
#include <stdio.h>
#include <iostream>
#include <iomanip>

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/video/background_segm.hpp"

using namespace std;
using namespace cv;

// Load frame from disk.
void readFrame(int frameNum, Mat &frame) {
    // Construct filenames
    Mat image;
    stringstream number, filename;

    number << setw(7) << setfill('0') << frameNum; // expecting over 1e10 images over the installation period.
    filename << "../images/store-" << number.str() << ".jpg"; // assumes jpegs!//
    cout << "Loading filename: " << filename.str() << endl;

    image = imread( filename.str() );

    if (image.empty() or !image.data) {
        cout << "Input image empty:\n";
    }

    frame = image.clone();
}

// Class to hold the perceptual chunks.
class percepUnit {

    public:
        cv::Mat image; // percept itself
        cv::Mat mask; // alpha channel

        // constructor method
        percepUnit(cv::Mat &ROI, cv::Mat &alpha, int ix, int iy, int iw, int ih, int area)  {
            image = ROI.clone();
            mask = alpha.clone();
        }
};

// Segment foreground from background
void segmentForeground(list<percepUnit*> &percepUnitsForeground, Mat &foreground, Mat &frame) {
    Mat contourImage = Mat(foreground.rows, foreground.cols, CV_8UC1, Scalar::all(0));
    vector<vector<Point>> contours;
    int area;

    // The following causes strange spikes in memory usage:
    // find contours
    findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);

    for (int idx = 0; idx < contours.size(); idx++) {

        area = contourArea(contours[idx]);

        if (area > 100) {

            percepUnit *thisUnit = new percepUnit(frame, contourImage, 0, 0, 100,100, area);
            percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
        }
    }

    /* The following does not:
    findContours(foreground, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);

    for (int idx = 0; idx < contours.size(); idx++) {
        area = contourArea(contours[idx]);
    }*/

    /* Neither does this:
    for (int idx = 0; idx < 10; idx++) {
        percepUnit *thisUnit = new percepUnit(frame, contourImage, 0, 0, 100,100, area);
        percepUnitsForeground.push_back(thisUnit); // Append to percepUnits
    }*/
}

int main(int argc, const char** argv)
{
    int frameCount = 78298; 
    Mat frame, foreground;
    BackgroundSubtractorMOG2 MOG2model;
    list<percepUnit*> scratchPercepUnitsForeground;

    // add rusage stuff
    struct rusage usage; // memory usage.

    for(int i=0; i<= 75; i++)
    {
        // run full segmenter here.  (background disabled)

        readFrame(frameCount, frame); // was frame = readFrame();

        // Only process if this frame actually loaded (non empty)
        if ( not frame.empty() ) {

            MOG2model(frame,foreground); // Update MOG2 model, downscale?

            // before we segment again clear scratch
            // TODO how to delete the actual memory allocated? Run delete on everything?
            for (list<percepUnit*>::iterator percepIter = scratchPercepUnitsForeground.begin(); 
                 percepIter != scratchPercepUnitsForeground.end();
                 percepIter++) {

                delete *percepIter; // delete what we point to.
                //percepIter = scratchPercepUnitsForeground.erase(percepIter); // remove the pointer itself, and update the iterator.
            }
            // Added with EDIT1
            scratchPercepUnitsForeground.clear();

            // Segment the foreground regions and generate boolImage to extract from background.
            segmentForeground(scratchPercepUnitsForeground, foreground, frame);

        }

        frameCount++;

        getrusage(RUSAGE_SELF, &usage);
        cout << "DEBUG leakTest_bug_report " << i << " " << usage.ru_maxrss/1024.0 << endl;
    }

    return 0;
}

If you use the images available here, you will find that the program's memory usage increases, and it seems to increase with the number of foreground contours. Since I'm clearing my storage (scratchPercepUnitsForeground) for each frame, I don't ... (more)