Nghia's profile - activity

Hi all,

I have this issue where I'm trying to read frames from a H.264 video as fast possible. The problem is that the read speed seems to be capped to the frame per second of the video (30fps), which I would not expect to be so. My machine is capable of playing the video much faster, tested it with mplayer -fps 120 and it ran fine.

The second is if I pause the frame grabbing for a few seconds and try again it reads much faster, as if it's trying to catch up for lost time, then returns back to the original frame rate. Here's the code I used for testing

VideoCapture vid("video.avi");
Mat img;

// First 300
chrono::time_point<std::chrono::system_clock> start, end;
start = chrono::system_clock::now();

for(int i=0; i < 300; i++) {
    vid.read(img);
}

end = std::chrono::system_clock::now();
int elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(end-start).count();
cout << "First 300 frames - elapsed " << elapsed <<  " ms" << endl;

// Wait a bit ...
cout << "Sleep 10 seconds" << endl;
sleep(10);


// Next 300
start = std::chrono::system_clock::now();

for(int i=0; i < 300; i++) {
    vid.read(img);
}

end = chrono::system_clock::now();
elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(end-start).count();
cout << "Next 300 frames - elapsed " << elapsed << " ms" << endl;

The output is

First 300 frames - elapsed 9996 ms
Sleep 10 seconds
Next 300 frames - elapsed 592 ms

The default border handling scheme is reflection (BORDER_REFLECT_101). You want to pass borderType = BORDER_CONSTANT. But for some reason filter2D doesn't let you specify the constant colour to use according to the doc. So not sure if it'll work. In the worse case you can make a slightly larger image, pad the borders with zeros, run the function and get the subimage.

cv::Rect(x1, y1, x2-x1, y2-y1) should be cv::Rect(x1, y1, x2-x1+1, y2-y1+1). Might not be the problem you are facing though ...

Put an isnan(g[i][j]) check in your for loop.

Since you only got 2 samples. You'll probably need to edit the default parameters to make it work. Try CvDTreeParams param; param.min_sample_count = 1. Pass param to your train() call.

If you can determine the first border pixel accurately eg. occurs near (0,0) then you might be able to

1. floodFill all black pixels starting at (0,0) or where ever, generate a binary image such that 0 is for border pixels, and 1 is for everything else
2. use findContour + boundingRect to find the bounding box of the inside pixels. The findContour is very fast, it only visits pixels on the edge (not inside like a floodFill). findContour will return an array of possible contours, pick the largest one.
3. crop the borders out using the bounding rect of the non-border pixels

Your last 2 checks look rather strange. Just do

if (roi_.x + roi_.width + 10 < img.cols) 
    roi_.width += 10;

if (roi_.y + roi_.height + 10 < img.rows) 
    roi_.height += 10;

Try CV_ROW_SAMPLE instead of CV_COL_SAMPLE.

Feel free to do a github pull request :)

Hang on, just so we're on the same page. winSize is the size of the "patch" it uses to do the search, not the search bounds in anyway. I can see where the confusion might be coming from.

I'm going to go with "yes" from my theoretical understanding. I can think of a very contrive example. Imagine a 1D image img = [0 1 2 3 4 5 6 7 8 .... 255]. Your patch is [253 254 255]. If you start searching at index 0 in img, then it will do a gradient descent all the way to the end because the error keeps reducing as you slide along. So that tiny patch just did a very big search. In practice, there's a fixed limit on how many iterations it performs so it won't go too far. I once tried implementing a basic Lucas-Kanade optical flow and have seen infinite loops (oscillation) during the iterations.

The Lucas-Kanade optical flow algorithm uses an iterative (gradient descent like) method to do its search. There's no fixed bound on the search space like in block matching. If it fails to converge correctly then it is possible for it to find a match very far away.

Have a look at the values of err, that might be a more reliable way to filter out bad matches, in addition to using status.

Also, keep in mind that Lucas-Kanade assumes a very small optical flow displacement eg. ~1 pixel, hence the need for pyramid. I'm surprise it works at pyramid level 0 at all.

That's a very low res video, might be hard to work with. Do you have one that is a bit higher resolution?

What I meant was the video you use for testing, the file that you load in. Would you be able to upload the video file, or is too big?

features_found is a vector containing "1" for good, "0" for bad. Would you be able to post the input video you use? That way we can see what's going on.

Can you post a pic of what you want? It is a bit unclear from your description.

As a first step I would confirm that you are getting good optical flow by drawing them on the screen.

As a second step I would remove bad optical flow vectors before passing them into estimateGlobalMotionRobust. Just loop through features_found and keep the good ones. This may or may not not be necessary, but if you can make life easier for the function then why not.

Transpose the matrix before adding the row?

input.channels() for number of channels

input.type() is one of CV_8U, CV_32F, CV_64F ...

I'm hoping someone knows an easier way. input.type() just returns an int, which means you have to look it up. It would be nice if there's a version that outputs a string, for easier debugging.

In my experience you can get away most of the time with just tracking the binary blobs with a Kalman filter. If blobs merge together to form a big one, you tell the Kalman filter to not track the big blob and switch to prediction. If the blobs separated as predicted then you lock onto them again.

If there is a fragmentation it would come from the use of OpenCV clone(), since that does a deep copy and allocates new memory.

Forgot you there :) Just had a thought. I'm going to try and run the loop for much longer see if it really does keep eating up more memory. The other is the possibility of memory fragmentation, due to lots of new and delete.

The project explicitly says it is released under GNU GPL v3. So you're a bit out of luck if you want to use it for a commercial project.

There is one circumstance I know where you can mix GNU GPL with a commercial product. If your commercial program calls a GPL executable eg. via system("gpl.exe");, then this is OK (at least back with the original GPL).

See if it opens in GIMP. If it does, re-save it as a bitmap (export to). There's a small chance it has a weird bitmap header.

I think you're on the right track, or at least the same as my interpretation. It appears the request is for something like:

Matx33f a = Matx33f::ones();
Vec3f b = Vec3f::ones(); <--- currently does not work, needs to be implemented

error: conversion from ‘cv::Matx<float, 3, 1>’ to non-scalar type ‘cv::Vec3f {aka cv::Vec<float, 3>}’ requested
     Vec3f b = Vec3f::ones();

I hope you succeed, it looks useful!

It turns out for cvtColor the destination matrix will always be allocated new memory so it will never point to the same data as the source. I was curious so I wrote a short test code:

void DoStuff(const Mat& img)
{
    Mat gray = img;

    cvtColor(img, gray, COLOR_BGR2GRAY);

    cout << "    gray: " << gray << endl;
    cout << "    img.data: " << (void*)img.data << endl;
    cout << "    gray.data: " << (void*)gray.data << endl;
}

int main()
{
    Mat img = Mat(1,1,CV_8UC3);

    img.at<Vec3b>(0)[0] = 0;
    img.at<Vec3b>(0)[1] = 100;
    img.at<Vec3b>(0)[2] = 200;

    cout << "img before: " << img << endl;

    DoStuff(img);

    cout << "img after: " << img << endl;
}

results in

img before: [0, 100, 200]
    gray: [118]
    img.data: 0x14eeb10
    gray.data: 0x14eeb40
img after: [0, 100, 200]

It was advantageous back in the old CPU days, probably not so much now. Here is a good post on it

http://stackoverflow.com/questions/2185944/why-must-stride-in-the-system-drawing-bitmap-constructor-be-a-multiple-of-4

Probably obvious, but are you trying to track blobs from MoG?

On Linux, in /usr/lib/gcc/x86_64-linux-gnu/4.8/include/float.h it has

#define FLT_EPSILON __FLT_EPSILON__

However if you try to grep for __FLT_EPSILON__ you won't find anything. Somehow gcc just knows.

cout << FLT_EPSILON << endl;

returns

1.19209e-07

It's a standard #define. On Linux, math.h pulls it in.

math.h:#  define MAXFLOAT   3.40282347e+38F
values.h:#define    MAXFLOAT    FLT_MAX

I'm leaning towards a wording bug in the doc because it makes sense to return 3 channels if your input is 3 channels.

Mat img = Mat(1,1,CV_8UC3);

img.at<Vec3b>(0)[0] = 0;
img.at<Vec3b>(0)[1] = 100;
img.at<Vec3b>(0)[2] = 200;

cout << (img != 0) << endl;

gives

[0, 255, 255]

They're already included in the OpenCV libs, no need to copy them over.

There's a degenerate case with the 8 point algorithm if all/most of the points lie on a single planar surface eg. a book, table, wall.

One common way that people check this is by finding the homography between the points and observe the error. If the error is relatively low (eg. < 3 pixels) then the points lie too close to a plane and are unstable to work with.

Your data is heavily imbalance so it puts more weight on the one with more data. Try to re-balance them by either reducing the bigger class, or inflate the smaller class (via duplication).

What kind of motion undergoes between the two images?

I'm a bit more familiar with the YAML format, but the important info to look for are: depth (tree depth), var (feature used to split on, index to your array), le (less than or equal), value (the value it is comparing to). If the current value is "less than or equal" to "value" the it branches left, else right. Helps to sketch it on paper.

Try saving the random forest out to an xml/yaml file. It won't be pretty to process manually by hand but it's do-able, at least for a single tree.

Check the values in mult eg. cout << mult << endl. Also, I noticed you're generating a random matrix with a given standard deviation of doubleSpinBox_sDev, then multiplying again by the sqrt. Why the extra multiplication by the sqrt?

You'll probably want to include c++ header rather than the old C API. You can do the lazy #include <opencv2/opencv.hpp>, which will include practically every header you ever need, instead of individual headers if you prefer.

How is (*(distIter->unit)) freed? I don't see any destructor or any delete code.