calcOpticalFlowPyrLK doesn't respect window size? [bug ?]

asked 2014-02-09 16:39:43 -0600

yes123
1412 ●12 ●28 ●52

updated 2014-02-10 12:21:33 -0600

I am trying to use optical flow with opencv 2.4.5: calcOpticalFlowPyrLK()

I am using similar parameters to the demo of the opencv:

    Size winSize(6,6);     // Small window size
    calcOpticalFlowPyrLK(
        prevFrame, currentFrame, points[0], points[1], 
        status, err, winSize,
        0, termcrit, 0, 0.001             // Only the input image
    );

But as you can see I have set the maxLevels to 0 (so it will process only the current layer).

The problem is that this function sometimes returns a distance between points of over 100, if I do:

for(int i=0;i<points[1].size();i++) {
 if( !status[i] ) {                     
     continue;
  }

 cout << points[1][i] - points[0][i] << endl;

}

This sometime prints:

(53,120)

Meaning that y has a difference of over 120 pixels. Is that normal with a window size of only (6,6)? Note that this happens even if i give more layers in the pyramid.

Note that this happens especially when a point gets occluded or goes out of the scene

edit retag flag offensive close merge delete

add a comment

answered 2014-02-10 02:37:51 -0600

Nghia

1000 ●5 ●16

The Lucas-Kanade optical flow algorithm uses an iterative (gradient descent like) method to do its search. There's no fixed bound on the search space like in block matching. If it fails to converge correctly then it is possible for it to find a match very far away.

Have a look at the values of err, that might be a more reliable way to filter out bad matches, in addition to using status.

Also, keep in mind that Lucas-Kanade assumes a very small optical flow displacement eg. ~1 pixel, hence the need for pyramid. I'm surprise it works at pyramid level 0 at all.

edit flag offensive delete link

Comments

Nghia! Thanks for the answer! So when you write "There's no fixed bound on the search space like in block matching." do you mean that the parameter winSize has not effect at all ?

yes123 ( 2014-02-10 06:51:45 -0600 )edit

I'm going to go with "yes" from my theoretical understanding. I can think of a very contrive example. Imagine a 1D image img = [0 1 2 3 4 5 6 7 8 .... 255]. Your patch is [253 254 255]. If you start searching at index 0 in img, then it will do a gradient descent all the way to the end because the error keeps reducing as you slide along. So that tiny patch just did a very big search. In practice, there's a fixed limit on how many iterations it performs so it won't go too far. I once tried implementing a basic Lucas-Kanade optical flow and have seen infinite loops (oscillation) during the iterations.

Nghia ( 2014-02-10 07:03:54 -0600 )edit

Nghia: you are right but don't forget that along with [253 254 255] you would pass its position in the frame i-1. (In the example points[0]). So it would be easy to start from that position and end the iteration once winSize is greater than your argument...

yes123 ( 2014-02-10 12:06:43 -0600 )edit

Hang on, just so we're on the same page. winSize is the size of the "patch" it uses to do the search, not the search bounds in anyway. I can see where the confusion might be coming from.

Nghia ( 2014-02-11 01:09:02 -0600 )edit

@Nghia! you got me there ! I thought winSize would be the max size to look for the new point! Thanks! I believe this should be clarified in docs: http://docs.opencv.org/modules/video/doc/motion_analysis_and_object_tracking.html

yes123 ( 2014-02-11 06:04:10 -0600 )edit

Feel free to do a github pull request :)

Nghia ( 2014-02-11 06:16:34 -0600 )edit

@Nghia I am using the Optical Flow algorithm to track a movement of human pupil. I did some tests with changing the winSize to see how the result changes accordingly. I saw that at a certain value of winSize, the result becomes best, but this is just a statistical job. Do you have any idea how to select a suitable winSize (i.e. based on object size...)?

harry52 ( 2016-11-23 03:44:12 -0600 )edit

add a comment

calcOpticalFlowPyrLK doesn't respect window size? [bug ?]

1 answer

Comments

Links

Question Tools

Stats

Related questions

calcOpticalFlowPyrLK doesn't respect window size? [bug ?] edit

1 answer

Comments

Links

Question Tools

Stats

Related questions

calcOpticalFlowPyrLK doesn't respect window size? [bug ?]