Using calcOpticalFlowPyrLK with multiple user defined point

asked 2015-10-14 09:33:17 -0600

R5 gravatar image

updated 2015-10-14 09:35:47 -0600

Currently, I had use the cv2.moment to get the center point of the targets, however when I apply calcOpticalFLowPyrLK to track these point, the tracking result is not very good, sometime it doesn't even manage to track the point.

I had also try using cv2.goodFeaturesToTrack to get the point, with this function, the calcOpticalFLowPyrLK is able to track the point, however there are too many unwanted point.

Can anyone suggested me a way to overcome this problem?

edit retag flag offensive close merge delete

Comments

The unwanted points are points that belong to the object or are points that are detected on the whole image ?

I think that you should use the Lucas Kanade optical flow method to have the motion of the object and then deduce the center point of the object.

Note: the object you want to track should have some texture, if not I am not sure the optical flow will work on textureless surface.

Eduardo gravatar imageEduardo ( 2015-10-14 10:23:55 -0600 )edit

There are quite a lot of point that it detected, both the object and the whole image.

Does this mean that, I should try to applying the LK optical flow on those detected object first, and then try to find the center point?

And about the texture, will the video quality affect the texture? I'm currently testing with a pedestrian video I found on YouTube so the quality of the video is not that great.

R5 gravatar imageR5 ( 2015-10-14 11:08:13 -0600 )edit

You cannot use the optical flow method to track only the center point because first the center point is maybe not a good point to track (textureless zone) and it is better to average or use the median of the optical flow.

One possible solution is:

  • initialization: detect the GoodFeaturesToTrack on the whole image
  • get the current image and apply the Lucas-Kanade method on the previous and current images with all the points
  • calculated the displacement for lost objects (average the flow for the desired object) only for points that are on the objects in the previous image
  • you may also need to reinitialize the process when too many points diverge

I think that the video quality / the compression artifacts will affect the optical flow calculation.

Eduardo gravatar imageEduardo ( 2015-10-14 11:18:39 -0600 )edit

does the 3rd point means that, after getting the new location of the points for the object, I need to get the average of all these points by comparing it with the previous points in the object?

R5 gravatar imageR5 ( 2015-10-14 11:32:06 -0600 )edit

Yes, you have to calculate the displacement as nextPts is directly the new coordinates as you can see it in the documentation.

Also, there is an example file for the Lucas-Kanade lkdemo.cpp.

Eduardo gravatar imageEduardo ( 2015-10-14 12:17:40 -0600 )edit

Thank you, I'll check out the example

However is it ok if I use the python version as an example? which is the lk_track.py (I do notice it lack of some function compare to c++ version), since I don't have much knowledge on c++

R5 gravatar imageR5 ( 2015-10-14 12:24:27 -0600 )edit
add a comment