1 | initial version |
The Lucas-Kanade optical flow algorithm uses an iterative (gradient descent like) method to do its search. There's no fixed bound on the search space like in block matching. If it fails to converge correctly then it is possible for it to find a match very far away.
Have a look at the values of err, that might be a more reliable way to filter out bad matches, in addition to using status.
Also, keep in mind that Lucas-Kanade assumes a very small optical flow displacement eg. ~1 pixel, hence the need for pyramid. I'm surprise it works at pyramid level 0 at all.