Estimate occluded marker's position based on its last known position and the leftover markers

asked 2015-07-17 07:09:42 -0600

noe gravatar image

updated 2015-07-17 07:23:18 -0600

Hi everyone,

I'm currently developing a marker based pose estimation application that calculates the pose of a flying kite based on a camera facing down along its rope. I'm using the Good features to track algorithm to detect the markers and use those inital positions to track them with the pyramidal LK algorithm to calculate the optical flow across frames. All that is working fairly well, however obviously the tracking fails when the marker is partly occluded by the kite's rope (worst case is the rope being falsly identified as the marker and thus I start tracking the rope instead...).

I am wondering if there is a way to estimate the partly occluded marker's position in the current frame based on the still visible markers for say 10 frames and then check if the marker is visible again within a certain region. If so the estimated position is replaced by the correct position (e.g. running Good features to track again in that region). How can I estimate the missing marker's position based on the known position of the other markers and the occluded marker's last known position? Is this possible using some sort of interpolation? Does openCV have anything to deal with this or are there other c++ libraries that I could use?

Any input whether it be a description of the necessary steps or libraries to code examples is welcome.

Thanks in advance!

edit retag flag offensive close merge delete


I think that you can use a Kalman filter to predict the position of the occluded markers. Another way I see is

  • to model the markers as a single object (each marker centroid has a 3D coordinate relative to the object frame)
  • after the tracking of each marker, you can calculate the pose of the camera
  • when a marker is occluded, you can estimate his projection on the image plane based on the pose calculated from the others tracked markers (there is a minimal number of points needed to estimate a pose)
Eduardo gravatar imageEduardo ( 2015-07-17 09:39:57 -0600 )edit