optical flow orientation and magnitude based background and foreground segmentation

edit

How can I segment background and foreground properly using optical flow for moving object detection. In my case object is moving and camera mounted on car is also moving.First I consider that whichever is moving in the opposite direction of camera is 'background' and in the direction of car is 'foreground'. Now we have optical flow X and Y values , based on this angle is calculated and after analyzing these values background is subtracted.

1. There is a chance that background contains some foreground object as noise.
2. There is a chance that foreground contains some background as noise.

Now based on optical flow magnitude, I can filter out these noise at some level. In case of background noise , object speed will be always higher than background speed.So this can be consider to remove noise.

But How can I deal the case in foreground using optical flow values when in different cases, when car speed is equal to object speed , car speed is less than object speed and car speed is higher than object speed? In all these case object position/speed varies.

Any help will be appreciated.Thanks.