Ask Your Question

Overlay in video after detection

asked 2014-10-21 03:34:13 -0500

Samjakar gravatar image

Hi, I am using cascade classifier (LBP) to identify a logo in a video. The video has focus on a brand in different zoom levels. I want to be replace this logo with a different image. Depending on the success rate of the logo match, I am able to achieve this by copying the image to the matched region as detected by the cascade classifier. The problem is that since I am replacing the ROI with my image on every frame, the modified video has a very jittery look (Since the matches has slightly varying bounding box). Is there any standard approach to address this? I am sure there are hundreds out there who have faced this issue. So, trying to get some ideas here.


edit retag flag offensive close merge delete


I think that you can do some kind of distortion detection and transform the image that you are using for replacing the ROI to seems more likely the one in the real video, but I have no idea how to do it, I think with some features and based on their places related to the others.

thdrksdfthmn gravatar imagethdrksdfthmn ( 2014-10-21 04:10:29 -0500 )edit

Hmm that 'jitter' is indeed caused by the cascade classifier detector, to solve this problem first store all your detections. Then perform temporal smoothing, which means you average detections over a loop of time/frames. That will reduce the errors drastically.

StevenPuttemans gravatar imageStevenPuttemans ( 2014-10-21 06:33:22 -0500 )edit

Hi Steve, Thanks for your suggestion. After posting this questions, I started buffering my video for x-frames (say amounting to 10 secs , I then find the average of the ROI for the x frames and then overlay in that. This reduces the jitteriness for that x frames.. but, then I start buffering from x+1 to 2x frames and overlay, the transition from the first to the second clip shows an obvious JUMP as the ROI of the (x+1)th frame is higher than the average of the first x frames. So, trying to see how better to smooth that out.. In the source video, the camera zooms in on my ROI. So, the matching rectangle size keeps increasing.In general, using opencv, if I overlay an image (that zooms/grows every frame), this behavior is expected? Would using OPENGL to render the overlay image be any better?

Samjakar gravatar imageSamjakar ( 2014-11-19 02:24:46 -0500 )edit

The solution beneath will solve that!

StevenPuttemans gravatar imageStevenPuttemans ( 2014-11-19 05:47:17 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2014-10-22 16:57:22 -0500

updated 2014-10-23 03:42:57 -0500

You can also use a Kalman filter on your bounding box in order to "smooth" the positions/size of your ROI. That way, you don't have to process the entire sequence to produce the output, you can do it in real-time.

If you don't want to use Kalman, you can also take the previous ROI and mix it with the new ROI:

newROI.x = prevROI.x * 0.3 + newROI.x * 0.7;
newROI.y = prevROI.y * 0.3 + newROI.y * 0.7;
newROI.width = prevROI.width * 0.3 + newROI.width * 0.7;
newROI.height = prevROI.height * 0.3 + newROI.height * 0.7;

Of course, 0.3 and 0.7 can be adjusted as you want (sum should be equal to 1...).

In case you want to use Kalman, you should first init the filter (where ROI is a cv::Rect):

cv::KalmanFilter kalFilter;
kalFilter.transitionMatrix = (Mat_<float>(8, 8) <<
kalFilter.statePre.setTo(0);<float>(0) = static_cast<float>(ROI.x);<float>(1) = static_cast<float>(ROI.y);<float>(2) = static_cast<float>(ROI.width);<float>(3) = static_cast<float>(ROI.height);
setIdentity(kalFilter.processNoiseCov, Scalar::all(1e-4));
setIdentity(kalFilter.measurementNoiseCov, Scalar::all(1e-1));
setIdentity(kalFilter.errorCovPost, Scalar::all(.1));

Then, at each new frame, you will filter the new positions:

Mat values = (Mat_<float>(4,1) << newROI.x, newROI.y, newROI.width, newROI.height );
kalFilter.correct(values);//update Kalman...
Mat kalmanEstim = kalFilter.predict();//get the filtered position
filteredROI.x = static_cast<int>(<float>(0));
filteredROI.y = static_cast<int>(<float>(1));
filteredROI.width = static_cast<int>(<float>(2));
filteredROI.height = static_cast<int>(<float>(3));

I hope you will find this useful!

edit flag offensive delete link more



Hi petititi, Thanks for your suggestion. I have not used KalmanFilter. I will try it out and will let you know . Thanks

Samjakar gravatar imageSamjakar ( 2014-11-19 02:27:42 -0500 )edit

Question Tools


Asked: 2014-10-21 03:34:13 -0500

Seen: 846 times

Last updated: Oct 23 '14