Calculating the tilt in pixels
I have a working object detection&tracking system already implemented in OpenCV. It detects, tracks and maps the objects in the world frame. However, since my camera is mounted on a moving vehicle, it tilts and therefore the coordinates of the tracked objects are changing drastically in the image frame, which results in wrong mapping of objects in the world frame.
Is there a way to understand how much the camera tilts between frames, so that I can add or subtract the amount of tilting for the coordinates of the detected objects?
Things to keep in mind:
- I do not have a gyroscope or anything like tilt-sensor, I cannot make use of additional hardware. I am looking for a vision solution.
- I am using only single camera (No stereovision).
What I tried so far:
- I have my horizon line detected, I tried to take that as a reference point, but horizon also tilts, therefore this did not work.
- Same goes for keypoints, their coordinates change when the tilt happens, this did not work either.
Thanks in advance.
have you try to estimate speed with keypoints? If yes when your camera tilt have you got noise in your speed estimation?
My Kalman filter works with the center of the detected object's bounding box. My problem is not with Kalman filter actually, it works just fine. The stuff which I want is, when the camera tilts, the center of bounding boxes should be updated accordingly to the tilt.
If you estimate speed v(t) with keypoints you can derivate v(t): v(t1)-v(t0). I think that when your camera tilt then this difference become high. Your vehicule cannot give you such acceleration that's your tilt. Of course when your vehicle climb I don't think that such method can detect tilt like a tilt sensor. May be your kalman filter smooth too much your speed and a simple method may be give you more about your tilt
My vehicle is a boat sailing on the sea surface, so there is no climbing. However the tilt happens all the time. And if the sea is wavy, the tilt becomes even higher. The thing is not only understand whether there is tilt or not, but also how much we tilt.
I understand your problem. I think you have to make some experience with your system. Use a tilt sensor and acceloremeter gyroscope... and accumulate data. After I hope you can find some relation between your speed estimation and tilt angle.
I am not allowed to modify the hardware of the boat, it does not belong to me and I am supposed to solve the problem using a vision approach. Thanks though.
It's only an experience . But may be you use a good smatrphone with all sensors on a another boat and make a video and off line analyse data to find a relation between your speed estimation in your tilt.