Ask Your Question

Translating from camera space, into see through display space

asked 2019-06-25 14:48:25 -0500

eric_engineer gravatar image

I'm new to image processing and opencv, so sorry if this is the wrong place for this question. I have a head mounted camera taking video that I am doing some object detection on. Then I want to draw the outline of the detected object on my see through eye mounted display. Now I see some obvious problems with this. First of course the camera is out of x,y alignment with my eye. I don't think moving it would be too hard.

But then the field of view of the camera and my display are both different, so I'm thinking I have to crop the camera to the smaller FOV of the display. I'm wondering if I can just use the ratio of the two FOVs for this?

Am I missing something? Is there a prefered or right way to come up with this translation?

Thank you for your time!

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2019-06-25 17:53:01 -0500

Tetragramm gravatar image

The short answer is, you can't do this properly, but you can fake it.

Long answer is, because your camera isn't aligned with your display, you need to know how far away to draw the object. Any single pixel in the camera, when drawn at different ranges in the display, makes a line across it. See the tutorial HERE.

You can fake it by picking a range to draw at. You take your camera, and the point you wish to draw. "Push" the point into 3d space, then project it back onto the display. If you had the real range from some source, you would use that.

To do this, you need to calibrate the camera, and the hypothetical camera that is the eye. You have to know the relationship (IE: rvec, tvec) between them.

You can fake that by scaling the FOV and translating, but unless what you're looking at is really far away, the camera is perfectly aligned with "front" and neither is very distorted, it probably won't look right.

edit flag offensive delete link more

Question Tools

1 follower


Asked: 2019-06-25 14:48:25 -0500

Seen: 89 times

Last updated: Jun 25 '19