# Real world camera position knowing marker geometry

Dear fellow developpers,

I am an embedded system engineer, and not so familiar with augmented reality and video techniques (but I've already used openCV).

I would like to get some advice on the general technique to use. I see a lot of different techniques and tutorials : object detection, target tracking, calib3D... and would like some advice on the path to follow, because it seems that a lot of different techniques can be used to achieve what I want to do.

The goal of this project is to get the 3D vector from the camera point to the center point (0,0,0) in real word units, with maximum reachable precision.

Four markers are set on the ground (z=0). The markers are basically A4 paper sheets with checker pattern printed on. - Known parameters : dimensions of the markers (lenghts A and B) + positions of the markers relatively to each other (lengths M and N). - The camera can move around three axis. - Due to mechanical constraints I cannot directly place a marker on the center point.

• The camera calibration is done using calib3D.

Here are a few questions :

• What's the best tool in OpenCV to achieve this goal with maximum precision in near real-time ?
• Can I use solvePNP with a large amount of points (since all markers are checker pattern) ?
• Is it better to use different checker patterns (two pure checker pattern and the two others would be assymetrical round patterns) ?
• Do you recomment to use the calib3D stuff or template matching ?
• Are there any libraries that could simplify that task / implementations of this problem already existing ?

I have stumbled upon this : http://stackoverflow.com/questions/20... but this is a no-go for me as I need the physical coordinates and this is not the objective.

This is also interesting (regarding drones again) : http://diydrones.com/profiles/blogs/q... but there's no explanation. Both methods are also a no-go since I can't place a marker on the center spot.

edit retag close merge delete

Did you find a solution to determining the X, Y, Z coordinates of the marker?

( 2015-06-11 13:51:04 -0500 )edit

Sort by » oldest newest most voted

What's the best tool in OpenCV to achieve this goal with maximum precision in near real-time ?

SolvePnP is exactly what you want to use

Can I use solvePNP with a large amount of points (since all markers are checker pattern) ?

a) Why don't you just try? b) Yes

Is it better to use different checker patterns (two pure checker pattern and the two others would be assymetrical round patterns) ?

asymetric pattern have the large advantage that you don't have to figure out the rotation on a higher level so I'd suggest several circle patterns with different sizes (number of features)

Do you recomment to use the calib3D stuff or template matching ?

Are there any libraries that could simplify that task / implementations of this problem already existing ?

OpenCV already makes this task very easy and straight forward. You can use any calibration tutorial.

For solvePnP you need the coordinates of your features both in 3d (in your case in meters as (xi,yi,0) and in Pixel coordinates. The first feauture (top left corner of pattern 1) should have the coordinates (-B/2, M/2+A,0) and so on. In This way, solvePnP gives you the position relative to your center, even if you don't have a marker there. (Let me guess, you want to place an object in the middle, and reconstruct it in 3d using multiple views)

I'd suggest a code like (Pseudo-Python)

3d_points = []
2d_features = []
for i = 1:4
found, 2d_features = findPattern(size_of_Pattern_i)
if found:
3d_points.append(3d_positions_of_pattern_i)
2d_points.append(2d_features)

if len(3d_points)> 0:
print "at least one marker visible"
camPose = solvePnP(3d_features, 2d_features, IntrinsicCalibrationData)


The pattern detector has sometimes problems if there are multiple patterns. In this case you could try to mask out a part of your image. If you are tracking your camera (and therefore have a rough guess where it is located), project your patterns into the image and just look at there positions.

more

1

Thank you very much for this very precise response :-) ! I have very limited time to come up with a solution and I didn't want to start something in a dead end. Now you confirm me this is the way to go :-) ! Thank you very much for your help ! I'll comment here if I get any problems implementing this. Have a nice day :)

( 2015-01-09 13:23:30 -0500 )edit

thank you FooBar for this clarification! once i got the tvec and rvec, how can i use them to determine the real (X,Y,Z) position of the marker. and i found untresting stuff in the net documentation but i didnt't know the difference between "camera pose" and "camera position"

more

Did you find a solution to determining the X, Y, Z coordinates of the marker?

( 2015-06-11 13:50:00 -0500 )edit

Official site

GitHub

Wiki

Documentation