# Real world coordinates from a pixel

Hi I'm new in opencv and I have a problem: I don't know which are the steps to obtain the pose an orientation (in the real world) to the one pixel of my image. I read about it but it is not clear for me. I have a calibrated camera stereo and I get the pixel coordinate of interest, but I don't know how to continue. Can anyone help me? Thanks!!

edit retag close merge delete

Sort by » oldest newest most voted

One very utilized method is to have a reference point in your scene which you can detect easily. This reference point is your world reference and has coordinates (0,0,0). Then, with your X, Y and Z pixel information, you can determine the real world coordinates relatively to your reference point.

This obviously only works for still setups which can have a known reference object. Either way, you'll have to always make assumptions in order to derive real world coordinates.

more

I can't use a reference point because the stereo are mounted in a robot arm. I read about other solution that use the opencv function reprojectImageTo3D() but I need the disparity array and Q matrix, but I'm not sure if I'm on the right direction.

( 2015-01-07 03:04:30 -0500 )edit

Can you tell me more about the application? Do you want to know the height of an object in order to give the right commands to pick it up with a claw?

( 2015-01-07 04:57:34 -0500 )edit

I have a work piece with holes and I want to determine where are the holes to move the robot near it.

( 2015-01-07 05:52:41 -0500 )edit

I once did a project where I had to detect coloured cubes and stack them accordingly with a robotic arm. What I was doing was moving the arm in order to make a 90º angle between the camera and the working top. Then I'd detect the object and move the arm until the centre of the object was in the centre of the image, at which point I would compensate the offset between the camera and the centre of the tool. I didn't use any absolute coordinates, just incremented movement to the claw until It was aligned with the object. If you can detect those holes you can also use this method. Check it out https://www.youtube.com/watch?v=yGOz2...

( 2015-01-07 06:06:22 -0500 )edit

Nice work!! But how do you determine the depth?

( 2015-01-07 06:39:45 -0500 )edit

The height of the working top and the cubes was known.

( 2015-01-07 06:46:43 -0500 )edit

OK. In my application the work piece is not always the same. So I need XYZ coord of the pixel.

( 2015-01-07 08:13:34 -0500 )edit

You told me that you had a working stereo set. With that, you can calculate the distance of your tool to the surface, and then calculate the height of your work piece. For this you'll need to have a plane model computation at the initialization phase. Define the working top plane at the beginning, and then calculate the height of your work pieces relatively to the plane

( 2015-01-07 08:49:06 -0500 )edit

Official site

GitHub

Wiki

Documentation