Ask Your Question

Converting a 2D image point to a 3D world point

asked 2014-07-25 09:47:57 -0600

Ilan gravatar image

I'm developing application for iOS. I'm using the camera matrix according to the book Mastering OpenCV. In my scenario I have a well known box. I know its real dimensions and I know exactly its corner's pixels. Using this information I calculate the camera rotation and the translation vector. From these parameters I'm able to calculate the camera position. I'm checking my calculation by projecting the 3D world coordinate back to the image and I get very accurate results.

The world origin in my case is the middle of the bottom line of the box. The box is open from one side. The image is taken in that direction, so I can see the content of the box.

Now, I have object in the box. I know very well image coordinate (2D) of the corners of this object. I know the real hight of the corner (the real Y and Y <> 0). How do I calculate the world X and Z of the corners of the object.

In this question in the answer by bjoernz he said: "All you can do with the matrices that you have, is to transform a 2D pixel into a 3D line where every point on this line would be projected onto the same 2D pixel.

You will definitely need additional information to reconstruct a 3D point."

I have the real Y (height of the object (40mm)). Also my reference object (the Box) is in the same image.

Here my code:

include "opencv2/core/core.hpp"

include "opencv2/imgproc/imgproc.hpp"

include "opencv2/calib3d/calib3d.hpp"

include "opencv2/highgui/highgui.hpp"

include <iostream>

include <ctype.h>

using namespace cv; using namespace std;

Point2f point; vector<vector<point2f>> objectPoints(1); vector<vector<point2f>> boxPoints(1);

Point3f calc3DPointOutOf2DwithYknown(double u, double v, float worldY, double fx, double fy, double cx, double cy, Mat tvec, Mat rotMat) { Point3f tmpPoint;

// This fiunction I need to complete
return tmpPoint; }

int main( int argc, char** argv ) {

///////// Loading image
Mat sourceImage = imread("/Users/Ilan/Xcode/LK Test/LK


namedWindow( "Source", 1 );

///// Setting box corners /////
point = Point2f((float)102,(float)367.5);

//640X480 boxPoints[0].push_back(point); circle( sourceImage, boxPoints[0][0], 3, Scalar(0,255,0), -1, 8);

point = Point2f((float)83,(float)90.5);

//640X480 boxPoints[0].push_back(point); circle( sourceImage, boxPoints[0][1], 3, Scalar(0,255,0), -1, 8);

point = Point2f((float)520,(float)82.5);

//640X480 boxPoints[0].push_back(point); circle( sourceImage, boxPoints[0][2], 3, Scalar(0,255,0), -1, 8);

point = Point2f((float)510.5,(float)361);

//640X480 boxPoints[0].push_back(point); circle( sourceImage, boxPoints[0][3], 3, Scalar(0,255,0), -1, 8);

///// Setting object corners /////
point = Point2f((float)403.5,(float)250);

//640X480 objectPoints[0].push_back(point); circle( sourceImage, objectPoints[0][0], 3, Scalar(0,255,0), -1, 8);

point = Point2f((float)426.5,(float)251.5);

//640X480 objectPoints[0].push_back(point); circle( sourceImage, objectPoints[0][1], 3, Scalar(0,255,0), -1, 8);

imshow ...
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2014-08-05 18:26:54 -0600

Ilan gravatar image

I succeed to solve it by myself. If it will help to any one, heres the code:

Point3f calc3DPointOutOf2DwithYknown(double u, double v, float worldY, double fx, double fy, double cx, double cy, Mat tvec, Mat rotMat)
    Point3f tmpPoint;

    float r1 =<double>(0,0);
    float r2 =<double>(0,1);
    float r3 =<double>(0,2);

    float r4 =<double>(1,0);
    float r5 =<double>(1,1);
    float r6 =<double>(1,2);

    float r7 =<double>(2,0);
    float r8 =<double>(2,1);
    float r9 =<double>(2,2);

    float t1 =<double>(0,0);
    float t2 =<double>(1,0);
    float t3 =<double>(2,0);

    float xt = (u/fx) - (cx/fx);
    float yt = (v/fy) - (cy/fy);

    float K1 = xt*r8*worldY + xt*t3 - r2*worldY - t1;
    float K2 = xt*r9 - r3;
    float K3 = r1 - xt*r7;

    float worldZ = (yt*r7*K1 + yt*K3*r8*worldY + yt*K3*t3 - r4*K1 - K3*r5*worldY - K3*t2)/
                    (r4*K2 + K3*r6 - yt*r7*K2 - yt*K3*r9);

    float worldX = (K1 + worldZ*K2)/K3;

    tmpPoint = Point3f(worldX, worldY, worldZ);

    return tmpPoint;
edit flag offensive delete link more


How can I estimate the worldY from an image if I don't have a actual value of worldY. Can I solve this using two stereo images ?

abhinavcoder gravatar imageabhinavcoder ( 2017-05-23 04:48:20 -0600 )edit

Hi Ilan, would you be able to elaborate on your code a bit more so its easy to understand. Especially what the exact paramters are and mean. Would be highly appreciated! Cheers.

pdsiedler gravatar imagepdsiedler ( 2018-04-17 10:45:49 -0600 )edit

@psiedler, please do not post answers here, if you have a question or comment, thank you.

berak gravatar imageberak ( 2018-04-17 10:49:59 -0600 )edit

Question Tools

1 follower


Asked: 2014-07-25 09:47:57 -0600

Seen: 12,011 times

Last updated: Aug 05 '14