# Pose estimation with SIFT and homography

Hello guys,

I am doing an application of object recognition using SIFT technique.
Now I would be able to estimate its 3d pose. Basically I would like to draw X,Y,Z axis.

Using homography i believe i have already its X and Y axis (by drawing the bounding box with homography and perspectiveTrasform i choose the first and second line to be X and Y axis)

Now what I need is the Z axis. Do you have any hint?
I was thinking I just need to find the line orthogonal at the plane formed by the first 2 line that i have already found with homography. Basically in this pic the first 2 green line could be the X and Y axis, at this point i just need to draw its Z axis..

edit retag close merge delete

Sort by » oldest newest most voted Simple way is using cross product of X and Y vectors. So if you know camera parameters and homography coefficients you can calculate all 3 vectors (x,y,z) analytically.

more

I mean that you can use plane 3d object as model of your journal. Then you can find R and T for your 3d object from homography. Then using calculated R and T you can calculate new 3d coordinates for all vectors X, Y, Z and coordinates of their projections to the image plane.

Yea cross product may be the right answer! But do I really need camera parameters? The bounding box calculated with homography isn't alone sufficient to calcuate Z axis?

@Alexander: do you have any docs I could read to use camera parametrs and homography to calculate all vectors? I think you should be a little more specific in your question.

If your asking how to obtain the rotation and translation (extrinsic parameters) of the camera using an homography, this problem is known as Homography Decomposition. For instance, if you want to insert an augmented object in your book, you will need these parameters (simply computing the perpendicular vector to X and Y will lead to errors).

Notice that to estimate the R and t from the homography you will need the instrinsic parameters of the camera, which can obtained using the calibration method present in OpenCV. These parameters will remain the same for all sequences of the camera.

Finally, you need to implement a couple of equations to extract the extrinsic parameters, they are well described in this great paper by Zhang:

A Flexible New Technique for Camera Calibration

Of course, this depends in what you really want to do in your application. I hope this can help.

more

Thanks for the suggestion! Also can you be more explict when you say "computing the perpendicular vector to X and Y will lead to errors" ?

I meant that depending on what your need to do is not that simple. As I mentioned, if your objective is to insert a virtual object on the book, you will need to extract the extrinsic parameters as I told.

@gfhur: no i don't need to insert a virtual object, i just want to draw those axis :)

You still need to recover K, R and t, otherwise how can you recover/project the Z axis with only the homography? Notice that an homography is a transformation between planes (in your case, the book cover and the image plane). In addition to the paper of Zhang in my answer, you can also look to this survey: http://cvlab.epfl.ch/~lepetit/papers/lepetit_ftcgv05.pdf (section Mathematical Tools) which can help you with the problem.

Thanks for all the info. What I would achive is something like this: http://www.youtube.com/watch?v=ZDNH4BT5Do4 But don't you think with the homography in my pic (in the question) I have already the first 2 axis? (X and Y) ? I have drawn manually the axis in my pic: http://i46.tinypic.com/n1th54.png isn't it just orthogonal to X and Y ? If I am not mistaken, one way for you to get the Z axis is to find a perpendicular vector to X axis, or Y axis. Here is a handy Java code to find a perpendicular vector.

// Point pt0, pt1: define the line you want to find a perpendicular vector from.
// Point intersect: is the point of intersection between that line and the perpendicular vector.
// double vectorLength: the lenght in pixels of perpendicular vector.

double angle = Math.PI/2; // 90 degrees
double perpendicularCos = Math.cos(angle); // or just use '= 0' because angle is 90º
double perpendicularSin = Math.sin(angle); // or just use '= 1' because angle is 90º
double d = pointsDist(pt0, pt1); // distance in pixels between 2 points.

// (x,y) define a vector of length vectorLength perpendicular to (pt0,pt1)
double x = intersec.x - ((pt1.x - pt0.x) * perpendicularCos - (pt1.y - pt0.y) * arrowSin) * (vectorLength / d);
double y = intersec.y - ((pt1.y - pt0.y) * perpendicularCos + (pt1.x - pt0.x) * arrowSin) * (vectorLength / d);

more

well i believe i need perpendicular vector both to X and Y axis, like it was a 3d space

Official site

GitHub

Wiki

Documentation