OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Sun, 04 Feb 2018 22:57:08 -0600Birds eye view perspectivetransform from camera calibrationhttp://answers.opencv.org/question/183753/birds-eye-view-perspectivetransform-from-camera-calibration/I am trying to get the bird's eye view perspective transform from camera intrinsic, extrinsic matrices and distortion coefficients.
I tried using the answer from [this][1] question.
The image used is the sample image left02.jpg from the opencv official github repo
[![The image to be prospectively un-distored left02.jpg image from opencv sample images i.e get the bird's eye view of the image][2]][2]
I calibrated the camera and found the intrinsic, extrinsic matrices and the distortion co-efficients.
I undistored the image and found the pose. To check if the params are right.
[![Image after un-distortion and visualising pose][3]][3]
The equations I used to find the perspective transformation matrix are (Refer the above link):
`Hr = K * R.inv() * K.inv()` where R is rotational matrix (from cv2.Rodrigues()) and K is obtained from cv2.getoptimalnewcameramatrix()
[ 1 0 | ]
Ht = [ 0 1 | -K*C/Cz ]
[ 0 0 | ]
Where `C=-R.inv()*T` Where T is translational vector from `cv2.solvePnP()`
and Cz is the 3rd component of the C vector
The required transformation is: `H = Ht * Hr`
The code I used to construct the above equation is:
K = newcameramtx # from cv2.getoptimalnewcameramatrix()
ret,rvec,tvec = cv2.solvePnP(world_points,corners2,K,dist)
R,_ = cv2.Rodrigues(rvec)
_,R_inv = cv2.invert(R)
_,K_inv = cv2.invert(K)
Hr = np.matmul(K,np.matmul(R_inv,K_inv))
C = np.matmul(-R_inv,tvec)
Cz = C[2]
temp_vector = np.matmul(-K,C/Cz)
Ht = np.identity(3)
for i,val in enumerate(temp_vector):
Ht[i][2] = val
homography = np.matmul(Ht,Hr)
warped_img =cv2.warpPerspective(img,homography,(img.shape[1],img.shape[0]))
# where img is the above undistored image with visualized pose
The resulting warped image is not correct.
[![With homographic matrix = Ht*Hr][4]][4]
If I remove the translation from the homography by using the below code
homography = Hr.copy()
warped_img =cv2.warpPerspective(img,homography,(img.shape[1],img.shape[0]))
I am getting the following image
[![With homographic matrix = Hr][5]][5]
I think the above image shows that my rotational part is correct but my translation is wrong.
Since the translational matrix (Ht) is an augmented matrix am unsure whether my construction of the above matrix is correct.
I specifically want to figure out the bird's eye perspective transformation from the camera calibration.
So, How do I correct the above equations so that I am getting the perfect bird's eye view of the chessboard image
Could anyone also please explain the math on how the above equations for Ht and Hr are derived? I don't have much exposure to Linear algebra so these equations are not very obvious to me.
**UPDATE:**
homography = np.matmul(Ht,Hr)
warped_img =cv2.warpPerspective(img,homography,(img.shape[1],img.shape[0]),flags=cv2.WARP_INVERSE_MAP)
cv2.WARP_INVERSE_MAP flag gave me a different result
[![][6]][6]
Still not the result I am looking for!
[1]: https://stackoverflow.com/questions/23275877/opencv-get-perspective-matrix-from-translation-rotation
[2]: https://i.stack.imgur.com/vUmcl.png
[3]: https://i.stack.imgur.com/mNLBy.png
[4]: https://i.stack.imgur.com/PnO3L.png
[5]: https://i.stack.imgur.com/bLlYD.png
[6]: https://i.stack.imgur.com/Y4GqK.pngThu, 01 Feb 2018 23:15:21 -0600http://answers.opencv.org/question/183753/birds-eye-view-perspectivetransform-from-camera-calibration/Comment by abhijit for <p>I am trying to get the bird's eye view perspective transform from camera intrinsic, extrinsic matrices and distortion coefficients. </p>
<p>I tried using the answer from <a href="https://stackoverflow.com/questions/23275877/opencv-get-perspective-matrix-from-translation-rotation">this</a> question.</p>
<p>The image used is the sample image left02.jpg from the opencv official github repo</p>
<p><a href="https://i.stack.imgur.com/vUmcl.png"><img alt="The image to be prospectively un-distored left02.jpg image from opencv sample images i.e get the bird's eye view of the image" src="https://i.stack.imgur.com/vUmcl.png"></a></p>
<p>I calibrated the camera and found the intrinsic, extrinsic matrices and the distortion co-efficients. </p>
<p>I undistored the image and found the pose. To check if the params are right. </p>
<p><a href="https://i.stack.imgur.com/mNLBy.png"><img alt="Image after un-distortion and visualising pose" src="https://i.stack.imgur.com/mNLBy.png"></a></p>
<p>The equations I used to find the perspective transformation matrix are (Refer the above link):</p>
<p><code>Hr = K * R.inv() * K.inv()</code> where R is rotational matrix (from cv2.Rodrigues()) and K is obtained from cv2.getoptimalnewcameramatrix()</p>
<pre><code> [ 1 0 | ]
Ht = [ 0 1 | -K*C/Cz ]
[ 0 0 | ]
</code></pre>
<p>Where <code>C=-R.inv()*T</code> Where T is translational vector from <code>cv2.solvePnP()</code>
and Cz is the 3rd component of the C vector </p>
<p>The required transformation is: <code>H = Ht * Hr</code></p>
<p>The code I used to construct the above equation is:</p>
<pre><code>K = newcameramtx # from cv2.getoptimalnewcameramatrix()
ret,rvec,tvec = cv2.solvePnP(world_points,corners2,K,dist)
R,_ = cv2.Rodrigues(rvec)
_,R_inv = cv2.invert(R)
_,K_inv = cv2.invert(K)
Hr = np.matmul(K,np.matmul(R_inv,K_inv))
C = np.matmul(-R_inv,tvec)
Cz = C[2]
temp_vector = np.matmul(-K,C/Cz)
Ht = np.identity(3)
for i,val in enumerate(temp_vector):
Ht[i][2] = val
homography = np.matmul(Ht,Hr)
warped_img =cv2.warpPerspective(img,homography,(img.shape[1],img.shape[0]))
# where img is the above undistored image with visualized pose
</code></pre>
<p>The resulting warped image is not correct.
<a href="https://i.stack.imgur.com/PnO3L.png"><img alt="With homographic matrix = Ht*Hr" src="https://i.stack.imgur.com/PnO3L.png"></a> </p>
<p>If I remove the translation from the homography by using the below code</p>
<pre><code>homography = Hr.copy()
warped_img =cv2.warpPerspective(img,homography,(img.shape[1],img.shape[0]))
</code></pre>
<p>I am getting the following image
<a href="https://i.stack.imgur.com/bLlYD.png"><img alt="With homographic matrix = Hr" src="https://i.stack.imgur.com/bLlYD.png"></a></p>
<p>I think the above image shows that my rotational part is correct but my translation is wrong. </p>
<p>Since the translational matrix (Ht) is an augmented matrix am unsure whether my construction of the above matrix is correct.</p>
<p>I specifically want to figure out the bird's eye perspective transformation from the camera calibration. </p>
<p>So, How do I correct the above equations so that I am getting the perfect bird's eye view of the chessboard image</p>
<p>Could anyone also please explain the math on how the above equations for Ht and Hr are derived? I don't have much exposure to Linear algebra so these equations are not very obvious to me. </p>
<p><strong>UPDATE:</strong></p>
<pre><code>homography = np.matmul(Ht,Hr)
warped_img =cv2.warpPerspective(img,homography,(img.shape[1],img.shape[0]),flags=cv2.WARP_INVERSE_MAP)
</code></pre>
<p>cv2.WARP_INVERSE_MAP flag gave me a different result</p>
<p><a href="https://i.stack.imgur.com/Y4GqK.png"><img alt="" src="https://i.stack.imgur.com/Y4GqK.png"></a></p>
<p>Still not the result I am looking for!</p>
http://answers.opencv.org/question/183753/birds-eye-view-perspectivetransform-from-camera-calibration/?comment=184007#post-id-184007@Eduardo the tutorial is not clear to me. You mind giving a small sample code in python?Sun, 04 Feb 2018 22:57:08 -0600http://answers.opencv.org/question/183753/birds-eye-view-perspectivetransform-from-camera-calibration/?comment=184007#post-id-184007Comment by Eduardo for <p>I am trying to get the bird's eye view perspective transform from camera intrinsic, extrinsic matrices and distortion coefficients. </p>
<p>I tried using the answer from <a href="https://stackoverflow.com/questions/23275877/opencv-get-perspective-matrix-from-translation-rotation">this</a> question.</p>
<p>The image used is the sample image left02.jpg from the opencv official github repo</p>
<p><a href="https://i.stack.imgur.com/vUmcl.png"><img alt="The image to be prospectively un-distored left02.jpg image from opencv sample images i.e get the bird's eye view of the image" src="https://i.stack.imgur.com/vUmcl.png"></a></p>
<p>I calibrated the camera and found the intrinsic, extrinsic matrices and the distortion co-efficients. </p>
<p>I undistored the image and found the pose. To check if the params are right. </p>
<p><a href="https://i.stack.imgur.com/mNLBy.png"><img alt="Image after un-distortion and visualising pose" src="https://i.stack.imgur.com/mNLBy.png"></a></p>
<p>The equations I used to find the perspective transformation matrix are (Refer the above link):</p>
<p><code>Hr = K * R.inv() * K.inv()</code> where R is rotational matrix (from cv2.Rodrigues()) and K is obtained from cv2.getoptimalnewcameramatrix()</p>
<pre><code> [ 1 0 | ]
Ht = [ 0 1 | -K*C/Cz ]
[ 0 0 | ]
</code></pre>
<p>Where <code>C=-R.inv()*T</code> Where T is translational vector from <code>cv2.solvePnP()</code>
and Cz is the 3rd component of the C vector </p>
<p>The required transformation is: <code>H = Ht * Hr</code></p>
<p>The code I used to construct the above equation is:</p>
<pre><code>K = newcameramtx # from cv2.getoptimalnewcameramatrix()
ret,rvec,tvec = cv2.solvePnP(world_points,corners2,K,dist)
R,_ = cv2.Rodrigues(rvec)
_,R_inv = cv2.invert(R)
_,K_inv = cv2.invert(K)
Hr = np.matmul(K,np.matmul(R_inv,K_inv))
C = np.matmul(-R_inv,tvec)
Cz = C[2]
temp_vector = np.matmul(-K,C/Cz)
Ht = np.identity(3)
for i,val in enumerate(temp_vector):
Ht[i][2] = val
homography = np.matmul(Ht,Hr)
warped_img =cv2.warpPerspective(img,homography,(img.shape[1],img.shape[0]))
# where img is the above undistored image with visualized pose
</code></pre>
<p>The resulting warped image is not correct.
<a href="https://i.stack.imgur.com/PnO3L.png"><img alt="With homographic matrix = Ht*Hr" src="https://i.stack.imgur.com/PnO3L.png"></a> </p>
<p>If I remove the translation from the homography by using the below code</p>
<pre><code>homography = Hr.copy()
warped_img =cv2.warpPerspective(img,homography,(img.shape[1],img.shape[0]))
</code></pre>
<p>I am getting the following image
<a href="https://i.stack.imgur.com/bLlYD.png"><img alt="With homographic matrix = Hr" src="https://i.stack.imgur.com/bLlYD.png"></a></p>
<p>I think the above image shows that my rotational part is correct but my translation is wrong. </p>
<p>Since the translational matrix (Ht) is an augmented matrix am unsure whether my construction of the above matrix is correct.</p>
<p>I specifically want to figure out the bird's eye perspective transformation from the camera calibration. </p>
<p>So, How do I correct the above equations so that I am getting the perfect bird's eye view of the chessboard image</p>
<p>Could anyone also please explain the math on how the above equations for Ht and Hr are derived? I don't have much exposure to Linear algebra so these equations are not very obvious to me. </p>
<p><strong>UPDATE:</strong></p>
<pre><code>homography = np.matmul(Ht,Hr)
warped_img =cv2.warpPerspective(img,homography,(img.shape[1],img.shape[0]),flags=cv2.WARP_INVERSE_MAP)
</code></pre>
<p>cv2.WARP_INVERSE_MAP flag gave me a different result</p>
<p><a href="https://i.stack.imgur.com/Y4GqK.png"><img alt="" src="https://i.stack.imgur.com/Y4GqK.png"></a></p>
<p>Still not the result I am looking for!</p>
http://answers.opencv.org/question/183753/birds-eye-view-perspectivetransform-from-camera-calibration/?comment=183925#post-id-183925See the tutorial: [Demo 3: Homography from the camera displacement](https://docs.opencv.org/3.4.0/d9/dab/tutorial_homography.html#tutorial_homography_Demo3)Sat, 03 Feb 2018 16:33:21 -0600http://answers.opencv.org/question/183753/birds-eye-view-perspectivetransform-from-camera-calibration/?comment=183925#post-id-183925