Difference between undistortPoints() and projectPoints() in OpenCV

edit

undistortPoints is a special case of the reverse projectPoints function.

So, if you pass undistortPoints with no P parameter, then projectPoints with rvec=tvec=(0,0,0) and the camera matrix will return them to the original location. If you do give undistortPoints a P parameter, then using the identity matrix will return them to the original location. Note that for both of these you must convert the 2d points to 3d before passing to projectPoints.

projectPoints has several other uses. If you have true 3d points, you can place them onto an image (either distorted or undistorted). Pass in the camera rvec and tvec, as well as the camera matrix. To place them onto a distorted image, also pass the camera distortion matrix. To place them on an undistorted image, pass noArray or zeros.

NOTE: I don't think project points checks direction, so if you have points in front of and behind the camera, you will get answers for both on the plane of the image. I think, I'm not in a place to test right now.

The Jacobian is useful for many things, but if you don't need it, don't worry about it. If you do need it, you ought to already know what it is and what it means.