My earlier camera calibration attempts went very smooth and gave good results. Now I wanted to get serious with it and calibrate each of my cameras (I'm using 3 of the same type in my system), each at their specific focus.
What I did before:
- used a 7x9 (inner corners) squares chessboard pattern with square size 10 mm
- printed on paper
- put on the floor to be hopefully very flat
- took ~20 pictures from various angles, trying to have the pattern appear in all areas of the image
- selected 12 best pictures (with respect to focus, movement, light)
- calibrated and created camera matrix and distortion vector
The result was good, the RMS reprojection error value is 0.425 (i believed that meant "good'), and the undistorted images looked nice and undistored (straight lines were straight).
Now, I got my chessboard pattern printed on aluminium di-bond using an online foto printing service, hoping to optimize the procedure. I used a 20 mm 7x9 pattern, but because I wanted to make it "beautiful" and made the outer squares end round (as I had seen in some example chessboard patterns), my high-resolution camera only recognizes 5x7 patterns in it (my lower resolution cameras somehow recognize 7x9 easily). Also, because of how the foto printing service works, my square size is not 20 mm but 17.08 mm - which I don't care much about.
Anyway, I put my perfectly straight 17.x mm 5x7 squares aluminium chesspattern in absolute perfect surround LED lighting and did my foto session. I thought "more is more" and took many pictures, from which I selected 20.
The result is devestating!
The RMS value is from 0.95 to 1.15 for my 3 cameras (maybe not too bad?). But the undistortion for 2 of the 3 results creates heavily undistorted images (of size 2048x1536), in which the actual undistorted part is concentrated in around 300x200 pixels somewhere in the bottom right or top right corner. The undistortion looks about right, but obviously something is wrong. This is what I would perhaps call "over-undistorted".
Just by luck, I selected only 12 images to calculate the camera matrix and distortion vector, and received a fairly good result. But there the RMS reprojection error was ~4.5 (which I believed was worse than the 0.4 which I had with my first setup).
I went ahead and selected different combinations of chessboard recordings, trying to find if there was one that destroyed the result. But there wasn't - any combination would lead to more or less over-undistorted results. Something in between with 17 images:
So obviously I am very much confused with the results I'm getting, especially compared with the first results I had, where everything was pretty easy and straight forward. I'm always reading that we need at least 10 images to get a good calibration. However, in my case, I might feel like the higher number gets me into trouble. This is not really true, because also with a lower number of samples I am in trouble. George Lecakes says in https://www.youtube.com/watch?v=v7jutAmWJVQ that he takes around 50 images.
I am wondering, will I have a chance to get my undistortion to give better results, if I increase the number of samples?