SolvePnP fluctuating results

edit

I am using SolvePnP to determine the camera position. But results in world coordinates fluctuate a lot, also estimated rotation changes a lot. Are there best practices for this or recommended settings to improve the results?

I am now currently not using any flags like 'CV_ITERATIVE' but could this help?