1 | initial version |
Is the jump really related to y-location? I believe it is caused by the pose of the object, which also changes as you move the object up. Can you fix the object location and change only its pose?
The solution of solvePnP is not constrained sufficiently in that particular pose which may cause instability. You can consider some temporal smoothing to reduce this effect.