Simplified pose estimation on collinear points, like solvePNP

edit

Let's say I have a pole that I can identify points on in the image, like a vertical post painted in contrast colors with a known pattern. And I know the distances between those points in world coordinates. But since it is a pole all the points belong to a single line. I tried to use solvePNP but it seems to be not very happy with collinear points. Is there any other methods in OpenCV similar to solvePNP but for collinear points? Apparently since this is a pole I can live without the rotation around the pole if it simplifies the problem. Thank you.