Revision history [back]

Here is sample code to calculate the line of sight vector to a point. Since you know the distance to the ball (the red circle), you can simply scale the line of sight by that distance to get your answer.

Note that this provides the answer in camera coordinates. Where X is to the right, Y is down, and Z is out of the camera. Transform as necessary.

    double x = pt.x;
    double y = pt.y;

    Mat LOS(3,1,CV_64F);

    LOS.at<double>(0) = ptsOut[0].x;
    LOS.at<double>(1) = ptsOut[0].y;
    LOS.at<double>(2) = 1; 

    LOS = cameraMatrix.inv()*LOS;