Revision history [back]

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

offhand I'm not aware of specific APIs in OpenCV that will all of solve this for you. if you need more support for linear algebra and matrix operations, add the Eigen library, or use python+numpy.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

offhand I'm not aware of specific APIs in OpenCV that will all of solve this for you. if you need more support for linear algebra and matrix operations, add the Eigen library, or use python+numpy.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

offhand I'm not aware of specific APIs in OpenCV that will all of solve this for you. if you need more support for linear algebra and matrix operations, add the Eigen library, or use python+numpy.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products, I think. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

offhand I'm not aware of specific APIs in OpenCV that will all of solve this for you. if you need more support for linear algebra and matrix operations, add the Eigen library, or use python+numpy.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products, I think. products. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

offhand I'm not aware of specific APIs in OpenCV that will all of solve this for you. if you need more support for linear algebra and matrix operations, add the Eigen library, or use python+numpy.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

offhand I'm not aware of specific APIs in OpenCV that will all of solve this for you. if you need more support for linear algebra and matrix operations, add the Eigen library, or use python+numpy.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

calculating these lines given image pixel coordinates should involve the camera projection matrix and its pose matrix. "invert" the projection matrix and give it a screen coordinate (inverting is numerically bad, so this is done in one "solve" step). you'll get a vector out (w-coordinate is 0, homogeneous coordinates).

offhand I'm not aware of specific APIs in OpenCV that will all of solve this for you. if you need more support for linear algebra and matrix operations, add the Eigen library, or use python+numpy. opencv does do cross products.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

calculating these lines given image pixel coordinates should involve the camera projection matrix and its pose matrix. "invert" the projection matrix and give it a screen coordinate (inverting is numerically bad, so this is done in one "solve" step). you'll get a vector out (w-coordinate is 0, homogeneous coordinates).point on the line.

offhand I'm not aware of specific APIs in OpenCV that will all of solve this for you. if you need more support for linear algebra and matrix operations, add the Eigen library, or use python+numpy. opencv does do cross products.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

calculating these lines given image pixel coordinates should involve the camera projection matrix and its pose matrix. "invert" the projection matrix and give it a screen coordinate (inverting is numerically bad, so this is done in one "solve" step). you'll get a point on the line.line (and image plane), in the camera frame.

offhand I'm not aware of specific APIs in OpenCV that will all of solve this for you. if you need more support for linear algebra and matrix operations, add the Eigen library, or use python+numpy. opencv does do cross products.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

calculating these lines given image pixel coordinates should involve the camera projection matrix and its pose matrix. "invert" the projection matrix and give it a screen coordinate (inverting is numerically bad, so this is done in one "solve" step). you'll get a point on the line (and image plane), in the camera frame.

offhand I'm not aware of I can't list the specific APIs in OpenCV that will all of solve this for you. if you need more support OpenCV happens to not have a function you need, maybe for linear algebra and matrix operations, algebra, add the Eigen library, or use python+numpy. opencv does do cross products.

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

calculating these lines given image pixel coordinates should involve the camera projection matrix and its pose matrix. "invert" the projection matrix and give it a screen coordinate (inverting is numerically bad, so this is done in one "solve" step). you'll get a point on the line (and image plane), in the camera frame.frame. you can turn that into a vector (4x1 homogeneous, x/y/z as you got them, w-coordinate is 0) and pass it into the pose matrix (4x4), to get that vector in world space.

offhand I can't list the specific APIs in OpenCV that will all of solve this for you. if OpenCV happens to not have a function you need, maybe for linear algebra, add the Eigen library, or use python+numpy. opencv does do cross product, norms (L2 norm in particular), and matrix multiplication (dot product).

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

calculating these lines given image pixel coordinates should involve the camera projection matrix and its pose matrix. "invert" the projection matrix and give it a screen coordinate (inverting is numerically bad, so this is done in one "solve" step). you'll get a point on the line (and image plane), in the camera frame. you can turn that into a vector (4x1 homogeneous, x/y/z as you got them, w-coordinate is 0) and pass it into the pose matrix (4x4), to get that vector in world space. combine with camera origin (0) mapped through pose matrix to get the origin point of the line/ray in world space.

offhand I can't list the specific APIs in OpenCV that will all of solve this for you. if OpenCV happens to not have a function you need, maybe for linear algebra, add the Eigen library, or use python+numpy. opencv does do cross product, norms (L2 norm in particular), and matrix multiplication (dot product).

you understand the math well. go with your intuition.

consider a 3D ray/line for an object in the image. lines for the same object, from both camera views, will intersect (or pass closely) in space. lines that intersect/come close may not be of the same object, due to the correspondence problem, as you understand.

you'll need to calculate those lines for your objects. then, to check feasible correspondence, calculate the closest distance for each pair and check if it's small. that's a little bit of linear algebra involving cross products. you'll know that you have an unresolvable ambiguity when multiple candidates come close.

to resolve ambiguities, you can try giving objects identity, and track them over time. give objects position and velocity. associate detections in a new video frame to objects given their predicted position over the time step, then adjust position and velocity estimate. you can formalize this as a Kalman filter, or go with the simple formulation I gave.

as for object detection, since you have white balls on dark background, simple "blob detection" (thresholding, contours/connected components, cv::moments for centroids) is probably good enough. I agree that feature matching entirely unsuitable here.

calculating these lines given image pixel coordinates should involve the camera projection matrix and its pose matrix. "invert" the projection matrix and give it a screen coordinate (inverting is numerically bad, so this is done in one "solve" step). you'll get a point on the line (and image plane), in the camera frame. you can turn that into a vector (4x1 homogeneous, x/y/z as you got them, w-coordinate is 0) and pass it into the pose matrix (4x4), to get that vector in world space. combine with camera origin (0) (0 vector, 4x1) mapped through pose matrix to get the origin point of the line/ray in world space.

offhand I can't list the specific APIs in OpenCV that will all of solve this for you. if OpenCV happens to not have a function you need, maybe for linear algebra, add the Eigen library, or use python+numpy. opencv does do cross product, norms (L2 norm in particular), and matrix multiplication (dot product).