Ask Your Question

cv::viz Point Cloud

asked 2015-07-04 17:20:50 -0500

Throwaway99 gravatar image

image description

I have a stereo webcam from which I compute disparities and project to 3D. I would like to be able to have a vtk viewing window continuously re-render these points. When I try to do so, however, I get inconsistent behavior. For one thing the points blink in and out. They should always be on the visible, but they are absent about half the frames. Also, there appears to be issues with the viewer such that when I approach the points too closely they disappear, something like a near culling distance, but this applies to the entire cloud, not just those points that would have been closest.

    // Project disparity to 3D:
    cv::Mat xyz;
    cv::reprojectImageTo3D( disparity32, xyz, Q, false );

    Points3DFrame points3DFrame( xyz, stereoframe.timestamp );
    points3DBuffer->addFrame( points3DFrame );

My viewer thread looks like this:

/// Create a window
cv::viz::Viz3d myWindow("Coordinate Frame");

Points3DFrame points3DFrame;

while( points3DFrame.points.empty() ) {
    points3DBuffer->getFrame( &points3DFrame );

unsigned long lastFrameTime = timeGetTime();

while( !myWindow.wasStopped() )
    points3DBuffer->waitNewFrame( &points3DFrame, lastFrameTime );
    lastFrameTime = points3DFrame.timestamp;

    cloud_widget = cv::viz::WCloud( points3DFrame.points );
    cloud_widget.setRenderingProperty( cv::viz::POINT_SIZE, 4 );

    myWindow.showWidget( "Depth", cloud_widget );

    myWindow.spinOnce( 33, true );

Any ideas on what I can do to achieve a stable cloud viewer?

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2015-10-14 09:09:14 -0500

Throwaway99 gravatar image

Never determined the real issue, but the following works as expected:

while( isRunning() && !myWindow.wasStopped() )
    points3DFrame = points3DBuffer.waitNewFrame( lastFrameTime );
    lastFrameTime = points3DFrame->timestamp;

    cv::Vec3f* ptsPtr = std::const_pointer_cast<Points3DFrame>(points3DFrame)->points.ptr<cv::Vec3f>();

    int index = 0;
    for( int i = 0; i < height; ++i ) {         
        for( int j = 0; j < width; ++j, ++index ) {         
            if( ptsPtr[index][2] > 0.01 && ptsPtr[index][2] < 255 ) {
                pCloudPtr[index] = ptsPtr[index];
            } else {
                pCloudPtr[index] = cv::Vec3f();

    cv::viz::WCloud cloud_widget = cv::viz::WCloud( pCloud, cv::viz::Color::green() );
    cloud_widget.setRenderingProperty( cv::viz::POINT_SIZE, 2 );

    myWindow.showWidget( "Depth", cloud_widget );

    myWindow.spinOnce( 30, true );

As you can see, I'm removing those points that I know are likely to be invalid ( negative depth or depth beyond 2.5 meters ).

edit flag offensive delete link more

answered 2015-10-06 09:18:41 -0500

tdiv gravatar image

Hi, I work also with stereovision using openCV and viz, but I don't need to re-render these points continously. However I encountered problems similiar to "near culling". I mixed some vtk dll Debug/Release version (they have the same names). In debug sometimes the "near culling" problem still happens but in release version with proper dll it is OK.

edit flag offensive delete link more


I have just encountered problably the same problem with "near culling". This happens when Q matrix is improper. Calculate Q matrix only once during calibration and load it. If You use stereoRectify (calculates Q) online before sgbm and reprojectImageTo3D with this Q the scene coordinates are reversed in a strange way.

tdiv gravatar imagetdiv ( 2015-10-06 11:14:45 -0500 )edit

Question Tools

1 follower


Asked: 2015-07-04 17:17:09 -0500

Seen: 8,592 times

Last updated: Oct 14 '15