Suggestion: drop VTK for VIZ and use simple OpenGL

asked 2018-02-09 06:12:57 -0500

TomPollok gravatar image

updated 2018-02-09 08:32:37 -0500

Hello there,

would it be possible to remove the whole VTK dependency from OpenCV and create a simple OpenGL based basic vizualization instead that comes per default with the opencv package?

Reasons: 1. Not having the balast of this VTK library. Compiling it is not fun. Also when someone just wants to vizualize sth quickly it is not quickly set up. 2. VTK pointcloud renderer is slow. To be fair, thats my experience when using it with PCL - it took about 60ms on a high end desktop pc until a dense pointcloud with 1920x1200 points (about 2.3 Million 3d points) was rendered. Just for uploading and rendering it. This is so slow. 3. You can create really fast renderers that directly upload your cv::Mat 3d vertices an opengl vertex buffer array. I have written a very simple prototype that used simple OpenGL code and renders it very fast.

Here i prepared a desktop recording of a scene with 1920x1200 image resolution. In this video i do the following: Reading stereo images from disk with 1920x1200 resolution Estimating a depthmap from those images using block matching Calculating 3d vertices from the depth map uploading to a pointcloud viewer that i have written

Here the video: https://youtu.be/-Bw_m85EAto If someone wants to know (at the begining i used a blocksize of 19 and at the end of the video i changed it interactively to 47 - with 19 blocksize it takes 50ms and with 47 blocksize the whole pipeline takes 80ms)

What users get: - high performance rendering of point cloud sequences for example from a stereo block matching result - no vtk dependency and viz should then come per default in opencv

Does OpenCV use VTK else where? I mean is it such a useful library for opencv? In my opinion, when someone wants to use VTK for vizualization, then VTK should be a dependency of the end user project but not of opencv library.

This is not a question, but rather a discussion. What do you thing? I thought maybe for the coming opencv 4.0 release this would be a nice idea.

edit retag flag offensive close merge delete

Comments

I cannot access the YouTube video.

I know another person who complains also about the viz / VTK dependencies so you are not alone.

The librealsense uses glfw for rendering and the results are pretty good in my opinion.

Eduardo gravatar imageEduardo ( 2018-02-09 08:19:16 -0500 )edit

I updated the private link

https://youtu.be/-Bw_m85EAto

TomPollok gravatar imageTomPollok ( 2018-02-09 08:33:33 -0500 )edit

If you're going to use OpenGL, you might have to feed in square textures. Beware!

Do you need MVP matrix creation code?

sjhalayka gravatar imagesjhalayka ( 2018-02-09 09:36:34 -0500 )edit
1

It's not a good idea. if you don't want to use VTK write your own interface using opengl.

You can use ovis module too.

You can use openscenegraph too outside opencv it is easy.

LBerger gravatar imageLBerger ( 2018-02-09 10:43:35 -0500 )edit

It might be a good proposal for OpenCV GSoC 2018, building a fast and robust integrated visualisation! You can always do a proposal!

StevenPuttemans gravatar imageStevenPuttemans ( 2018-02-12 09:29:32 -0500 )edit

@TomPollok

I think there is a related discussion topic about highgui improvement where you can share your ideas if you want.

Eduardo gravatar imageEduardo ( 2018-03-07 05:04:54 -0500 )edit