Ask Your Question
1

Which cameras do you recommend for stereo with a larger baseline in OpenCV?

asked 2016-08-19 02:47:15 -0600

closecall gravatar image

I want to have a pair of cameras for a stereo research project (disparity maps etc.) with OpenCV from live feed video.

I am thinking of some modern USB HD webcams, but I am not entirely sure because of codec or other issues with OpenCV.

I also want to have a larger baseline compared to stereo cams, so I need to have two separate cameras instead of a stereo camera.

The optimal case would be a simultaneous lag-free full HD video capture of both cameras in OpenCV with high FPS and a way to fix the cameras in place.


1) Which issues could present themselves with OpenCV? What should I pay attention to when I buy?

2) What kind of cameras do you recommend? Do you have a specific camera in mind?

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
3

answered 2016-08-19 14:13:45 -0600

Eduardo gravatar image

updated 2016-08-19 14:36:32 -0600

What baseline do you plan to use (or what is your expected depth range)?

If money is not a big issue, you could consider buying directly a professional stereo camera. For example, you could search for stereo cameras used in the autonomous car field. The one I know for example:

  • ZED stereo camera - Depth Range: 0.7 - 20 m (2.3 to 65 ft) ; Stereo Baseline: 120 mm (4.7") : max video resolution: 2.2K, 15 FPS, 4416x1242
  • stereo cameras from PointGrey
  • search for companies similar to PointGrey

As these professional cameras are quite expensive, in my opinion what you should pay attention if you want to build your own stereo cameras rig are:

  • how will you synchronise the streams? Maybe for static scene it could be ok but I think you will need a synchronization signal or the possibility to get a timestamped frame.
  • you want an HD resolution (1280x720) + a live stream depth map + high FPS: you have to consider that reconstructing the depth map in real time is quite costly. This is not my field but I doubt it is possible currently to compute the depth map with an HD resolution at > 30 FPS on a CPU. You will need to consider to use the GPU in my opinion. In my opinion, don't expect to have a stereo reconstruction algorithm with a high quality and with a good computation time available for free.
  • I don't think the acquisition backend will be an issue for the webcams you can find on the public market (OpenCV should be able to capture the frames)
  • you should pay attention to the quality of the optic (low distorsion), possibility to manually set the parameters (exposure, shutter time, enable / disable the white balance correction, etc.) and try to have two identical optics
  • you will have to carefully mount the cameras to align them (fronto parallel view) in order to facilitate the reconstruction step (an object that appears in the left image appears in the same row in the right image, you only have to perform a column search)

You should look in the hack community, for example the ultra low cost PS3Eye camera. It doesn't have an HD resolution but it has a high FPS, the driver is available on Linux and it is very easy to hack the camera:

edit flag offensive delete link more

Comments

Thank you for your great reply. My final goal baseline is 1 m (depends on some other structure), which is huge, I think. Also, I think it's not possible right now to have that high FPS even with GPU algorithms at the moment. All your points are valid and I will keep them in mind, so thanks again! I think I will start with some good webcams and fix them to a rig.

closecall gravatar imageclosecall ( 2016-08-22 02:34:31 -0600 )edit

What is the expected depth range for a baseline of 1 m? Just curious, can you describe your application a little?

Eduardo gravatar imageEduardo ( 2016-08-22 12:03:29 -0600 )edit

Basically I want to make a see-through AR application. Imagine three cameras (A,B,C) mounted outside to a wall ("#") with windows ("__"): A#__#B#__#C The distance between the cameras is 1 m, so I cannot make it smaller. In the wall part, I want to render the outside from my virtual viewpoint. Hm, I think a depth range starting at 0.5 m seems good.

A large baseline makes my stereo very far-sighted, right? With z=(f*b)/d, assuming a 50 mm focal length and a baseline of 1 m, we get for a disparity of 0.1 m (or is it pixels?) a depth of 0.5 m.

closecall gravatar imageclosecall ( 2016-08-23 02:48:12 -0600 )edit

Yes, a large baseline allows to see very far-sighted. The disparity is the pixel difference between the left and the right coordinates of the same object point viewed in the left and the right cameras. More information here, here or here.

Eduardo gravatar imageEduardo ( 2016-08-23 10:17:16 -0600 )edit

I was thinking, you should delegate the stereo part and focus to the AR application as it is rather a big project that will need lots of work in my opinion.

What I would do:

  • use a dedicated depth sensor like the Intel RealSense R200 (be careful, there are 2 models and one, should be the R200, can work outside even with the sun with some tweaks like exposure, ...), it is not so expensive and you will get directly a real-time depth and color stream with a good quality
  • if inside, you can even find a cheap Kinect 1 and many projects
  • i would place the RealSense in the other side of the wall and find a way to hide the cables, hide the connection with the computer/projector, etc.
Eduardo gravatar imageEduardo ( 2016-09-01 11:48:15 -0600 )edit

Moreover, if your goal is the AR application and not a research project on stereo reconstruction part, using directly a depth sensor should be the way to go as stereo reconstruction is not easy if you are new to this in my opinion. The Intel RealSense is quite small and if it is an artistic project, it could be quite impressive as you won't see any device when you look at the wall.

You will be able to move your installation without the need to calibrate each time and/or build a specific setup.

The only issue would be to place the depth sensor and deal with the cables and the connections with the computer. What I see is something like this, I don't know what you have in mind in your project:

     RealSense

####__####__####

       ^^^^

    Projector + PC
Eduardo gravatar imageEduardo ( 2016-09-01 11:55:55 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2016-08-19 02:47:15 -0600

Seen: 2,293 times

Last updated: Aug 19 '16