[Solved] How to make OpenCV to ask libv4l YUYV pixelformat instead of BGR24?

asked 2013-03-19 12:19:02 -0500

fliker09 gravatar image

updated 2013-03-21 11:02:17 -0500

I use at the moment OpenCV 2.4.4 on Linux Mint 14 (aka Ubuntu 12.10). I got 2 identical webcams (Logitech C270) and I need to run them simultaneously. From the start I want to say - using 2 USB hosts is not a solution for me. It is possible to run them simultaneously at 640x480@30 fps. But with one condition - use YUYV pixelformat. As all of us knows OpenCV uses BGR24 pixelformat. And here starts strange things... I am running v4l2-ctl -d /dev/video0 --list-formats and I get this:

    Index       : 0
    Type        : Video Capture
    Pixel Format: 'YUYV'
    Name        : YUV 4:2:2 (YUYV)

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'MJPG' (compressed)
    Name        : MJPEG

As you can see we don't have here BGR24 pixelformat. For testing purposes I use qv4l2 application. It can list supported pixelformats and one of them is BGR24! How is that possible? Strange things continues... If I try BGR24 with qv4l2 (OpenCV force this one) I can't get higher than 15 fps for simultaneous running. How is that possible - call for unlisted pixelformat and get it?! The most strange thing - it gets it and USB host can't let through such a large datastream :\ This is weird because it means that v4l doesn't convert YUYV to BGR24, it just gets BGR24 from camera (???)! One the solution would be lowering fps, but I don't want to do that. As far as I know to achieve what I want I have to modify ./modules/highgui/src/cap_libv4l.cpp (cap_v4l.cpp doesn't play role, because it isn't even compiled). From the start I want to say - I am lame at programming. I made this changes:

1) I took from cap_v4l.cpp code which converts YUYV to RGB24:

/* convert from 4:2:2 YUYV interlaced to RGB24 */
/* based on ccvt_yuyv_bgr32() from camstream */
#define SAT(c) \
   if (c & (~255)) { if (c < 0) c = 0; else c = 255; }

static void
yuyv_to_rgb24 (int width, int height, unsigned char *src, unsigned char *dst)
{
   unsigned char *s;
   unsigned char *d;
   int l, c;
   int r, g, b, cr, cg, cb, y1, y2;

   l = height;
   s = src;
   d = dst;
   while (l--) {
      c = width >> 1;
      while (c--) {
         y1 = *s++;
         cb = ((*s - 128) * 454) >> 8;
         cg = (*s++ - 128) * 88;
         y2 = *s++;
         cr = ((*s - 128) * 359) >> 8;
         cg = (cg + (*s++ - 128) * 183) >> 8;

         r = y1 + cr;
         b = y1 + cb;
         g = y1 - cg;
         SAT(r);
         SAT(g);
         SAT(b);

     *d++ = b;
     *d++ = g;
     *d++ = r;

         r = y2 + cr;
         b = y2 + cb;
         g = y2 - cg;
         SAT(r);
         SAT(g);
         SAT(b);

     *d++ = b;
     *d++ = g;
     *d++ = r;
      }
   }
} 

2) Made OpenCV ask for YUYV instead BGR24 (here I show already modified code):

  /* libv4l will convert from any format to V4L2_PIX_FMT_YUYV */
  CLEAR (capture->form);
  capture->form.type                = V4L2_BUF_TYPE_VIDEO_CAPTURE;
  capture->form.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
  capture->form.fmt.pix.field       = V4L2_FIELD_ANY;
  capture->form.fmt.pix.width = capture->width;
  capture->form.fmt.pix.height = capture->height;
...
  if (V4L2_PIX_FMT_YUYV != capture->form.fmt.pix.pixelformat) {
      fprintf( stderr, "HIGHGUI ...
(more)
edit retag flag offensive close merge delete

Comments

Asked for help on Logitech forums. Here is what I got:

http://www.quickcamteam.net/documentation/faq/what-video-streaming-formats-are-supported">http://web.archive.org/web/20110807021632/http://www.quickcamteam.net/documentation/faq/what-video-streaming-formats-are-supported

Not much help... It seems what I will have to write on V4L2 forums :\

fliker09 gravatar imagefliker09 ( 2013-03-21 06:37:32 -0500 )edit

I found the answer. Cameras are able to output exclusively YUYV and MJPG, as enumerated by v4l2-ctl. qv4l2 enumerates RGB24, BGR24, YU12 and YV12 in addition to mentioned above because it is from libv4l family. Libv4l library is able to convert any known input pixelformat to mentioned ones. So qv4l2 enumerates them as available because libv4l can convert to them! But that doesn't mean cameras are actually able to output in this pixelformats. Regarding FPS. I was checking with the help of mplayer if cameras are able to output 640x480@30fps. And it was lying (I understood that when observed extensive frame dropping). But here we get another mystery - mencoder and vlc gets 30 fps but bit depth gets down to 8 bits from 16 (so it means that instead of YUV 4:2:2 they get YUV 4:1:0).

fliker09 gravatar imagefliker09 ( 2013-03-21 10:56:18 -0500 )edit

How is that possible I don't know (if anybody could only explain to me...). So conclusion is simple - using YUYV it is possible with my USB host to get 2 cameras working at 640x480 only with 15 fps. I made necessary modification in cap_libv4l.cpp and recompiled OpenCV. Cameras are working nicely, thread closed as solved. My code from the question is unnecessary as libv4l do all the work.

fliker09 gravatar imagefliker09 ( 2013-03-21 11:01:10 -0500 )edit

Starting from 2.4.6 where is no more need for this dirty workaround, CV_CAP_PROP_FPS is finally implemented. Hooray!

P.S. Regarding the speeds. Try this workaround: unplug your cameras, sudo modprobe -r uvcvideo, sudo modprobe uvcvideo quirk=0x80, plug your cameras back and check. In this way I achieved 1280x960@5fps instead of 640x480@15fps. No comments needed I hope ;)

fliker09 gravatar imagefliker09 ( 2013-09-26 15:08:06 -0500 )edit