Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

How does VideoCapture from USB Webcam on Windows using DirectShow and MJPG work?

Hello everybody,

my webcam Logitech BRIO supports MJPEG for higher resolutions or for higher frames per second. Sadly, I'm unable to set the VideoCapture backend to DirectShow and then to MJPG/mjp2/mjpa/mjpb in order to receive the compressed pictures from the camera (in 640x480@120fps).

A little help would be very much appreciated.

here is the code, the ffmpeg information dump and the OpenCV build information

#include <iostream>
#include <opencv2/opencv.hpp>
#include <opencv2/videoio.hpp>
#include <ctime>

int main(int argc, char** argv)
{

  cv::Mat in_frame;
  int apiBackend = cv::CAP_DSHOW;
  //int apiBackend = 2200;//cv::CAP_OPENCV_MJPEG; // this is not available in my built of OpenCV3
  cv::VideoCapture camera(0+apiBackend);
  if (!camera.isOpened())
  {
    std::cout << "Error! Camera not ready." << std::endl;
    return -1;
  }
  int fps = 120;
  int codec = cv::VideoWriter::fourcc('m','j','p','2');
  //int codec = CV_FOURCC('M','J','P','G');
  camera.set(CV_CAP_PROP_FOURCC, codec);
  camera.set(CV_CAP_PROP_FPS, fps);
  camera.set(CV_CAP_PROP_FRAME_WIDTH, 640);
  camera.set(CV_CAP_PROP_FRAME_HEIGHT, 480);
  int frame_counter = 0;
  std::cout << cv::getBuildInformation() << std::endl;
  std::cout << "opened video capure device at idx " << 0+cv::CAP_DSHOW << std::endl;
  std::cout << "start reading" << std::endl;
  std::clock_t begin = std::clock();
  while (1)
  {
    if (frame_counter > 1000) break;
    camera >> in_frame;
    if (++frame_counter % 30 == 0)
    {
      std::clock_t end = std::clock();
      double timePassed = double(end-begin);
      begin = end;
      if (timePassed > 0)
      {
        double zack = 30/(timePassed/1000);
        std::cerr << "fps: " << zack << " of set " << fps << " fps in " << timePassed/1000 << "s @" << frame_counter << std::endl;
      }
    }
  }
  return 0;
}

here are the supported modes of the Logitech BRIO webcam

ffmpeg.exe -f dshow -list_options true -i video="Logitech BRIO"
[dshow @ 00000000001f6a20] DirectShow video device options (from video devices)
[dshow @ 00000000001f6a20]  Pin "Capture" (alternative pin name "0")
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=640x480 fps=5 max s=640x480 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=640x480 fps=5 max s=640x480 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=160x120 fps=5 max s=160x120 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=160x120 fps=5 max s=160x120 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=176x144 fps=5 max s=176x144 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=176x144 fps=5 max s=176x144 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=320x180 fps=5 max s=320x180 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=320x180 fps=5 max s=320x180 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=320x240 fps=5 max s=320x240 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=320x240 fps=5 max s=320x240 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=352x288 fps=5 max s=352x288 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=352x288 fps=5 max s=352x288 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=340x340 fps=30 max s=340x340 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=340x340 fps=30 max s=340x340 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=424x240 fps=5 max s=424x240 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=424x240 fps=5 max s=424x240 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=440x440 fps=30 max s=440x440 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=440x440 fps=30 max s=440x440 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=480x270 fps=5 max s=480x270 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=480x270 fps=5 max s=480x270 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=640x360 fps=5 max s=640x360 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=640x360 fps=5 max s=640x360 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=800x448 fps=5 max s=800x448 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=800x448 fps=5 max s=800x448 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=800x600 fps=5 max s=800x600 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=800x600 fps=5 max s=800x600 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=848x480 fps=5 max s=848x480 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=848x480 fps=5 max s=848x480 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=960x540 fps=5 max s=960x540 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=960x540 fps=5 max s=960x540 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=1024x576 fps=5 max s=1024x576 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=1024x576 fps=5 max s=1024x576 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=1280x720 fps=5 max s=1280x720 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=1280x720 fps=5 max s=1280x720 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=1600x896 fps=5 max s=1600x896 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=1600x896 fps=5 max s=1600x896 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=1920x1080 fps=5 max s=1920x1080 fps=30
[dshow @ 00000000001f6a20]   pixel_format=yuyv422  min s=1920x1080 fps=5 max s=1920x1080 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=640x480 fps=5 max s=640x480 fps=120
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=640x480 fps=5 max s=640x480 fps=120
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=160x120 fps=5 max s=160x120 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=160x120 fps=5 max s=160x120 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=176x144 fps=5 max s=176x144 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=176x144 fps=5 max s=176x144 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=320x180 fps=5 max s=320x180 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=320x180 fps=5 max s=320x180 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=320x240 fps=5 max s=320x240 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=320x240 fps=5 max s=320x240 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=352x288 fps=5 max s=352x288 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=352x288 fps=5 max s=352x288 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=424x240 fps=5 max s=424x240 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=424x240 fps=5 max s=424x240 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=480x270 fps=5 max s=480x270 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=480x270 fps=5 max s=480x270 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=640x360 fps=5 max s=640x360 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=640x360 fps=5 max s=640x360 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=800x448 fps=5 max s=800x448 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=800x448 fps=5 max s=800x448 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=800x600 fps=5 max s=800x600 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=800x600 fps=5 max s=800x600 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=848x480 fps=5 max s=848x480 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=848x480 fps=5 max s=848x480 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=960x540 fps=5 max s=960x540 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=960x540 fps=5 max s=960x540 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=1024x576 fps=5 max s=1024x576 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=1024x576 fps=5 max s=1024x576 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=1280x720 fps=5 max s=1280x720 fps=90.0001
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=1280x720 fps=5 max s=1280x720 fps=90.0001
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=1600x896 fps=5 max s=1600x896 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=1600x896 fps=5 max s=1600x896 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=1920x1080 fps=5 max s=1920x1080 fps=60.0002
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=1920x1080 fps=5 max s=1920x1080 fps=60.0002
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=2560x1440 fps=5 max s=2560x1440 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=2560x1440 fps=5 max s=2560x1440 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=3840x2160 fps=5 max s=3840x2160 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=3840x2160 fps=5 max s=3840x2160 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=4096x2160 fps=5 max s=4096x2160 fps=30
[dshow @ 00000000001f6a20]   vcodec=mjpeg  min s=4096x2160 fps=5 max s=4096x2160 fps=30
[dshow @ 00000000001f6a20]   pixel_format=nv12  min s=640x480 fps=5 max s=640x480 fps=30
[dshow @ 00000000001f6a20]   pixel_format=nv12  min s=640x480 fps=5 max s=640x480 fps=30
[dshow @ 00000000001f6a20]   pixel_format=nv12  min s=640x360 fps=5 max s=640x360 fps=30
[dshow @ 00000000001f6a20]   pixel_format=nv12  min s=640x360 fps=5 max s=640x360 fps=30
[dshow @ 00000000001f6a20]   pixel_format=nv12  min s=1280x720 fps=5 max s=1280x720 fps=30
[dshow @ 00000000001f6a20]   pixel_format=nv12  min s=1280x720 fps=5 max s=1280x720 fps=30
[dshow @ 00000000001f6a20]   pixel_format=nv12  min s=1920x1080 fps=5 max s=1920x1080 fps=30
[dshow @ 00000000001f6a20]   pixel_format=nv12  min s=1920x1080 fps=5 max s=1920x1080 fps=30

this is the build information of my windows installation of OpenCV 3

General configuration for OpenCV 3.2.0 =====================================
  Version control:               3.2.0

  Platform:
    Timestamp:                   2016-12-23T14:53:43Z
    Host:                        Windows 10.0.14393 AMD64
    CMake:                       3.7.0
    CMake generator:             Visual Studio 14 2015 Win64
    CMake build tool:            C:/Program Files (x86)/MSBuild/14.0/bin/MSBuild.exe
    MSVC:                        1900

  C/C++:
    Built as dynamic libs?:      YES
    C++ Compiler:                C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe  (ver 19.0.24215.1)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR /EHa  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /wd4251 /wd4324 /wd4275 /wd4589 /MP8  /MD /O2 /Ob2 /DNDEBUG  /Zi
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR /EHa  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /wd4251 /wd4324 /wd4275 /wd4589 /MP8  /D_DEBUG /MDd /Zi /Ob0 /Od /RTC1
    C Compiler:                  C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi    /MP8  /MD /O2 /Ob2 /DNDEBUG  /Zi
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi    /MP8  /D_DEBUG /MDd /Zi /Ob0 /Od /RTC1
    Linker flags (Release):      /machine:x64  /INCREMENTAL:NO  /debug
    Linker flags (Debug):        /machine:x64  /debug /INCREMENTAL
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 core flann imgproc ml photo video imgcodecs shape videoio highgui objdetect superres features2d calib3d stitching videostab world
    Disabled:                    -
    Disabled by dependency:      -
    Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev java python2 ts viz

  Windows RT support:            NO

  GUI:
    QT:                          NO
    Win32 UI:                    YES
    OpenGL support:              NO
    VTK support:                 NO

  Media I/O:
    ZLib:                        build (ver 1.2.8)
    JPEG:                        build (ver 90)
    WEBP:                        build (ver 0.3.1)
    PNG:                         build (ver 1.6.24)
    TIFF:                        build (ver 42 - 4.0.2)
    JPEG 2000:                   build (ver 1.900.1)
    OpenEXR:                     build (ver 1.7.1)
    GDAL:                        NO
    GDCM:                        NO

  Video I/O:
    Video for Windows:           YES
    DC1394 1.x:                  NO
    DC1394 2.x:                  NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (ver 57.64.101)
      avformat:                  YES (ver 57.56.100)
      avutil:                    YES (ver 55.34.100)
      swscale:                   YES (ver 4.2.100)
      avresample:                YES (ver 3.1.0)
    GStreamer:                   NO
    OpenNI:                      NO
    OpenNI PrimeSensor Modules:  NO
    OpenNI2:                     NO
    PvAPI:                       NO
    GigEVisionSDK:               NO
    DirectShow:                  YES
    Media Foundation:            NO
    XIMEA:                       NO
    Intel PerC:                  NO

  Parallel framework:            Concurrency

  Other third-party libraries:
    Use IPP:                     9.0.1 [9.0.1]
         at:                     C:/build/master_winpack-build-win64-vc14/build/3rdparty/ippicv/ippicv_win
    Use IPP Async:               NO
    Use Lapack:                  NO
    Use Eigen:                   NO
    Use Cuda:                    NO
    Use OpenCL:                  YES
    Use OpenVX:                  NO
    Use custom HAL:              NO

  OpenCL:                        <Dynamic loading of OpenCL library>
    Include path:                C:/build/master_winpack-build-win64-vc14/opencv/3rdparty/include/opencl/1.2
    Use AMDFFT:                  NO
    Use AMDBLAS:                 NO

  Python 2:
    Interpreter:                 C:/utils/soft/python27-x64/python.exe (ver 2.7.12)

  Python 3:
    Interpreter:                 C:/utils/soft/python35-x64/python.exe (ver 3.5.2)

  Python (for build):            C:/utils/soft/python27-x64/python.exe

  Java:
    ant:                         C:/utils/soft/apache-ant-1.9.7/bin/ant.bat (ver 1.9.7)
    JNI:                         C:/Program Files/Java/jdk1.8.0_112/include C:/Program Files/Java/jdk1.8.0_112/include/win32 C:/Program Files/Java/jdk1.8.0_112/include
    Java wrappers:               NO
    Java tests:                  NO

  Matlab:                        Matlab not found or implicitly disabled

  Tests and samples:
    Tests:                       NO
    Performance tests:           NO
    C/C++ Examples:              NO

  Install path:                  C:/build/master_winpack-build-win64-vc14/install

  cvconfig.h is in:              C:/build/master_winpack-build-win64-vc14/build
-----------------------------------------------------------------

start reading