CvCapture_MSMF::initStream Failed to set mediaType after switching to CUDA build of OpenCV

edit

Yesterday I got set up with OpenCV (precompiled libraries for windows) and everything was working great! I had a little program that would read a video feed from my Valve Index, undistort each lens, and then display it back. It was super slow though, so I looked into using CUDA to speed it up and found that the precompiled libraries didn't include CUDA support. So I followed the tutorial for CMake and git-bash, and used this slightly modified build script:

#!/bin/bash -e
myRepo=$(pwd)
CMAKE_CONFIG_GENERATOR="Visual Studio 16 2019"
if [  ! -d "$myRepo/opencv"  ]; then
    echo "cloning opencv"
    git clone https://github.com/opencv/opencv.git
    mkdir -p Build/opencv
    mkdir -p Install/opencv
else
    cd opencv
    git pull --rebase
    cd ..
fi
if [  ! -d "$myRepo/opencv_contrib"  ]; then
    echo "cloning opencv_contrib"
    git clone https://github.com/opencv/opencv_contrib.git
    mkdir -p Build/opencv_contrib
else
    cd opencv_contrib
    git pull --rebase
    cd ..
fi
RepoSource=opencv
pushd Build/$RepoSource
CMAKE_OPTIONS='-DBUILD_PERF_TESTS:BOOL=OFF -DBUILD_TESTS:BOOL=OFF -DBUILD_DOCS:BOOL=OFF  -DWITH_CUDA:BOOL=ON -DBUILD_EXAMPLES:BOOL=OFF -DINSTALL_CREATE_DISTRIB=ON'
cmake -A "x64" -G"$CMAKE_CONFIG_GENERATOR" $CMAKE_OPTIONS -DOPENCV_EXTRA_MODULES_PATH="$myRepo"/opencv_contrib/modules -DCMAKE_INSTALL_PREFIX="$myRepo"/install/"$RepoSource" "$myRepo/$RepoSource"
echo "************************* $Source_DIR -->debug"
cmake --build .  --config debug
echo "************************* $Source_DIR -->release"
cmake --build .  --config release
cmake --build .  --target install --config release
cmake --build .  --target install --config debug
popd

Things I modified: I changed the config generator to Visual Studio 16 2019, I added -A x64 to build on that architecture, and I changed -DWITH_CUDA:BOOL=OFF to -DWITH_CUDA:BOOL=ON.

It took over 6 hours (guess it was only using one thread) but I finally got everything set up and switched my OPENCV_DIR over. But now my simple program no longer works!

[ WARN:0] global D:\lib\opencv\modules\videoio\src\cap_msmf.cpp (686) CvCapture_MSMF::initStream Failed to set mediaType (stream 0, (0x0 @ 1) (HRESULT -2147024809)
OpenCV(4.2.0-dev) Error: Assertion failed (!_src.empty()) in cv::cvtColor, file D:\lib\opencv\modules\imgproc\src\color.cpp, line 182

The video source can't be read, so the dimensions are set to 0x0 which triggers an assert later on. Debugging the video capture initialization, it looks like it's only considering 3 backends: GSTREAMER, MSMF, and DSHOW. I don't have DLLs for GSTREAMER so I assume that won't work. MSMF passed the test, but not before issuing the initStream warning. It looks like CvCapture_MSMF::configureOutput is the culprit, formats.findBest returns the 0x0 invalid format.

Upon closer inspection it looks like the precompiled libraries also use MSMF, and it reads the format correctly:

+       cv::IVideoCapture   {...}   opencv_world420d.dll!cv::IVideoCapture
        MF  {...}   opencv_world420d.dll!`anonymous-namespace'::Media_Foundation &
+       filename    ""  opencv_world420d.dll!std::string
        camid   0   int
        captureMode MODE_HW (1) opencv_world420d.dll!CvCapture_MSMF::MSMFCapture_Mode
+       D3DDev  {p={0x000001e71836e9f8 <Information not available, no symbols loaded for d3d11.dll>} }  opencv_world420d.dll!`anonymous-namespace'::ComPtr<ID3D11Device>
+       D3DMgr  {p={0x000001e717eca580 <Information not available, no symbols loaded for mfplat.dll>} } opencv_world420d.dll!`anonymous-namespace'::ComPtr<IMFDXGIDeviceManager>
+       videoFileSource {p={0x000001e71c56d260 <Information not available, no symbols loaded for mfreadwrite.dll>} }    opencv_world420d.dll!`anonymous-namespace'::ComPtr<IMFSourceReader>
        dwStreamIndex ...