Yesterday I got set up with OpenCV (precompiled libraries for windows) and everything was working great! I had a little program that would read a video feed from my Valve Index, undistort each lens, and then display it back. It was super slow though, so I looked into using CUDA to speed it up and found that the precompiled libraries didn't include CUDA support. So I followed the tutorial for CMake and git-bash, and used this slightly modified build script:
#!/bin/bash -e
myRepo=$(pwd)
CMAKE_CONFIG_GENERATOR="Visual Studio 16 2019"
if [ ! -d "$myRepo/opencv" ]; then
echo "cloning opencv"
git clone https://github.com/opencv/opencv.git
mkdir -p Build/opencv
mkdir -p Install/opencv
else
cd opencv
git pull --rebase
cd ..
fi
if [ ! -d "$myRepo/opencv_contrib" ]; then
echo "cloning opencv_contrib"
git clone https://github.com/opencv/opencv_contrib.git
mkdir -p Build/opencv_contrib
else
cd opencv_contrib
git pull --rebase
cd ..
fi
RepoSource=opencv
pushd Build/$RepoSource
CMAKE_OPTIONS='-DBUILD_PERF_TESTS:BOOL=OFF -DBUILD_TESTS:BOOL=OFF -DBUILD_DOCS:BOOL=OFF -DWITH_CUDA:BOOL=ON -DBUILD_EXAMPLES:BOOL=OFF -DINSTALL_CREATE_DISTRIB=ON'
cmake -A "x64" -G"$CMAKE_CONFIG_GENERATOR" $CMAKE_OPTIONS -DOPENCV_EXTRA_MODULES_PATH="$myRepo"/opencv_contrib/modules -DCMAKE_INSTALL_PREFIX="$myRepo"/install/"$RepoSource" "$myRepo/$RepoSource"
echo "************************* $Source_DIR -->debug"
cmake --build . --config debug
echo "************************* $Source_DIR -->release"
cmake --build . --config release
cmake --build . --target install --config release
cmake --build . --target install --config debug
popd
Things I modified: I changed the config generator to Visual Studio 16 2019, I added -A x64 to build on that architecture, and I changed -DWITH_CUDA:BOOL=OFF to -DWITH_CUDA:BOOL=ON.
It took over 6 hours (guess it was only using one thread) but I finally got everything set up and switched my OPENCV_DIR over. But now my simple program no longer works! Working output with precompiled libaries:
[ INFO:0] global C:\build\master_winpack-build-win64-vc15\opencv\modules\videoio\src\videoio_registry.cpp (187) cv::`anonymous-namespace'::VideoBackendRegistry::VideoBackendRegistry VIDEOIO: Enabled backends(7, sorted by priority): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); MSMF(970); DSHOW(960); CV_IMAGES(950); CV_MJPEG(940)
[ INFO:0] global C:\build\master_winpack-build-win64-vc15\opencv\modules\videoio\src\backend_plugin.cpp (353) cv::impl::getPluginCandidates Found 2 plugin(s) for GSTREAMER
[ INFO:0] global C:\build\master_winpack-build-win64-vc15\opencv\modules\videoio\src\backend_plugin.cpp (172) cv::impl::DynamicLib::libraryLoad load D:\opencv\build\x64\vc15\bin\opencv_videoio_gstreamer420_64.dll => FAILED
[ INFO:0] global C:\build\master_winpack-build-win64-vc15\opencv\modules\videoio\src\backend_plugin.cpp (172) cv::impl::DynamicLib::libraryLoad load opencv_videoio_gstreamer420_64.dll => FAILED
Output with self-compiled CUDA libaries:
[ INFO:0] global D:\lib\opencv\modules\videoio\src\videoio_registry.cpp (187) cv::`anonymous-namespace'::VideoBackendRegistry::VideoBackendRegistry VIDEOIO: Enabled backends(7, sorted by priority): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); MSMF(970); DSHOW(960); CV_IMAGES(950); CV_MJPEG(940)
[ INFO:0] global D:\lib\opencv\modules\videoio\src\backend_plugin.cpp (353) cv::impl::getPluginCandidates Found 2 plugin(s) for GSTREAMER
[ INFO:0] global D:\lib\opencv\modules\videoio\src\backend_plugin.cpp (172) cv::impl::DynamicLib::libraryLoad load D:\lib\Install\opencv\x64\vc16\bin\opencv_videoio_gstreamer420_64.dll => FAILED
[ INFO:0] global D:\lib\opencv\modules\videoio\src\backend_plugin.cpp (172) cv::impl::DynamicLib::libraryLoad load opencv_videoio_gstreamer420_64.dll => FAILED
[ WARN:0] global D:\lib\opencv\modules\videoio\src\cap_msmf.cpp (686) CvCapture_MSMF::initStream Failed to set mediaType (stream 0, (0x0 @ 1) (HRESULT -2147024809)
OpenCV(4.2.0-dev) Error: Assertion failed (!_src.empty()) in cv::cvtColor, file D:\lib\opencv\modules\imgproc\src\color.cpp, line 182
The video source can't be read, so the dimensions are set to 0x0 which triggers an assert later on. Debugging the video capture initialization, it looks like it's only considering 3 backends: GSTREAMER, MSMF, and DSHOW. I don't have DLLs for GSTREAMER so I assume that won't work. MSMF passed the test, but not before issuing the initStream warning. It looks like CvCapture_MSMF::configureOutput is the culprit, formats.findBest returns the 0x0 invalid format.
Here's my test program:
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/calib3d.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio.hpp>
#include <iostream>
#include <thread>
static void read_config(const char* filename, cv::Mat &cameraMatrix, cv::Mat &distCoeffs)
{
cv::FileStorage fs(filename, cv::FileStorage::READ);
if (!fs.isOpened())
{
std::cout << "Could not open the configuration file: \"" << filename << "\"" << std::endl;
return;
}
fs["camera_matrix"] >> cameraMatrix;
fs["distortion_coefficients"] >> distCoeffs;
}
int main(int argc, char** argv)
{
cv::VideoCapture video(0);
cv::Mat frame;
cv::Mat temp;
cv::Mat cameraMatrix_L, distCoeffs_L;
read_config("index_left_cropped.xml", cameraMatrix_L, distCoeffs_L);
cv::Mat cameraMatrix_R, distCoeffs_R;
read_config("index_right_cropped.xml", cameraMatrix_R, distCoeffs_R);
cv::namedWindow("Display window", cv::WINDOW_AUTOSIZE); // Create a window for display.
while (true)
{
if (!video.read(frame))
continue;
/*
cv::undistort(frame(cv::Rect(0, 0, 800, 800)), temp, cameraMatrix_L, distCoeffs_L);
temp.copyTo(frame(cv::Rect(0, 0, 800, 800)));
cv::undistort(frame(cv::Rect(800, 0, 800, 800)), temp, cameraMatrix_R, distCoeffs_R);
temp.copyTo(frame(cv::Rect(800, 0, 800, 800)));
*/
cv::imshow("Display window", frame); // Show our image inside it.
char c = (char)cv::waitKey(16); // Wait for a keystroke in the window
if (c == 27)
break;
}
return 0;
}
Is MSMF just not compatibile with my video source? Is there a way I can get it to use FFmpeg (preferably without going through the 6+ hour recompile)?
