Kozuch's profile - activity

I want to use cv::sfm::triangulatePoints from opencv_contrib in OpenCV 3.2.0. I get following error:

OpenCV Error: Assertion failed (points2d_tmp[i].rows == 2 && points2d_tmp[i].cols == n_points) in triangulatePoints, file /home/kozuch/install/opencv_contrib-3.2.0/modules/sfm/src/triangulation.cpp, line 140 terminate called after throwing an instance of 'cv::Exception' what(): /home/kozuch/install/opencv_contrib-3.2.0/modules/sfm/src/triangulation.cpp:140: error: (-215) points2d_tmp[i].rows == 2 && points2d_tmp[i].cols == n_points in function triangulatePoints

Here is my code:

vector<cv::Point2f> points1, points2;
featureDetection(img1, points1); // my custom function
featureTracking(img1,img2,points1,points2); // my custom function

vector<vector<cv::Point2f> > sfmPoints2d;
sfmPoints2d.push_back(points1);
sfmPoints2d.push_back(points2);

//~ cv::Mat matFeatures1(points1);
//~ cv::Mat matFeatures2(points2);
//~ matFeatures1 = matFeatures1.t(); // transpose - change rows for columns
//~ matFeatures2 = matFeatures2.t();
//~ vector<cv::Mat> sfmPoints2d;
//~ sfmPoints2d.push_back(matFeatures1);
//~ sfmPoints2d.push_back(matFeatures2);

vector<cv::Mat> sfmProjMats;
sfmProjMats.push_back(projMat0);
sfmProjMats.push_back(projMat1);
cv::Mat points3d;

cv::sfm::triangulatePoints(sfmPoints2d, sfmProjMats, points3d);

sfmPoints2d does the porblem. I tried to use Mats instead of inner vectors but the error remained the same. Even tried to transpose the Mats (change rows for cols). The docs here say:

points2d - Input vector of vectors of 2d points (the inner vector is per image). Has to be 2 X N.

projection_matrices - Input vector with 3x4 projections matrices of each image.

points3d - Output array with computed 3d points. Is 3 x N.

Can someone tell me what am I doing wrong?

FOUND AN ANSWER (can not post real answer so putting it here):

Found the solution finally (the matrices in the top should be ideally empty).

cv::Mat points1Mat = (cv::Mat_<double>(2,1) << 1, 1);
cv::Mat points2Mat = (cv::Mat_<double>(2,1) << 1, 1);

for (int i=0; i < points1.size(); i++) {
    cv::Point2f myPoint1 = points1.at(i);
    cv::Point2f myPoint2 = points2.at(i);
    cv::Mat matPoint1 = (cv::Mat_<double>(2,1) << myPoint1.x, myPoint1.y);
    cv::Mat matPoint2 = (cv::Mat_<double>(2,1) << myPoint2.x, myPoint2.y);
    cv::hconcat(points1Mat, matPoint1, points1Mat);
    cv::hconcat(points2Mat, matPoint2, points2Mat);
}

vector<cv::Mat> sfmPoints2d;
sfmPoints2d.push_back(points1Mat);
sfmPoints2d.push_back(points2Mat);

vector<cv::Mat> sfmProjMats;
sfmProjMats.push_back(projMat0);
sfmProjMats.push_back(projMat1);
cv::Mat points3d;

//~ cout << "matFeatures1 = " << endl << " " << matFeatures1 << endl << endl;

cv::sfm::triangulatePoints(sfmPoints2d, sfmProjMats, points3d);

Does anyone of you have an experience with how precise smartphone cameras are manufactured on a given higher-end smartphone model (I mean phones like Samsung Galaxy S 5/6/7, iPhones etc.)? I need to employ computer vision within a community project where people will use supported smartphone models and the question is whether I will be able to pre-calibrate cameras in advance and only supply calibration data so that the "users" wont have to bother with a camera calibration themselves? I think this goes back to the manufacturing precision - if it is good enough, a pre-calibration may be possible for a given phone model. The camera focus will be fixed (probably to infinity).

I am building stereo camera device that should do the processing (=stereo matching) on board too. My requirement is to possibly do high FPS (=60 FPS) but the resolution can be as small as QVGA (320x240) or even QQVGA (160x120). Such small resolution should be faster to process. I would like to use an ARM Cortex-A9 quad-core (namely i.MX6 Quad by Freescale) for the stereo computation because I have an OEM who also supplies compatible camera modules.

My intention is to use StereoBM - I tried using it on Raspberry Pi 2 (ARM Cortex-A7 quad core). However I noticed that the lower the resolution is the worse the CPU tends to perform. I am able to control the number of threads of StereoBM (by setting nstripes variable and/or setNumThreads) and thus the number of CPU cores used (number of threads translates directly to CPU usage - can be seen on CPU utilization %).

However, I reached a state where further paralellisation (read using more cores) does not help or even is counterproductive. Here are my results for StereoBM on RPi2:

QVGA (320x240):

• 2 threads (OpenCV default) = ~50ms
• 4 threads forced = ~50ms

QQVGA (160x120):

• 1 thread (OpenCV default) = ~20ms
• 4 threads forced = ~20ms

I know 20ms at QQVGA is 50 FPS but I would like QVGA to be that fast too. Is there a way to further speed up the computation on an ARM CPU? Is it possible to use multiple threads/cores more effectively for small images? I suspect the parallelization overhead may be too large for small data and thus not effective. From my tests bigger images tend to be more effective with multithreading.

The solution would most likely be to rewrite StereoBM to FPGA but I want to avoid this.

Note: I dont really know what type of threads is used. I compiled OpenCV with TBB support but multithreading worked even without TBB support (I suspect OpenMP may have been used instead).

I am very interested in learning about Embedded Profile OpenCV support!

I am too interested in learning about Embedded profile OpenCV compatibility or whether there are plans to support it in the future or not.

As Python is a performance killer I would go with C++ for better results.

+1 for the nice free e-book. I did not test your code but I believe it may work.

@rossi_lhf: I see here http://answers.opencv.org/question/58... you probably already made OpenCL work. May I ask you what was the problem and how did you solve it? I would like to get the Embedded profile running...

@rossi_lhf: I see you struggled with OpenCL as reported here http://answers.opencv.org/question/25... but you already seem to be up and running here. May I ask you what was the problem and how did you solve it? I would like to get the Embedded profile running...

I want to use the StereoBM and StereoSGBM methods for stereo matching. It would be very useful for me to know the average computing power per source image pixel that is needed to process input stereo images within given time. The computing power could be measured in some units that are architecture independent (GFLOPS?) so that x86, GPU, ARM etc. could be compared against each other.

Example: Say I have X Mpix source stereo images and want to use StereoBM and need to get Y frames per second (FPS). The result should say I need Z GFLOPS to do that task.

I am getting started with computer stereo vision. I am using very simple VideoCapture code and trying to save two videos that are synchronized against each other. So far I wrote code to save image pair when key is hit (source here). I used Android stopwatch apps (UltraChron Stopwatch Lite and Stopwatch Timer) on my Samsung Galaxy S3 mini to realize that my saved still images are out of sync (show different time on stopwatch).

I moved the waitKey function before "cap1 >> frame1;" and increased timeout to 1000ms (waitKey(1000);) - this helped getting about 1 FPS but they still were not 100% in sync (only some frames were in sync). The webcams I use are Microsoft LifeCam Studio (resolution 640x480).

An ideal result would be a 100% synchronized stereo video stream that has X FPS. Using OpenCV 2.4.8 in Ubuntu 14.04.

Thanks for help in advance!

I did stereo camera calibration with stereo_calib.cpp sample and got the intrinsics.yml and extrinsics.yml files which contain also the Q matrix. That is the meaning of its values? This answer shows following items in the matrix: Cx, Cy, f, a and b. I guess the f is focal length but I am not sure on the other values.

Also, I need following data for my further coding:

• focal length in pixels
• principal point (u-coordinate) in pixels
• principal point (v-coordinate) in pixels
• baseline in meters

These are the parameters needed for libviso2 visual odometry.

Thanks for help in advance!

I dont know if your code is correct, but I think the calibration sample code provided with OpenCV (stereo_calib.cpp) should be correct since it is being included as "official" sample. When I look at the two images (especially the left one with a kind of "circular" error) - I have an impression I got similar error too - in my case the calibration method was not able to properly detect chessboard corners in one of my images - say I had 2 x 10 images and the 5th left image was wrong, because either not all corners were visible in the image or findChessboardCorners was not able to properly detect them. I guess the calibration sample might not be intelligent enough to toss the whole image pair (where one corner detection failed) and probably the final computation is screwed because the source image data was shifted by one image.

So make sure that corners were detected in every single image. The stereo_calib.cpp code in OpenCV 2.4.8 shows the corner detection so you should be able to control the process visually.

Another reason might be you are feeding left image where right should be - so just switch sides.

You basically have to follow Martin Peris' earlier blog post on calibration (and do the calibration), then you have to obtain a disparity image (code included in his calibration example) and RGB left image - these all will be inputs for his PCL example. So far, I made all this including the PCL point cloud to work on Ubuntu 14.04 without much effort.

Ok, I see there is a sample code (stereo_calib.cpp) within OpenCV distribution samples... probably the best we have, right?

Can someone provide me with a (correct and working) stereo calibration full example code in C++? I used so far Martin Peris's stereo calibration (here) but it is in C which it quite difficult for me to understand (but is probably working well). I also found stereo calibration by Jay Rambhia (blog post, code) which is C++ but probably has a bug (does not write YML to disk for me and also the final rectified images do not look correct).

I am quite a beginner to both C and C++ and I have a fundamental question - I come across various code snippets, some in C and some in C++. Sometimes, it would really be nice to be able to combine these two together. I saw OpenCV C code being saved as .cpp and compiled with g++ (example here).

I tried to combine some code - I guess C++ compiler is most likely able to compile C code, but I faced incompatible data structures between OpenCV C and C++ API. Is it possible to write hybrid code then?

I am quite a beginner to C/C++ development in general and I did not manage to set up OpenCV 2.4.8 with Visual Studio 2013 Express on Windows 7 64bit (LNK1104: cannot open file opencv_videostab248d.lib - although I think all is set up properly). So I switched to Ubuntu 14.04 in VirtualBox - I just installed the opencv packakage (sudo apt-get install libopencv-dev) and maybe cmake too but that was it - I am able to compile with cmake and run C++ code.

Can I use cmake on Windows too? Or is there a "low-level" opencv solution on Windows where I do not need to depend on an IDE (Visual Studio, Eclipse etc.)?