ThorbjornSomod's profile - activity

Would that be true if the cascades are trained using the exact same parameters? I know that each cascade will find different features to focus on, but the amount of stages that the sample passes through (if it does) should be equal for n m-stage classifiers?

Hi!

I have trained a couple of highly successful local binary pattern (LBP) cascade classifiers for object detection using OpenCV 3.2. So far I have used the classifiers sequentially and used the combined output as a "semi" multi-class classifier. This approach is flawed however, as an area can obviously be detected as multiple objects, which should be impossible. Because of this I am now interested in acquiring a form of confidence score for each detection by the individual cascade classifiers. With such a confidence score I could compare the overlapping regions of interest as proposed by each classifier and remove the regions with the lowest confidence score. I realize that this method is somewhat crude, but it is the best I could think of for the cascade classifiers supplied by OpenCV.

So far I have browsed a bit trying to find something that provides such a score, and I stumbled upon this thread that discusses the overloaded version of the detectMultiScale function (as found here). This discussion combined with another source makes me think that a sort of confidence measure could be calculated for each of the output Rect() objects (after non-maximum suppression) of detectMultiScale. In the last linked source the author quotes a paper that mentions a possible way of doing this:

"In the OpenCV implementation, stage_sum is computed and compared against the i stage_threshold for each stage to accept/reject a candidate window. We define the detection score for a candidate window as K*stage_when_rejected + stage_sum_for_stage_when_rejected. If a window is accepted by the cascade, we just K*last_stage + stage_sum_for_last_stage. Choosing K as a large value e.g., 1000, we ensure that windows rejected at stage i have higher score than those rejected at stage i-1." from http://vis-www.cs.umass.edu/fddb/faq....

Does this seem like a possible solution to my problem? If so, how could I go about implementing it?

PS: I know a multi-class SVM is better for this sort of a problem, so I have implemented that too. The reason I want the confidence measure is so I can compare my two implementations.

Well. In order for ANYONE to be able to help you out with this we need some more information. What kind of format is the video you're trying to open? What is the exact error message you're getting? Does the video you're trying to open have "delays" or jitter when being opened in a media player like VLC?

Yes that is correct :) The histograms that you retrieve are commonly used as the feature vectors when doing classification. So, when using the ordinary LBP operator, the histogram of a 8 bit grayscale image describes the occurrence of each of the 256 grayscale values after converting to LBP form. This can then be described as a vector of 256 elements. However, it is not common to use the ordinary LBP operator because of it's large dimension. I would look into uniform local binary patterns for a start (LBP-U2). This operator reduces the dimension from 256 -> 59 without harming the accuracy notably. Good luck!

The difference between a LBP and a LBPH is that a LBP refers to the specific binary code that you get from using the LBP operator on a given pixel in a grayscale image, while the LBPH is a histogram representing the number of occurances for each binary code for a given image patch. The basic LBP operator works as follows:

For a given pixel p of a binary image patch check the n pixels in a radius r around p in a clockwise or counter-clockwise manner. If a pixel around p has a higher intensity than p that pixel is denoted as a 1, if not a 0. For a pixel p with n = 8 and r = 1 this results in a 8 bit binary code (LBP), for example 10101011. This code is then translated to a decimal value, 171 in our example, which gets assigned to p. This is then done for the whole image patch and the occurance of each decimal value is counted. The number of occurances result in the LBPH.

Hope this helps to clarify the issue :)

Hi!

Im working on an implementation using LBP-HF features for object detection as described in this paper. The OpenCV implementation of the LBP cascade classifier currently uses the MB-LBP features described here, but I believe the LBP-HF features to be more invariant to rotation. Would it be possible to modify the code for the LBP cascade classifier supplied by OpenCV to use these features? What parts of the code would I need to change? I already have C++ code for the extraction of LBP-HF features using FFTW. Any help would be appreciated! Cheers! :)

Ah alright! Well thank you for taking a look at it! :)

I trained C-SVM models using an RBF kernel. I have also used LIBLINEAR to train models using the L2 regularized primal solver. As for LIBSVM I don't think they use a sparse representation unless you explicitly set a flag when training? The reason why I wanted to use the OpenCV implementation is that they have a lot of optimized functions that I could make use of :) So far what I have done is to extract feature vectors on a format recognized by libSVM and make wrapper functions. It works fine, but is less optimal when trying to obtain real-time performance.

Hi!

I have trained some SVM classifier models for an object recognition by the use of LibSVM. I trained them using LibSVM even though I planned to use them with OpenCV because I thought that since OpenCV's SVM implementations are (or were?) based on LibSVM this would be fine. However, OpenCV uses the .yml format for SVM models while LibSVM uses their own .model format. Is there any way to convert between the two? Does anyone know of any convenient tool that could do said conversion for me? Any help would be appreciated. Cheers!

Hi!

I have been searching around for a bit for an optimized version of the sliding window approach based on OpenCV, but I have not yet found one. What I am looking for is some version of the approach that is multi-threaded and/or employs a variable stride without basing the stride on classification confidences or the like. If anyone knows of any such existing approach a heads up would be appreciated :) Cheers!

PS: I am also utilizing the a pyramid implementation in my scheme. Would it be better to multi-thread the sliding window for each layer of the pyramid, or for each row/column in each image?

Alright thank you for the wrap up, marking this as answered :)

EDIT: I can also confirm that the edgebox implementation is indeed faster than the selective search implementation. albeit not as good at finding regions for my data.

Thank you for you answer!

I have actually considered OpenCV's implementation of the work presented here from the ximgproc module of the opencv_contrib package as a possible solution. With the selective search I could then isolate regions of interest and run my sliding window approach on those. Would the edgeboxes algorithm be faster? The selective search implementation uses multiple seconds on my 1920x1080 images. Also, when would the edgebox algorithm be merged into opencv_contrib?

Also, I have indeed implemented my own pyramid scheme. Currently it is based on OpenCVs resize() function and does not involve blurring. Would OpenCVs pyramid implementation be better? Would I run the risk of loosing information with blurring?

Hi!

I'm currently implementing a system for multi-class detection in an underwater environment. I have chosen to use the LBP-HF features found here to deal with rotation and illumination invariance. For the actual detection I have trained classifiers using libSVM, and I have successfully detected objects in my input images (1920 x 1080) by the use of a sliding window to achieve location invariance and a image scale pyramid for scale invariance. The problem with this approach is that it is very slow on my large input images, so I am trying to find ways of speeding it up. I have read a bit about efficient subwindow search (ESS), but to me it seems that this technique requires confidence scores for each prediction to be used. Is this true? Are there any other ways I could speed up my detection scheme? Any help would be appreciated!

PS: The reason why this is posted here is because the code is written for a framework using OpenCV.

@seidju: Hi! I am trying to achieve nearly the same thing (RTSP stream to OpenCV mat via LibSVM), but I have run into an issue. I have added your code to my own, but when I try to connect to the IP camera I get the following error in the vlc_log file:

-- logger module started -- stream_out_transcode error: cannot find video encoder (module:any fourcc:R24 ). Take a look few lines earlier to see possible reason. stream_out_transcode error: cannot create video chain core error: cannot create packetizer output (h264) core error: ES_OUT_RESET_PCR called

Do you know of any reason why I would be receiving this error?

@ClementD: I am currently trying to read an h264 RTSP stream using LibVLC and converting it to an OpenCV Mat. Could you share your code for the solution? Currently running OpenCV 3.2 on Ubuntu 16.04 using LibVLC 2.2.5.1 :)

True, the data transfer is the real killer, but since it seems that support has been dropped for the cuda:video_reader and cuda:video_writer functions I am stuck with that. The reason drawing the ROIs seems to take a long time for me is that each frame is in 1080p and contains up to 30 objects. I am using a Cascade Classifier based on LBP features that I trained myself 😉 I have also considered rendering using OpenGL, but the data conversions would be even worse then 🙁

Hi!

I am plain and simple wondering if there is any way to draw a ROI cv::rectangle on a cv::cuda::GpuMat? I have written a program for object detection on a live video feed, but drawing the rectangles using the CPU is slowing down the visual performance a lot. If there are any libraries out there or any nifty tricks that would let me do this on a GpuMat i would be very happy if you had any information for me!

Did you get any further on this issue? I am trying the same thing but I have not trained my network yet.

Did you get any further regarding this issue? I am trying the same thing on VS15 and getting the same error. I am also trying to build OpenCV 3.2 with CUDA 8.0 and Caffe, but I am getting this error when I try to build the solution in VS:

LNK1104: cannot open file '..\..\lib\Debug\opencv_world320d.lib

I find this particularly weird since I have successfully compiled OpenCV 3.2 with CUDA 8.0 before on the same machine (Windows 10 + GTX970).

Hmm well im building with ffmpeg. Do you think I should try building with libav, or is that an automatic configuration? Unfortunately I have to compile with Visual Studio 14 2015 version for it to be compatible with the rest of my program =/

Hmm maybe it is not supported anymore and they have forgot to remove it from the documentation then. Well, that sucks, as there are no other ways of writing directly from the GPU to disk as far as I know :( Thanks for you help anyways!

Yeah I noticed that aswell! Very weird that it is not working with my GTX970 though? Doesnt seem to find the CUDA device

Hmm when running opencv_test_video I'm getting a lot of errors, but most of them are related to the optical flow module, and not the cudacodec module. Do you want the output from opencv_test_video and opencv_test_videod?

I'll try the test project as soon as I get back to the office tomorrow morning! Although I don't think it's a compatibility issue...

Hmm I rebuilt the solution with BUILD_TESTS = ON but the problem persists. I still can't use cv::cudacodec::createVideoWriter() =/

Alright, I'll try it! Thanks :) And also yeah I am only building the Maxwell architecture for compute compatibility 5.2 (+ 5.0 which comes automatically) since im running on a GTX970

Hmmm i'll try to do that. But earlier when I tried to build OpenCV with CUDA 8.0 I got a lot of errors when building with BUILD_PERF_TESTS and BUILD_TESTS on. I'll get back to this once I rebuild. Oh btw, another quick question. When I rebuild the solution, do I need to clear the build directory of the previous build? If I have to rebuild the entire solution it's going to take a couple of hours.

I cloned OpenCV 3.2 from the git repo yes :) I added the output from getBuildInformation to the original post!

Hi!

I am trying to build OpenCV 3.2 with CUDA support for Microsoft Visual Studio 15 on Windows 10, but I have run into some trouble. I'll try to go through step by step what I have done so far to make it easier for you guys to understand if and where I have done anything wrong. So:

• I startet out downloading the pre-built binaries of OpenCV 3.2 for Windows 10 from www.opencv.org/downloads.html. I placed OpenCV at C:\opencv for an easy path.
• I then opened a command promt and navigated to the C:\opencv directory and cloned the git repository for the opencv_contrib modules into that directory. The contents of C:\opencv is so far the folders build, opencv_contrib and sources.
• I then downloaded the latest version of the CUDA Toolbox, although I have read that this comes with OpenCV 3.2 already.
• Next, I downloaded the latest version of Cmake and opened the GUI. I set the source code folder to C:/opencv/sources and the build folder to C:/opencv/build.
• I then clicked configure, waited for it to be done and changed the OPENCV_EXTRA_MODULES_PATH to C:/opencv/opencv_contrib/modules. I then clicked reconfigure and acquired the list of all modules that the source material would allow me to generate a solution for.
• Next, I ticked on the opencv_contrib modules that I need (ximgproc, xfeatures2d, plot, text and xobjdetect) and all the cuda modules. I also ticked on BUILD_CUDA_STUBS. I made sure that WITH_CUBLAS and WITH_CUDA was ticked on, set my CUDA_ARCH_BIN to 5.2 and CUDA_GENERATION to Maxwell (I'm using a GTX 960). I then generated the solution. This all went fine with no errors. Note: The generation was done with the Microsoft Visual Studio 14 2015 C++ compiler, as this is the only one I have gotten the opencv_contrib modules to work with before.
• I then opened the OpenCV.sln project and built the ALL_BUILD and INSTALL projects in both debug and release, in that order. This produced the warnings mentioned here but completed with no errors.

• I thought everything was fine at this point and decided to make an example project to test out OpenCV's capabilities with CUDA support. I made a new VS15 project and set up the include directories, linkers and library directories. Now, when I try to compile my example project I get the error:

  OpenCV Error: No CUDA support (The library is compiled without CUDA support) in throw_no_cuda, file C:\build\master_winpack-build-win64-vc14\opencv\modules\core\include\opencv2/core/private.cuda.hpp, line 97
  

Anyone else experienced anything similar? And does anyone know how to fix this? Any help would be appreciated.

Thank you! I see some differences as to what I have tried before, and I believe this will fix my issues! Marked as aswered.