Shervin Emami's profile - activity

MrPilli's answer is quite good, but has a few incorrect characters in the command. This is the command to install Gstrea

If most of your faces are around 20x20 (eg: 17x17 and above) then GilLevi's idea of upscaling the image should be fine, especially if you customize all the optional parameters you call OpenCV's cascade detector with, to do a more fine-grained search. You can also apply multiple face detectors and combine the results, if you would rather get more faces detected and don't mind if it takes longer and gives more non-faces too. eg: use several of the Haar face classifier that come with OpenCV and also use the LBP face classifier and perhaps also create your own LBP classifier, then you will have faces from many detectors and you combine the ones that are almost identical rectangles.

But if you want to detect faces of around 10x10 then it is much harder to do reliably, whether you use OpenCV's face detector or a different one. There is very little information the detector can get from 10x10 images, so any algorithm is going to give a lot of non-face detections.

There are several ways to make OpenCV code faster on Android, particularly using NEON (OpenCV For Tegra) and multi-threading (including TBB that is also available in OpenCV For Tegra).

But it sounds like first you should work on your overall pipeline, because you said you want to display the live camera preview without lag from the image processing. So for that, you should modify your code so that Android automatically displays your camera preview using hardware acceleration, without any lag. Most iOS & Android devices should support this, since the camera typically generates both a camera stream and a preview stream, where the preview stream is designed to be shown on the screen without being lagged from whatever happens to the high-res camera stream.

Once you have that working, you can look into speeding up your image processing using multi-threading and/or NEON optimizations, either by yourself or simply using OpenCV For Tegra (on a Tegra based device).

Recent versions of OpenCV have many small library modules instead of a few big ones. If you use CMake to build your project then it doesn't matter because CMake finds them all for you, but unfortunately if you don't use CMake for your project then you have to list all the lib files manually, and change it if you want the debug libraries, and change all of them again every time you use a slightly different version of OpenCV :-(

If you use Linux or Mac then if you install "pkg-config" before building OpenCV then the command "pkg-config opencv --libs" gives you the GCC flags for all the lib files (and "pkg-config opencv --cflags" gives you it for the header files). For Windows, you should look in the generated lib folder to see all the names. OR use CMake to build your project, then you don't have to worry about modules, and your build settings will be portable between Windows, Linux & Mac!

They already stopped developing new functionality for the C API, and yes they want to completely remove C API from future versions of OpenCV, OpenCV developers are getting ready to move to OpenCV 3.0 that is quite different to 2.0, and OpenCV 3.0 would no longer have a C API at all. In fact I don't think even the existing C++ API will be present much in OpenCV 3.0, it would be a whole new C++ API!

The defaults should be good enough for most people. But if your CPU supports a certain version of MMX & SSE and/or multi-core (and modern CPUs almost all will), then enable the WITH_SSE settings and WITH_TBB for those optimizations. And if you want to use the GPU (NVIDIA CUDA) or OCL (OpenCL) module of OpenCV, then make sure you have the CUDA and/or OpenCL SDK installed and then enable WITH_CUDA or WITH_OCL in CMake. The rest of the settings usually don't need worrying about.

It also works for me (on Debian). Maybe your internet connection had some problems at the time.

I solved a similar problem by setting the whole image to black before copyTo(), since setting it all to black is a lot quicker than analyzing each pixel to decide if it should be changed or not:

// Clear the output image to black, so that the mask will be black (ie: not drawn).
memset((char*)dst.data, 0, dst.step * dst.rows);

// Use the srcColor image, except for the masked parts that we will leave black.
srcColor.copyTo(dst, mask);

OpenCV currently has several times more CUDA functions than OpenCL functions and it is a lot less buggy since it has been officially supported for several years, whereas the OpenCL module has only recently been integrated.

Like berak mentioned, you don't need to use OpenCVManager, you can statically link directly to OpenCV.

But it is recommended to use OpenCVManager, for many reasons such as if your user installs multiple apps using OpenCV then they only download OpenCV once to their device instead of multiple times, and because OpenCV gets speedups & bugfixes over time, and because OpenCVManager can potentially make use of Tegra hardware acceleration if it is used on Tegra devices.

If you are saying that normal OpenCV works fine in your Eclipse NDK project, but your custom ORB code is full of errors, then tt sounds like maybe you just need to tell Eclipse where to find your custom header & source files. Unfortunately Eclipse doesn't follow your Android.mk files, so if you want to compile & run your code from within Eclipse you should add your header folders both to your Android.mk file AND your Eclipse project settings.

If you can't be bothered updating Eclipse, then you can always just compile it using "ndk-build", since that does make use of your Android.mk file. Or try changing the severity of errors, like ShadowTS mentioned.

Yes like Steven said, the very first call to a GPU function will take very long, say around 1 second for the GPU to start your program, but then you can use it at full speed after that first call. So anytime you measure speed of GPU, you should do a GPU call first, such as one iteration of your loop, since this will take nearly 1 second, and THEN you start timing all the iterations after that.

Just like when measuring the speed of OpenCV CPU code, you don't measure from the time you click on the icon in the Start Menu until the whole app has finished, you wait till the CPU has initialized your program and then you measure within your CPU code. The difference with GPU is that unfortunately the GPU module does not have a function like "initializeGPU()" to do this, instead it is initialized automatically when you first do a calculation on the GPU.

Another problem with most GPUs is that you need to transfer your image to GPU memory from main memory and then also transfer the result back to main memory (ie: upload() & download()), and if you are doing something simple like adding numbers, this transfer can take more time than the calculations. But you often should still include this transfer time in your timing measurements, since you will need to do that transfer on every frame, not just for the first frame.

Search the web on how to perform "Histogram Matching" (also called "Histogram Specification", it lets you force the color histogram of one image to be the color histogram of another image.

Face recognition is not a solved problem where you can just run a tool and get reliable results. New algorithms & techniques are coming out each year, look at any computer vision related conference like CVPR and you will see many new face recognition techniques each year, because the results from current techniques are typically not satisfactory for real-world conditions.

The algorithms in OpenCV's FaceRecognizer are simple enough that you can learn without having a PhD in face recognition, but they still can give decent results. But it depends completely on your training images and your input images. If your training data does not give a very good representation of your actual input then it will give terrible results (even 0%) and if your training data does cover your actual conditions well then you can get good results (even 100%), so the most important thing is to have good training data.

This usually means a combination of 2 things: good input, and good preprocessing:

1. Good input typically means restricting your conditions a lot so that the input images during training are almost identical to the input images during tests. eg: Making sure the people are all in the same room while training & testing, and are all facing the same direction, with the lighting all the same, and using the same camera, with the camera in the same position, and doing the same facial behaviour such as always smiling or always neutral, etc.
2. Good preprocessing typically means applying many face preprocessing steps to make sure the slight changes in conditions will have less effect, such as standardizing the brightness & contrast and lighting distribution and size and position and rotation, etc.

If you follow all these sorts of steps, to make it as much like "laboratory conditions" as possible, then you should get good recognition rates (say 50% to 90% reliability). But if you simply try to use use a face recognition algorithm on "real-world conditions" (ie: without all these types of restrictions and preprocessing), then you will usually get very bad results (< 20% reliability).

Another option is to perform 3D face recognition, or perhaps face recognition from video, or combine face recognition with some other information source (such as voice recognition or shirt recognition or using their friends list if you are performing face recognition on a social media website).

You need to look at the data between each OpenCV call to find the problem. For example, right now you aren't sure if the XML files were loaded properly, or if your image (or camera) is correct, or if detectMultiScale() actually found something. Try putting printf statements between them and possibly save images to file (or display on screen) to find which part is giving you the problem.

There are 3 blob detection libraries that were available some years ago that haven't been updated: cvBlobsLib, cvblob, and Dave Grossman's Blob Detection code. Since they are all outdated, none of them come prebuilt for VS2010, so it means you have to compile them yourself with VS2010. Several useful links for this are:

• "http://stackoverflow.com/questions/8503748/how-to-configure-visual-studio-2010-with-opencv-cvblobs-library"
• "http://stackoverflow.com/questions/14558452/opencv-difficulty-in-working-with-cvblobs-library?lq=1"

• "http://www.shervinemami.info/files/BlobAnalysisPackage_DG.zip" (Dave Grossman's Blob library, since the original file is not available anymore on the Yahoo! OpenCV site)

• If you search around the web you will find some people have posted prebuilt versions of the blob detection libraries for various compilers, since many other VS2010 users and OpenCV 2.3+ users are having the same problems, but I recommend to just compile it yourself.

Yes, like Berak said, you should look for GSOC items near the bottom of the OpenCV developments page at "http://code.opencv.org/projects/opencv/wiki/2013", it shows what the official OpenCV developers are thinking about for potential GSOC projects. If there is any of those that you like then start learning about it, and then when you are more familiar with it you can tell the OpenCV developers that you are interested in 1 or 2 of the projects and you have some experience in it already (ideally you should show real proof if you have already started working on it).

There is a tutorial about Optical Character Recognition (OCR) at "http://blog.damiles.com/2008/11/basic-ocr-in-opencv/", and a new tutorial about Automatic Number Plate Recognition (ANPR) by the same guy in the book "Mastering OpenCV with Practical Computer Vision Projects".

There are several OpenCV based face recognition tutorials on the web for beginners:

• my old one "http://shervinemami.info/faceRecognition.html"
• my new one (Chapter 8 of the book "Mastering OpenCV with Practical Computer Vision Projects")
• the tutorials at "http://docs.opencv.org/modules/contrib/doc/facerec/index.html".

I have also noticed this problem, that the FaceRecognizer's method of determining if someone is known or unknown is too unreliable. The task can be treated as "Face Verification", where you have a photo of a person and you need to decide if it really is that person or not. I wrote a more reliable solution for the Mastering OpenCV book (code is at "https://github.com/MasteringOpenCV/code/tree/master/Chapter8_FaceRecognition"):

// Generate a face approximation by back-projecting the eigenvectors & eigenvalues.
Mat reconstructedFace;
reconstructedFace = reconstructFace(model, preprocessedFace);

// Verify whether the reconstructed face looks like the preprocessed face, otherwise it is probably an unknown person.
double similarity = getSimilarity(preprocessedFace, reconstructedFace);

It usually works quite well for Eigenfaces/PCA because you have a lot of images available to reconstruct the face. For Fisherfaces it won't work quite as well but it still seems to be much more reliable than FaceRecognizer's existing solution of just loking at the Euclidean distance as the confidence metric to threshold.

Feel free to replace the threshold technique of FaceRecognizer with some of the code from the book if you think it is a good idea.

-Shervin Emami.

You can definitely read & write files from native code, you just need to make sure you have the correct path to the file, since Android only let's your app access files in a few possible locations (the same is true for Java code). For example, you can copy the XML file to the base of your SD card, then access it as something like "/sdcard/lbpfrontalface.xml", etc. There are also ways to put the XML file in your APK instead of the SD card, etc.

There is a Contribute page on the user website showing how to contribute to OpenCV in general.

Most importantly, since you already have code you want to submit, you should follow all the steps on the Code Submissions page of the developer website.

The most up-to-date installation information is currently kept in the Install Guide. It mentions how to obtain the latest code being developed (updated each day).

Right now you can checkout sources with the following command in your shell:

svn co http://code.opencv.org/svn/opencv/trunk/opencv

It is also possible to access specific stable versions through SVN, such as OpenCV v2.4.2. You can see the list of tags by viewing the page "http://code.opencv.org/svn/opencv/tags/" in a web browser.

Note that each tag has an "opencv" folder and an "opencv_extra" folder. You normally just want the "opencv" folder, but if you want to download large sets of data for running the OpenCV tests, you can also get "opencv_extra".