Ftahir192's profile - activity

Thanks for the edit, that is perfect! When I call minMaxLoc, the coordinates of the object will be in terms of the new, shrinked image. How can I make it such that these coordinates map to the full sized image? I am drawing a rectangle over the frame to show the location of the object being matched.

Hi. Thank you for the reply. I'm wondering if you can explain the numbers you used to calculate the rectangles parameters? Thanks.

Hi.

I'm trying to speed up matchTemplate() for Android. I have already downsampled the image and template to half the original size, but the matching is still somewhat (understandably) slow on some older devices.

I acknowledged that I can pass in a mask to the matchTemplate() method. I'm trying to utilize this in the following way:

1) I know the location of the object from the previous frame, lets say its (180, 30). 2) I want to search only around some specified distance around this point, rather than the whole frame. 3) I create a region of interest based on this distance.

My question is, how can I utilize this roi such that I can take the current image being searched, and set everything outside the ROI to 0? I can then pass this in as a mask, and thus matchTemplate() would only search the image where pixel values are not 0 (so everything in the ROI)?

I have something such as the following:

cameraFrameROI = new Rect(topLeftX, topLeftY, width, height);
Mat mask = new Mat(mCameraMatResized);
// mask = 0 if not part of ROI, otherwise keep original value
Imgproc.matchTemplate(new Mat(mCameraMatResized, cameraFrameROI), mTemplateMatResized, mResult, mMatchMethod);

But I now need to utilize the ROI to create the actual mask.

Would this speed up template matching, and if so, I'm hoping if anyone can provide pointers.

Thanks.

Ah I see, I'm calling it for every frame! I'll look into this, I'll see firstly if downscaling helps. Thanks a lot for your help!

Hmm, I've been advised on SO to stick to matchTemplate as it is already well optimised.

Sounds good, if I also scale the template matching, I'd be doing only 1/4 of the evaluations as opposed to non-scaling, which is a massive difference in and of itself.

Regarding downscaling, would using the resize() function built into OpenCV be considered a fast-scaling method, or can I do better?

If I go with downscaling, what image and template size is recommended? As stated currently, the image size is 1280x720, and template is normally no more than half of that resolution. Thanks for the help.

Wouldn't this affect the end result if I decrease the source image size? Furthermore, I would have to resize it for each image that comes in, which could be 20+ resizes a second. Would that not be costly also? The size of the source images are 1280x720 currently.

I am using the template size as an indicator to what size the rectangle bounding box would be, and if I decrease the template size further, I'm not sure how I could indicate what size the bounding box (to indicate detected object) should be drawn at.

Hi.

I'm using OpenCV4Android to try out template matching on an android phone. Essentially what I'm doing is taking a picture of some object that I wish to detect, cropping it and saving it as a template.

I then use my android camera and a surface view to constantly get images from the camera. I am then applying template matching on each image, converting the image to a Mat first. However, when applying template matching I only get around 3-4 fps.

What I am essentially doing is this:

** mCameraMat = inputFrame.rgba();

            int matchMethod = Imgproc.TM_CCOEFF_NORMED;

            // mTemplateMat resized in terms of video size in prepareMediaRecorder.
            // Very hacky solution so need to fix it!
            int result_cols = mCameraMat.cols() - mTemplateMat.cols() + 1;
            int result_rows = mCameraMat.rows() - mTemplateMat.rows() + 1;


            mResult = new Mat(result_rows, result_cols, CvType.CV_32F);

            // Move this to a new thread.
            Imgproc.matchTemplate(mCameraMat, mTemplateMat, mResult, matchMethod);
            Core.normalize(mResult, mResult, 0, 1, Core.NORM_MINMAX, -1, new Mat());

            // Localizing the best match with minMaxLoc
            MinMaxLocResult mmr = Core.minMaxLoc(mResult);

            Point matchLoc;
            if (matchMethod == Imgproc.TM_SQDIFF || matchMethod == Imgproc.TM_SQDIFF_NORMED) {
                matchLoc = mmr.minLoc;
            } else {
                matchLoc = mmr.maxLoc;
            }


            // Draw a boundary around the detected object.
            Imgproc.rectangle(mCameraMat, matchLoc, new Point(matchLoc.x + mTemplateMat.cols(),
                matchLoc.y + mTemplateMat.rows()), new Scalar(TrackingActivity.r, TrackingActivity.g,
                TrackingActivity.b, TrackingActivity.a), 2);

**

Where mTemplateMat is the template bitmap image converted into a Mat object.

The bottleneck is on the line

Imgproc.matchTemplate(mCameraMat, mTemplateMat, mResult, matchMethod);

If I remove that line, I get around 25 fps, which is much more acceptable. I'd be fine with anything above 13-14. I understand that template matching is a very expensive process and doing it every frame can be costly. I have tried to do it every 20 frames, but it still slows down the processing considerably, and the end video looks worse as there is a constant transition from a smooth fps display to a low fps display.

What are my options in optimising matchTemplate? Any tips are much appreciated.

I've been attempting to record the JavaCameraView for a few days now, to no avail. I'm trying to use MediaRecorder to record.

I have a JavaCameraView in my layout file, have added a CvCameraViewListener as follows:

mCameraView.enableView();
mCameraView.setCvCameraViewListener(this);

where mCameraView = (CameraBridgeViewBase) findViewById(R.id.cameraPreview);

and R.id.camera_preview is

<org.opencv.android.JavaCameraView
        android:id="@+id/camera_preview"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:visibility="visible"
        opencv:show_fps="false"
        opencv:camera_id="back"/>

I am therefore getting a preview on the screen correctly, and onCameraFrame is being called.

The problem I am having with using MediaRecorder and the Camera API to record is that JavaCameraView.java already has an instance of the camera open, and so I cannot open another one to record with.

The mCamera object in JavaCameraViewis protected, so I've attempted to make a class CameraView that extends JavaCameraView, and through there, call the mCamera.lock()/mCamera/unlock() methods needed. This isn't working either.

My last resort was to record the surface instead, and this is being done as follows:

// Step 1: Set sources
mMediaRecorder.setAudioSource(MediaRecorder.AudioSource.DEFAULT );
mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);

// Step 2: Set a CamcorderProfile (requires API Level 8 or higher)
mMediaRecorder.setProfile(CamcorderProfile.get(CamcorderProfile.QUALITY_HIGH));

// Step 3: Set output file (handled in CameraHelper)
String date = CreateFiles.getDate();
mRootFolder = CreateFiles.getOutputFolder(date);
Log.i(TAG, "Output folder is " + mRootFolder.toString());

mMediaFile = CreateFiles.getOutputMediaFile(
            CreateFiles.MEDIA_TYPE_VIDEO, mRootFolder, date);
mDataFile = CreateFiles.getOutputDataFile(mRootFolder, date);

mMediaRecorder.setVideoSize(mCameraView.getmFrameWidth(), mCameraView.getmFrameHeight());
mMediaRecorder.setOutputFile(mMediaFile.getPath());

mMediaRecorder.prepare();
mMediaRecorder.start();

However when I stop the MediaRecorder, it crashes with the following stack trace:

**10-14 00:28:48.239 23590-23590/uk.ac.objecttracker E/MediaRecorder: stop failed: -1007
10-14 00:28:48.239 23590-23590/uk.ac.objecttracker D/AndroidRuntime: Shutting down VM
10-14 00:28:48.249 23590-23590/uk.ac.objecttracker E/AndroidRuntime: FATAL EXCEPTION: main
                                                                             Process: uk.ac.ed.faizan.objecttracker, PID: 23590
                                                                           java.lang.AssertionError: java.lang.RuntimeException: stop failed.
                                                                               at uk.ac.objecttracker.CameraPreview.releaseMediaRecorder(CameraPreview.java:122)
                                                                               at uk.ac.objecttracker.TrackingActivity.onClick(TrackingActivity.java:127)
                                                                               at android.view.View.performClick(View.java:5702)
                                                                               at android.view.View$PerformClick.run(View.java:22541)
                                                                               at android.os.Handler.handleCallback(Handler.java:739)
                                                                               at android.os.Handler.dispatchMessage(Handler.java:95)
                                                                               at android.os.Looper.loop(Looper.java:158)
                                                                               at android.app.ActivityThread.main(ActivityThread.java:7229)
                                                                               at java.lang.reflect.Method.invoke(Native Method)
                                                                               at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1230)
                                                                               at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1120)
                                                                            Caused by: java.lang.RuntimeException: stop failed.**

And I have no idea what gives. So now I'm asking for help here and would greatly appreciate anybody give some hints as to what I might be doing wrong in my code above, or how I can use the Camera to record (even though it's being used by JavaCameraView).

Thanks a lot in advance.

Hi.

I've come across an error in the Opencv4android SDK which I can't quite explain. This bug only shows on my Google Tango tablet, but is non existent when I run my app on my Galaxy S6.

10-12 17:41:53.625 24608-24847/uk.ac.ed.faizan.objecttracker E/SurfaceHolder: Exception locking surface
                                                                              java.lang.IllegalArgumentException
                                        at android.view.Surface.nativeLockCanvas(Native Method)
                                        at android.view.Surface.lockCanvas(Surface.java:244)
                                        at android.view.SurfaceView$4.internalLockCanvas(SurfaceView.java:818)
                                        at android.view.SurfaceView$4.lockCanvas(SurfaceView.java:786)
                                        at org.opencv.android.CameraBridgeViewBase.deliverAndDrawFrame(CameraBridgeViewBase.java:409)
                                        at org.opencv.android.JavaCameraView$CameraWorker.run(JavaCameraView.java:350)

The error originates from JavaCameraView and I've done some research but can't find anything about it. It's not part of my own code, so I'm at a bit of a loss here.

Any advice on what this could be? I can't even explain the error well because I'm not sure exactly what's causing it.

Any help is much appreciated.