Barry Thomas's profile - activity

Hi

My OpenCV demo on Google Play, a superset of some of the demo apps, gives me a blank screen on Android 4.2 I have tried using OpenCV 2.4.3 and 2.4.0 with the same result. Here's my stack trace:

02-28 14:19:55.318: D/dalvikvm(10526): Trying to load lib /data/app-lib/barry.opencvdemo-1/libopencv_java.so 0x415d0b30
02-28 14:19:55.318: D/dalvikvm(10526): Shared lib '/data/app-lib/barry.opencvdemo-1/libopencv_java.so' already loaded in same CL 0x415d0b30
02-28 14:19:55.318: D/CAMERA_ACTIVITY(10526): Library name: libopencv_java.so
02-28 14:19:55.318: D/CAMERA_ACTIVITY(10526): Library base address: 0x7406a000
02-28 14:19:55.338: D/CAMERA_ACTIVITY(10526): Libraries folder found: /data/app-lib/barry.opencvdemo-1/
02-28 14:19:55.338: D/CAMERA_ACTIVITY(10526): CameraWrapperConnector::connectToLib: folderPath=/data/app-lib/barry.opencvdemo-1/
02-28 14:19:55.338: E/CAMERA_ACTIVITY(10526): ||libnative_camera_r2.3.3.so
02-28 14:19:55.338: E/CAMERA_ACTIVITY(10526): ||libnative_camera_r4.0.3.so
02-28 14:19:55.338: E/CAMERA_ACTIVITY(10526): ||libnative_camera_r2.2.0.so
02-28 14:19:55.338: E/CAMERA_ACTIVITY(10526): ||libnative_camera_r3.0.1.so
02-28 14:19:55.338: E/CAMERA_ACTIVITY(10526): ||libnative_camera_r4.0.0.so
02-28 14:19:55.338: D/CAMERA_ACTIVITY(10526): try to load library 'libnative_camera_r4.0.3.so'
02-28 14:19:55.338: D/CAMERA_ACTIVITY(10526): CameraWrapperConnector::connectToLib ERROR: cannot dlopen camera wrapper library /data/app-lib/barry.opencvdemo-1/libnative_camera_r4.0.3.so, dlerror="Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "_ZN7android14SurfaceTextureC1Ejbjb" referenced by "libnative_camera_r4.0.3.so"..."
02-28 14:19:55.338: D/CAMERA_ACTIVITY(10526): try to load library 'libnative_camera_r4.0.0.so'
02-28 14:19:55.348: D/CAMERA_ACTIVITY(10526): CameraWrapperConnector::connectToLib ERROR: cannot dlopen camera wrapper library /data/app-lib/barry.opencvdemo-1/libnative_camera_r4.0.0.so, dlerror="Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "_ZN7android14SurfaceTextureC1Ejbj" referenced by "libnative_camera_r4.0.0.so"..."
02-28 14:19:55.348: D/CAMERA_ACTIVITY(10526): try to load library 'libnative_camera_r3.0.1.so'
02-28 14:19:55.348: D/CAMERA_ACTIVITY(10526): CameraWrapperConnector::connectToLib ERROR: cannot dlopen camera wrapper library /data/app-lib/barry.opencvdemo-1/libnative_camera_r3.0.1.so, dlerror="Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "_ZN7android14SurfaceTextureC1Ej" referenced by "libnative_camera_r3.0.1.so"..."
02-28 14:19:55.348: D/CAMERA_ACTIVITY(10526): try to load library 'libnative_camera_r2.3.3.so'
02-28 14:19:55.348: D/CAMERA_ACTIVITY(10526): Loaded library '/data/app-lib/barry.opencvdemo-1/libnative_camera_r2.3.3.so'
02-28 14:19:55.348: D/OpenCV_NativeCamera(10526): CameraHandler::initCameraConnect(0x7436b479, 0, 0x73004b60, 0x0)
02-28 14:19:55.348: D/OpenCV_NativeCamera(10526): Connecting to CameraService v 2.3
02-28 14:19:55.698: D/OpenCV_NativeCamera(10526): Instantiated new CameraHandler (0x7436b479, 0x73004b60)

I don't get any compile or run errors, but no frames are being processed because some part of the lib is failing to load. Can anyone point me in the right direction here please? I'm struggling.

Many thanks

Barry

I suspect OpenCV is used for /far/ more morally suspect tasks... you seem to be reading a lot into the OPs question, it might be simply to ID someone who is not on the employee list, some kind of security thing, no?

Canny detects edges of things in an image, returning a black and white version of the image, where white pixels indicate part of an edge - the contours. The returned object is still an image (in effect a bitmap, if not a Bitmap).

Hough Lines takes the results from the Canny process and looks for straight sections among the highlighted edges and returns the end points of those straight lines, giving you a set of vector lines to play with.

Until the new feature comes out in the next release, here's a temporary fix:

http://answers.opencv.org/question/8639/android-getting-largest-preview-image-possible/

B.

I was looking for the same thing and @StevenPuttemans directed me here. This is another solution, based on the format of the example apps in 2.4.4:

In your layout, use two views, initially both invisible, at the same location on the screen. One uses the default camera, one the second. This code assumes you have two cameras and that neither are locked or otherwise unavailable. You'll need to add your own code to check to see if a second camera is available and if not, don't allow this code to run:

<LinearLayout
  xmlns:android="http://schemas.android.com/apk/res/android"
  xmlns:tools="http://schemas.android.com/tools"
  xmlns:opencv="http://schemas.android.com/apk/res-auto"
  android:layout_width="match_parent"
  android:layout_height="match_parent" 
  android:screenOrientation="landscape"
  >

  <barry.opencvd2.OpenCVDemo2View
    android:layout_width="0dp"
    android:layout_height="0dp"
    android:visibility="gone"
    android:adjustViewBounds="true"
    android:gravity="center_vertical"
    android:id="@+id/java_surface_view0" />

  <barry.opencvd2.OpenCVDemo2View
    android:layout_width="0dp"
    android:layout_height="0dp"
    android:visibility="gone"
    opencv:camera_id="1" 
    android:adjustViewBounds="true"
    android:gravity="center_vertical"
    android:id="@+id/java_surface_view1" />

</LinearLayout>

In your onCreate, get a handle to the two views and set up listeners for both cameras:

@Override
public void onCreate(Bundle savedInstanceState) {
  //Log.d(TAG, "called onCreate");
  super.onCreate(savedInstanceState);
  getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);

  setContentView(R.layout.tutorial2_surface_view);

  mOpenCvCameraView0 = (OpenCVDemo2View) findViewById(R.id.java_surface_view0);
  mOpenCvCameraView1 = (OpenCVDemo2View) findViewById(R.id.java_surface_view1);

  mOpenCvCameraView0.setVisibility(SurfaceView.VISIBLE);
  mOpenCvCameraView0.setCvCameraViewListener(this);
  mOpenCvCameraView0.setLayoutParams(new LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));

  mOpenCvCameraView1.setVisibility(SurfaceView.GONE);
  mOpenCvCameraView1.setCvCameraViewListener(this);
  mOpenCvCameraView1.setLayoutParams(new LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));

  OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_4, this, mLoaderCallback);
  }

Also make sure the views are disabled in onPause and onDestroy:

@Override
public void onPause() {
  super.onPause();
  if (mOpenCvCameraView0 != null) 
      mOpenCvCameraView0.disableView();
  if (mOpenCvCameraView1 != null) 
      mOpenCvCameraView1.disableView();
  }

public void onDestroy() {
  super.onDestroy();
  if (mOpenCvCameraView0 != null) 
      mOpenCvCameraView0.disableView();
  if (mOpenCvCameraView1 != null) 
      mOpenCvCameraView1.disableView();
  }

In my menu I have an option for swapping cameras, here's what it does:

if (item.getItemId() == R.id.swapcameras) {
  if (iCamera == 0) {
    mOpenCvCameraView0.setVisibility(SurfaceView.GONE);
    mOpenCvCameraView0.disableView();
    mOpenCvCameraView1 = (OpenCVDemo2View) findViewById(R.id.java_surface_view1);
    mOpenCvCameraView1.setCvCameraViewListener(this);
    mOpenCvCameraView1.setVisibility(SurfaceView.VISIBLE);

    iCamera = 1;
    }
  else {
    mOpenCvCameraView1.setVisibility(SurfaceView.GONE);
    mOpenCvCameraView1.disableView();
    mOpenCvCameraView0 = (OpenCVDemo2View) findViewById (R.id.java_surface_view0);
    mOpenCvCameraView0.setCvCameraViewListener(this);
    mOpenCvCameraView0.setVisibility(SurfaceView.VISIBLE);

    iCamera = 0;
    }
}

If anyone wants the entire source code, just let me know.

Putting my answer as an alternative to the answer on the link provided by Steven.

OK, thanks for the amusing link, and while I deserved the slap yesterday, I did search for this - honest. I even searched here - this kind of thing, which was no help: http://answers.opencv.org/questions/?search=&query=android+opencv-2.4.4+front+camera

Knowing the 'right' local search engine terms is a handy thing...

Have a good weekend and stay away from the computer :)

Hi

I can't find any good reference on how to change the 2.4.4 tutorials to work with the front facing camera. Can someone point me in the right direction please?

With this in the manifest:

<uses-feature android:name="android.hardware.camera.front" android:required="true"/>

And this in onCreate:

mOpenCvCameraView.selectCamera(1);

Which calls in OpenCVDemo2View:

public void selectCamera (int i) {
    mOpenCvCameraView.releaseCamera();
    mCamera.open(i);
    }

... I get images from the default camera - same if I don't try to select a camera. But if I use selectCamera(0) I get a message saying, "It seems that you [sic] device does not support camera (or it is locked). Application will be closed."

Any pointers to good info on this would be welcomed.

Many thanks

Barry

That's great, and you'll probably do a neater job than I did! :)

Here's what I did (thanks Alexander):

This code goes into CameraBridgeViewBase.java in order to scale the bitmap to the size of the screen. The file is at:

[your path to the sdk]\OpenCV-2.4.4-android-sdk\sdk\java\src\org\opencv\android

protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
    Mat modified;

    if (mListener != null) {
        modified = mListener.onCameraFrame(frame);
    } else {
        modified = frame.rgba();
    }

    boolean bmpValid = true;
    if (modified != null) {
        try {
            Utils.matToBitmap(modified, mCacheBitmap);
        } catch(Exception e) {
            Log.e(TAG, "Mat type: " + modified);
            Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
            Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
            bmpValid = false;
        }
    }

    if (bmpValid && mCacheBitmap != null) {
        Canvas canvas = getHolder().lockCanvas();
        if (canvas != null) {
            canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);


            /////////////////////////////////////////////////////
            ////// THIS IS THE CHANGED PART /////////////////////
            int width = mCacheBitmap.getWidth();
            int height = mCacheBitmap.getHeight();
            float scaleWidth = ((float) canvas.getWidth()) / width;
            float scaleHeight = ((float) canvas.getHeight()) / height;
            float fScale = Math.min(scaleHeight,  scaleWidth);
            // CREATE A MATRIX FOR THE MANIPULATION
            Matrix matrix = new Matrix();
            // RESIZE THE BITMAP
            matrix.postScale(fScale, fScale);

            /////////////////////////////////////////////////////

            // RECREATE THE NEW BITMAP
            Bitmap resizedBitmap = Bitmap.createBitmap(mCacheBitmap, 0, 0, width, height, matrix, false);

            canvas.drawBitmap(resizedBitmap, (canvas.getWidth() - resizedBitmap.getWidth()) / 2, (canvas.getHeight() - resizedBitmap.getHeight()) / 2, null);
            if (mFpsMeter != null) {
                mFpsMeter.measure();
                mFpsMeter.draw(canvas, 20, 30);
            }
            getHolder().unlockCanvasAndPost(canvas);
        }
    }
}

So I now have an image which looks like this:

Which was what I wanted.

Well deserved. I thank you.

Reminder to self: If the bug isn't where you are looking then it is probably somewhere else.

I can't believe it. [facepalm].

Add this to your manifest file Barry:

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>

Would somebody come and slap me please? I managed to remove the write permission from the manifest so really there shouldn't be any surprise that it wouldn't write...

Doh.

Barry

@Rui Interesting... thanks. Fortunately I'm just experimenting with OpenCV really and don't have a serious need for high speed, but 10fps is very disappointing :(

I'd be interested to know the answer to this too... I recently changed from an HTC Desire S to a Nexus 4 and despite the significant increase in processor power, I get very similar frame rates in my OpenCV apps - using the preview at the same resolution (800x480).

I expected the simple increase in computing power to give me a clear boost, even tho of course at the same time I changed from Android 2.3.x to 4.2 and OpenCV from 2.4.0 to 2.4.4, but even with very light use of OpenCV calls I never get above 10fps.

I'd be interested to know the answer to this too... I recently changed from an HTC Desire S to a Nexus 4 and despite the significant increase in processor power, I get very similar frame rates in my OpenCV apps - using the preview at the same resolution (800x480).

I expected the simple increase in computing power to give me a clear boost, even tho of course at the same time I changed from Android 2.3.x to 4.2 and OpenCV from 2.4.0 to 2.4.4, but even with very light use of OpenCV calls I never get above 10fps.

The Nexus 4 has a screen res of 1280x768 (with no title bar etc).

According to getResolutionList() the largest preview image from OpenCV (item 0 in the list) is 1280x720, so in onCameraFrame, why does the image always come through at a maximum of 800x480.

Even when I try to force the size to the maximum using setResolution(mResolutionList.get(0)) it sticks at 800x480.

Can anyone give me some clue as to what I'm doing wrong?

Many thanks

Baz

EDIT: ps I'm using 2.4.4 on Android 4.2.

EDIT: To add some explanation about the answer, here's a picture:

The total screen res on the Nexus is 1280x768, but that includes the buttons on the right, so the total screen I have to play with is 1196x768 (the red dotted line). The largest preview image reported by getResolutionList is 1280x720, but that won't fit, so despite being listed as available... it isn't really.

Next job: Work out how to scale up the largest preview which will fit (800x480 - the blue dotted line) to fill the screen without slamming performance.

Hi

This code used to work but on Android 4.2 and OpenCV 2.4.4 it fails, but I don't know why. Can anyone shed any light on it for me?

Thanks for any help.

Baz

public void SaveImage (Mat mat) {
    Mat mIntermediateMat = new Mat();

    Imgproc.cvtColor(mRgba, mIntermediateMat, Imgproc.COLOR_RGBA2BGR, 3);

    File path = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES);
    String filename = "barry.png";
    File file = new File(path, filename);

    Boolean bool = null;
    filename = file.toString();
    bool = Highgui.imwrite(filename, mIntermediateMat);

    if (bool == true)
        Log.d(TAG, "SUCCESS writing image to external storage");
    else
        Log.d(TAG, "Fail writing image to external storage");
    }

bool comes back false every time. The file path/name is as it should be ("storage/emulated/0/Pictures/barry.png") for User #0 and the directory Pictures is there.

What can make imwrite return false?

Many thanks

Baz