Hello
I'm trying to use OpenCV VideoCapture in an Android NDK Native Activity. Unfortunately I can't find a tutorial for integrating OpenCV with Native Activity, so I was trying to discover a solution myself.
This is my Android.mk file:
LOCAL_PATH := $(call my-dir)
SELF_DIR := $(dir $(lastword $(MAKEFILE_LIST)))
OPENCV_SDK_DIR := $(SELF_DIR)/../../OpenCV-2.4.4-android-sdk
include $(CLEAR_VARS)
OPENCV_CAMERA_MODULES:=on
OPENCV_INSTALL_MODULES:=on
OPENCV_LIB_TYPE:=SHARED
include $(OPENCV_SDK_DIR)/sdk/native/jni/OpenCV.mk
LOCAL_MODULE := CameraTest
LOCAL_SRC_FILES := AppEngine.cpp ConcreteApp.cpp main.cpp
LOCAL_LDLIBS += -llog -landroid -lEGL -lGLESv1_CM
LOCAL_STATIC_LIBRARIES += android_native_app_glue
include $(BUILD_SHARED_LIBRARY)
$(call import-module,android/native_app_glue)
The problem is that the app is only starting, when I do static linking of OpenCV with OPENCV_LIB_TYPE := STATIC. With dynamic linking, the app doesn't even start and I get the following exception:
03-04 00:19:39.880: E/AndroidRuntime(3667): java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.cameratest/android.app.NativeActivity}: java.lang.IllegalArgumentException: Unable to load native library: /data/data/com.example.cameratest/lib/libCameraTest.so
I actually wouldn't mind static linking, but I discovered other problems with this method: The video capture doesn't seem to work. I tried the camera examples in the samples folder of the OpenCV Android SDK and they all work. So the problem seems to be on my side.
As I said, the video capture doesn't work when linking statically. I get the following log messages:
03-04 00:26:59.090: D/OpenCV::camera(3818): CvCapture_Android::CvCapture_Android(0)
03-04 00:26:59.090: D/OpenCV::camera(3818): Library name: libCameraTest.so
03-04 00:26:59.090: D/OpenCV::camera(3818): Library base address: 0x80400000
03-04 00:26:59.100: D/OpenCV::camera(3818): Libraries folder found: /data/data/ch.bfh.cpvr.cameratest/lib/
03-04 00:26:59.100: D/OpenCV::camera(3818): CameraWrapperConnector::connectToLib: folderPath=/data/data/ch.bfh.cpvr.cameratest/lib/
03-04 00:26:59.100: E/OpenCV::camera(3818): ||libnative_camera_r4.2.0.so
03-04 00:26:59.100: E/OpenCV::camera(3818): ||libnative_camera_r3.0.1.so
03-04 00:26:59.100: E/OpenCV::camera(3818): ||libnative_camera_r2.3.3.so
03-04 00:26:59.100: E/OpenCV::camera(3818): ||libnative_camera_r2.2.0.so
03-04 00:26:59.100: E/OpenCV::camera(3818): ||libnative_camera_r4.0.3.so
03-04 00:26:59.100: E/OpenCV::camera(3818): ||libnative_camera_r4.1.1.so
03-04 00:26:59.100: E/OpenCV::camera(3818): ||libnative_camera_r4.0.0.so
03-04 00:26:59.100: D/OpenCV::camera(3818): try to load library 'libnative_camera_r4.2.0.so'
03-04 00:26:59.110: D/OpenCV::camera(3818): CameraWrapperConnector::connectToLib ERROR: cannot dlopen camera wrapper library /data/data/ch.bfh.cpvr.cameratest/lib/libnative_camera_r4.2.0.so, dlerror="Cannot load library: reloc_library[1311]: 1386 cannot locate '_ZN7android11BufferQueueC1EbRKNS_2spINS_19IGraphicBufferAllocEEE'...
03-04 00:26:59.110: D/OpenCV::camera(3818): "
03-04 00:26:59.110: D/OpenCV::camera(3818): try to load library 'libnative_camera_r4.1.1.so'
03-04 00:26:59.110: D/OpenCV::camera(3818): CameraWrapperConnector::connectToLib ERROR: cannot dlopen camera wrapper library /data/data/ch.bfh.cpvr.cameratest/lib/libnative_camera_r4.1.1.so, dlerror="Cannot load library: reloc_library[1311]: 1386 cannot locate '_ZN7android6Camera17setPreviewTextureERKNS_2spINS_15ISurfaceTextureEEE'...
03-04 00:26:59.110: D/OpenCV::camera(3818): "
03-04 00:26:59.110: D/OpenCV::camera(3818): try to load library 'libnative_camera_r4.0.3.so'
03-04 00:26:59.110: D/OpenCV::camera(3818): CameraWrapperConnector::connectToLib ERROR: cannot dlopen camera wrapper library /data/data/ch.bfh.cpvr.cameratest/lib/libnative_camera_r4.0.3.so, dlerror="Cannot load library: reloc_library[1311]: 1386 cannot locate '_ZN7android6Camera17setPreviewTextureERKNS_2spINS_15ISurfaceTextureEEE'...
03-04 00:26:59.110: D/OpenCV::camera(3818): "
03-04 00:26:59.110: D/OpenCV::camera(3818): try to load library 'libnative_camera_r4.0.0.so'
03-04 00:26:59.110: D/OpenCV::camera(3818): CameraWrapperConnector::connectToLib ERROR: cannot dlopen camera wrapper library /data/data/ch.bfh.cpvr.cameratest/lib/libnative_camera_r4.0.0.so, dlerror="Cannot load library: reloc_library[1311]: 1386 cannot locate '_ZN7android6Camera17setPreviewTextureERKNS_2spINS_15ISurfaceTextureEEE'...
03-04 00:26:59.110: D/OpenCV::camera(3818): "
03-04 00:26:59.120: D/OpenCV::camera(3818): try to load library 'libnative_camera_r3.0.1.so'
03-04 00:26:59.120: D/OpenCV::camera(3818): CameraWrapperConnector::connectToLib ERROR: cannot dlopen camera wrapper library /data/data/ch.bfh.cpvr.cameratest/lib/libnative_camera_r3.0.1.so, dlerror="Cannot load library: reloc_library[1311]: 1386 cannot locate '_ZN7android6Camera17setPreviewTextureERKNS_2spINS_15ISurfaceTextureEEE'...
03-04 00:26:59.120: D/OpenCV::camera(3818): "
03-04 00:26:59.120: D/OpenCV::camera(3818): try to load library 'libnative_camera_r2.3.3.so'
03-04 00:26:59.120: D/OpenCV::camera(3818): Loaded library '/data/data/ch.bfh.cpvr.cameratest/lib/libnative_camera_r2.3.3.so'
03-04 00:26:59.120: D/OpenCV_NativeCamera(3818): CameraHandler::initCameraConnect(0x804884a5, 0, 0x159398, 0x0)
03-04 00:26:59.120: D/OpenCV_NativeCamera(3818): Connecting to CameraService v 2.2
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Instantiated new CameraHandler (0x804884a5, 0x159398)
03-04 00:26:59.400: I/OpenCV_NativeCamera(3818): initCameraConnect: [antibanding=auto;antibanding-values=off,50hz,60hz,auto;brightness=3;brightness-def=3;brightness-max=6;brightness-min=0;cam-mode=0;continuous-af=off;contrast=5;contrast-def=5;contrast-max=10;contrast-min=0;effect=none;effect-values=none,mono,negative,solarize,sepia,posterize,aqua;exposure-compensation=0;exposure-compensation-step=0.5;flash-mode=off;flash-mode-values=off,auto,on,torch;focal-length=3.53;focus-mode=auto;focus-mode-values=auto,infinity;front-camera-mode=mirror;front-camera-mode-values=mirror,reverse;horizontal-view-angle=54.8;iso=auto;iso-values=auto,deblur,100,200,400,800,1250;jpeg-quality=100;jpeg-thumbnail-height=480;jpeg-thumbnail-quality=75;jpeg-thumbnail-size-values=640x480,512x384,384x288,0x0;jpeg-thumbnail-width=640;max-exposure-compensation=4;max-zoom=5;meter-mode=meter-center;meter-mode-values=meter-average,meter-center,meter-spot;min-exposure-compensation=-4;picture-format=jpeg;picture-format-values=jpeg;picture-size=2592x1952;picture-size-values=2592x1952,2592x1456
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Cameras: (null)
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Picture Sizes: 2592x1952,2592x1456,2592x1936,2592x1728,2592x1552,2048x1536,2048x1360,2048x1216,2048x1152,1600x1200,1584x1056,1280x960,1280x848,1280x768,1280x720,1024x768,640x480,640x416,640x384,640x368,512x384,400x400,272x272
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Picture Formats: jpeg
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Preview Sizes: 1280x720,800x480,768x432,720x480,640x480,576x432,480x320,400x240,384x288,352x288,320x240,272x272,240x240,240x160,176x144,160x120
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Preview Formats: yuv420sp
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Preview Frame Rates: 15
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Thumbnail Sizes: 640x480,512x384,384x288,0x0
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Whitebalance Modes: auto,incandescent,fluorescent,daylight,cloudy-daylight
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Effects: none,mono,negative,solarize,sepia,posterize,aqua
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Scene Modes: (null)
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Focus Modes: auto,infinity
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Antibanding Options: off,50hz,60hz,auto
03-04 00:26:59.400: D/OpenCV_NativeCamera(3818): Supported Flash Modes: off,auto,on,torch
03-04 00:26:59.410: D/OpenCV_NativeCamera(3818): initCameraConnect: preview format is set to yuv420sp
03-04 00:26:59.410: D/OpenCV_NativeCamera(3818): Starting preview
03-04 00:26:59.650: D/OpenCV_NativeCamera(3818): Preview started successfully
03-04 00:26:59.650: D/OpenCV_NativeCamera(3818): CameraHandler::setProperty(0, 480.000000)
03-04 00:26:59.650: D/OpenCV_NativeCamera(3818): CameraHandler::setProperty(1, 800.000000)
03-04 00:26:59.650: D/OpenCV_NativeCamera(3818): CameraHandler::applyProperties()
03-04 00:26:59.650: D/OpenCV_NativeCamera(3818): CameraHandler::applyProperties()
03-04 00:26:59.650: D/OpenCV_NativeCamera(3818): CameraHandler::applyProperties(): before previousCameraHandler->closeCameraConnect
03-04 00:26:59.950: D/OpenCV_NativeCamera(3818): CameraHandler::applyProperties(): after previousCameraHandler->closeCameraConnect
03-04 00:26:59.950: D/OpenCV_NativeCamera(3818): CameraHandler::applyProperties(): before initCameraConnect
03-04 00:26:59.950: D/OpenCV_NativeCamera(3818): CameraHandler::initCameraConnect(0x804884a5, 0, 0x159398, 0x44ae5dc4)
03-04 00:26:59.950: D/OpenCV_NativeCamera(3818): Connecting to CameraService v 2.2
03-04 00:26:59.970: D/OpenCV_NativeCamera(3818): CameraHandler destructor is called
03-04 00:27:00.180: D/OpenCV_NativeCamera(3818): Instantiated new CameraHandler (0x804884a5, 0x159398)
03-04 00:27:00.180: I/OpenCV_NativeCamera(3818): initCameraConnect: Setting paramers from previous camera handler
03-04 00:27:00.190: D/OpenCV_NativeCamera(3818): Starting preview
03-04 00:27:00.420: D/OpenCV_NativeCamera(3818): Preview started successfully
03-04 00:27:00.420: D/OpenCV_NativeCamera(3818): CameraHandler::applyProperties(): after initCameraConnect, handler=0x15bdf0
03-04 00:27:00.420: E/OpenCV::camera(3818): calling (*pGetPropertyC)(0x15bdf0, 0)
03-04 00:27:00.420: D/OpenCV_NativeCamera(3818): CameraHandler::getProperty(0)
03-04 00:27:00.420: E/OpenCV::camera(3818): calling (*pGetPropertyC)(0x15bdf0, 1)
03-04 00:27:00.420: D/OpenCV_NativeCamera(3818): CameraHandler::getProperty(1)
03-04 00:27:00.500: E/OpenCV::camera(3818): ERROR reading YUV buffer: width=480, height=800, size=576000, receivedSize=460800
03-04 00:27:00.500: E/OpenCV::camera(3818): CvCapture_Android::grabFrame: NO new frame
The C++ code to use the video capture is roughly like this:
cv::VideoCapture capture;
capture.open(CV_CAP_ANDROID + 0);
capture.set(CV_CAP_PROP_FRAME_WIDTH, screenWidth);
capture.set(CV_CAP_PROP_FRAME_HEIGHT, screenHeight);
while (capture.isOpened()) {
capture.read(frame);
if (!frame.empty()) {
// do something with frame
}
}
Does someone have an idea what the problem could be? Or is there a proper guide how to integrate OpenCV with Native Activity?