Revision history [back]

Failing to use IE Backend (Compiling CV with IE)

I've been trying to compile OpenCV with Inference Engine to Raspbian Buster. I followed documentation on github but I might be missing something or it is incomplete. Doc guide

What I want:

Inference Engine with NCS2 support
FFmpeg 4 support (because gstreamer it's not working very well with video IO)

What I did:

Built a docker with a slightly different code than documentation provides, to build with FFmpeg

FROM debian:buster

USER root

RUN dpkg --add-architecture armhf && \
    apt-get update && \
    apt-get install -y --no-install-recommends \
    crossbuild-essential-armhf \
    cmake \
    pkg-config \
    wget \
    xz-utils \
    libgtk2.0-dev:armhf \
    libpython-dev:armhf \
    libpython3-dev:armhf \
    python-numpy \
    python3-numpy \
    libgstreamer1.0-dev:armhf \
    libgstreamer-plugins-base1.0-dev:armhf \
ffmpeg:armhf \
libavcodec-dev:armhf \
libavformat-dev:armhf \
libavutil-dev:armhf \
libswscale-dev:armhf \
libavresample-dev:armhf

# Install Inference Engine
RUN wget --no-check-certificate https://download.01.org/opencv/2020/openvinotoolkit/2020.1/l_openvino_toolkit_runtime_raspbian_p_2020.1.023.tgz && \
    tar -xf l_openvino_toolkit_runtime_raspbian_p_2020.1.023.tgz

Built and compiled OpenCV with basically copy pasted build parameters from the guide. Only changed the path for the newer openvino toolkit and added -latomic flag. I used OpenCV 4.2.0 because that's the version OpenVINO includes.

General configuration for OpenCV 4.2.0 =====================================
  Version control:               unknown

  Platform:
    Timestamp:                   2020-04-27T16:03:10Z
    Host:                        Linux 4.19.97-v7l+ armv7l
    Target:                      Linux 1 arm
    CMake:                       3.13.4
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               Release

  CPU/HW features:
    Baseline:                    NEON
      required:                  NEON
      disabled:                  VFPV3

  C/C++:
    Built as dynamic libs?:      YES
    C++ Compiler:                /usr/bin/arm-linux-gnueabihf-g++  (ver 8.3.0)
    C Compiler:                  /usr/bin/arm-linux-gnueabihf-gcc
  (took flags out)

    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          dl m pthread rt
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python2 python3 stitching video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java js ts
    Applications:                apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:
    GTK+:                        YES (ver 2.24.32)
      GThread :                  YES (ver 2.58.3)
      GtkGlExt:                  NO

  Media I/O:
    ZLib:                        build (ver 1.2.11)
    JPEG:                        libjpeg-turbo (ver 2.0.2-62)
    WEBP:                        build (ver encoder: 0x020e)
    PNG:                         build (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.0.10)
    JPEG 2000:                   build (ver 1.900.1)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES
      avcodec:                   YES (58.35.100)
      avformat:                  YES (58.20.100)
      avutil:                    YES (56.22.100)
      swscale:                   YES (5.3.100)
      avresample:                YES (4.0.0)
    GStreamer:                   YES (1.14.4)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      NO
    Inference Engine:            YES (2019030000 / Unknown)
        * libs:                  /l_openvino_toolkit_runtime_raspbian_p_2020.1.023/inference_engine/lib/armv7l/libinference_engine.so
        * includes:              /l_openvino_toolkit_runtime_raspbian_p_2020.1.023/inference_engine/include
    nGraph:                      NO
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.5.1)

  OpenCL:                        YES (no extra features)
    Include path:                /opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 2:
    Interpreter:                 /usr/bin/python2.7 (ver 2.7.16)
    Libraries:
    numpy:                       /usr/local/lib/python2.7/dist-packages/numpy/core/include (ver undefined - cannot be probed because of the cross-compilation)
    install path:                lib/python2.7/dist-packages/cv2/python-2.7

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.7.3)
    Libraries:
    numpy:                       /usr/local/lib/python3.7/dist-packages/numpy/core/include (ver undefined - cannot be probed because of the cross-compilation)
    install path:                lib/python3.7/dist-packages/cv2/python-3.7

  Python (for build):            /usr/bin/python2.7

  Install to:                    /opencv/opencv_install

I used setup_vars_opencv4.sh in generated files to set the environmental variables to use my custom built OpenCV
I applied NCS2 USB rules too

Result - OpenCV fails to import:

ImportError: libinference_engine.so: cannot open shared object file: No such file or directory

I tried to setup openvino vars and then setup opencv and I managed to import cv2 successfully but it fails to run anything with Inference Engine:

[ INFO:0] global /opencv/modules/dnn/src/dnn.cpp (131) checkIETarget checkIETarget(0) has failed with message: OpenCV(4.2.0) /opencv/modules/dnn/src/op_inf_engine.cpp:774: error: (-2:Unspecified error) Failed to initialize Inference Engine backend (device = CPU): Device with "CPU" name is not registered in the InferenceEngine in function 'initPlugin'

Traceback (most recent call last):
  File "run_counter.py", line 28, in <module>
    max_disappeared, video_fps=video_fps, device=device)
  File "/home/pi/AI-Counter/people_counter_min.py", line 104, in people_counter
    detections = net.forward()
cv2.error: OpenCV(4.2.0) /opencv/modules/dnn/src/op_inf_engine.cpp:774: error: (-2:Unspecified error) Failed to initialize Inference Engine backend (device = MYRIAD): Device with "CPU" name is not registered in the InferenceEngine in function 'initPlugin'

And yes, my code has net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE) net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)

Someone has an idea of what's going on?