Ask Your Question
0

openCV Area Selection on live camera feed.

asked 2015-02-20 03:03:55 -0600

HSB gravatar image

Hello Guys, i am trying to do a area selection to create a mask on the camera view provided by CameraBridgeViewBase, is it possible? If yes how? I tried to look in this forum and over the stackoverflow site but there is no question of this type. If anybody can please help. Thank you.

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2015-02-20 06:49:04 -0600

berak gravatar image

let's say, you've got a mask image of the same type and size of your image, a white square in a black image, applying this works basically like this:

    mRgba = inputFrame.rgba();
    if (mask != null) {
        Core.bitwise_and(mRgba, mask, mRgba);
    }

if you had a Rect called sel, making the mask is not difficult either:

mask = Mat.zeros(mRgba.size(), mRgba.type()); // all black:
mask.submat(sel).setTo(Scalar.all(255));   // white square

now you need to make your Activity implement onTouchListener, and implement onTouch, 1st press should be tl corner of the rect, 2nd press br corner. all in all, we've got this:

package com.berak.cam3;

import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.Rect;
import org.opencv.core.Scalar;

import android.os.Bundle;
import android.util.Log;
import android.view.MotionEvent;
import android.view.View;
import android.view.View.OnTouchListener;
import android.app.Activity;

public class CamMaskActivity extends Activity implements CvCameraViewListener2, OnTouchListener {
    CameraBridgeViewBase mOpenCvCameraView;
    Rect sel = new Rect();
    Mat mRgba, mask;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_cam3);
        mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.cam3_surface_view);
        mOpenCvCameraView.setCvCameraViewListener(this);
        mOpenCvCameraView.setOnTouchListener(this);
    }

    private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
        @Override
        public void onManagerConnected(int status) {
            switch (status) {
            case LoaderCallbackInterface.SUCCESS:
                mOpenCvCameraView.enableView();
                break;
            default:
                super.onManagerConnected(status);
                break;
            }
        }
    };

    @Override
    public void onResume() {;
        super.onResume();
        OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_5,this, mLoaderCallback);
    }
    @Override
    public void onPause() {
        super.onPause();
        if (mOpenCvCameraView != null)
            mOpenCvCameraView.disableView();
    }
    @Override
    public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
        mRgba = inputFrame.rgba();
        if (mask != null) {
            Core.bitwise_and(mRgba, mask, mRgba);
        }
        return mRgba;
    }

    @Override
    public void onDestroy() {
        super.onDestroy();
    }

    @Override
    public void onCameraViewStarted(int width, int height) {
    }

    @Override
    public void onCameraViewStopped() {
        // TODO Auto-generated method stub

    }

    @Override
    public boolean onTouch(View v, MotionEvent event) {
        int cols = mRgba.cols();
        int rows = mRgba.rows();

        int xOffset = (mOpenCvCameraView.getWidth() - cols) / 2;
        int yOffset = (mOpenCvCameraView.getHeight() - rows) / 2;

        int x = (int)event.getX() - xOffset;
        int y = (int)event.getY() - yOffset;
        if (x<0||y<0||x>=cols||y>=cols) return false;
        if ((sel.x==0 && sel.y==0) || (sel.width!=0 && sel.height!=0))
        {
            mask = null;
            sel.x=x; sel.y=y;
            sel.width = sel.height = 0;
        } else {
            sel.width = x - sel.x;
            sel.height = y - sel.y;
            if ( sel.width <= 0 || sel.height <= 0 ) { // invalid, clear it all
                sel.x=sel.y=sel.width=sel.height = 0;
                mask = null;
                return false;
            }
            mask = Mat.zeros(mRgba.size(), mRgba.type());
            mask.submat(sel).setTo(Scalar.all(255));
        }
        Log.w("touch",sel.toString());
        return false;
    }
}
edit flag offensive delete link more

Comments

Very helpfull! Really one thanks isn't enough. :) I would like to ask some explanation about the onTouchMethod() to have a better understanding of what's going on: 1) At the line --> if (x<0||y<0||x>=cols||y>=cols) return false; Is it right y>=cols or it should be y>=rows?

2) xOffset and yOffset are used to match the view coordinates to the Mat?

3) At the end of the method why we always have to return false?

Algorithm improvement: Can you help me please, if possible, to capture some more complex motion event to create a complex mask? My aim is to built a feature as done in google translator app where we take an image to translate something, after the image is capture(in our case it will be a Mat frame) to highlight the zone and then process that zone(apply the mask ...(more)

HSB gravatar imageHSB ( 2015-02-23 08:43:43 -0600 )edit

1) it should be y>=rows? ? oh, yes, ofc. err on my side.

2) the img is smaller than the whole screen, so normalize to img coords..

3) no idea, look it up in the android docs, iirc, it's like 'message handled'

Algorithm imrovement) - no idea, what you mean there, atm.

berak gravatar imageberak ( 2015-02-23 08:54:56 -0600 )edit

Ok got it! 2) Where did you find normalizing formula or how to calculate it? 3) what do you mean by iirc? Algorithm imrovement) Am i not clear? What do you mean by "what you mean there, atm."?

HSB gravatar imageHSB ( 2015-02-23 09:11:49 -0600 )edit

can't remember, where i found it, guess it's from one of the samples even.

i just don't know the translator app, you're talking about.

berak gravatar imageberak ( 2015-02-23 09:21:14 -0600 )edit

I have uploaded a video to show you what i have in the mind, here is the link: https://www.dropbox.com/s/plbnghk0cte... Is this possibile in opencv where i am using CameraBridgeViewBase(NativeCamera)?

HSB gravatar imageHSB ( 2015-02-23 15:08:17 -0600 )edit

Question Tools

2 followers

Stats

Asked: 2015-02-20 03:03:55 -0600

Seen: 2,521 times

Last updated: Feb 20 '15