Ask Your Question
1

Opencv Android: FPS and RGB values [Solved]

asked 2017-09-17 12:09:15 -0500

miller619 gravatar image

updated 2017-09-21 15:37:37 -0500

hi, i am learning to use opencv in android application. I am trying to create an app that captures video. the purpose of it is that when i point my camera to a color, it should show me the rgb values separately and also show the fps at which it is capturing. i would also like to change the fps according to the device its used on. I have done tutorials and I know how to change RGB image to GREY scale and so on. However, i cannot understand that once i declare "inputFrame.rgba()" how the break it up and store the R, G and B values separately to show it on the log/screen and also display the fps at which the video is being captured.

  public class VideoRecordingActivity extends AppCompatActivity implements CameraBridgeViewBase.CvCameraViewListener2{

//custom toolbar
private Toolbar toolbar;

//java camera view
JavaCameraView javaCameraView;

Mat mRgba, hsv_scale;





//callback loader
BaseLoaderCallback mCallBackLoader = new BaseLoaderCallback(this) {
    @Override
    public void onManagerConnected(int status) {


        switch (status){
            case BaseLoaderCallback.SUCCESS:
                javaCameraView.enableView();
                break;
            default:
                super.onManagerConnected(status);
                break;
        }
    }
};


public static String TAG="VideoRecordingActivity";
public static String TAG2="Video";


@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_video_recording);

    //custom toolbar
    toolbar = (Toolbar)findViewById(R.id.nav_action);
    setSupportActionBar(toolbar);


    //connect the camera
    javaCameraView = (JavaCameraView)findViewById(R.id.java_camera_view);

    //set visibility
    javaCameraView.setVisibility(SurfaceView.VISIBLE);
    javaCameraView.setMaxFrameSize(1024,768);
    javaCameraView.clearFocus();

    //set callback function
    javaCameraView.setCvCameraViewListener(this);


}

@Override
protected void onPause() {
    super.onPause();

    if(javaCameraView!=null){
        javaCameraView.disableView();
    }
}

@Override
protected void onDestroy() {
    super.onDestroy();
    if (javaCameraView!=null){
        javaCameraView.disableView();
    }
}

@Override
protected void onResume() {
    super.onResume();

    if (OpenCVLoader.initDebug()){
        Log.d(TAG, "Connected");

        //display when the activity resumed,, callback loader
        mCallBackLoader.onManagerConnected(LoaderCallbackInterface.SUCCESS);


    }else{
        Log.d(TAG, "Not connected");
        OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_3_0, this, mCallBackLoader);
    }
}

@Override
public void onCameraViewStarted(int width, int height) {
    //4 channel
    mRgba = new Mat(height, width, CvType.CV_8UC4);
    hsv_scale = new Mat(height, width, CvType.CV_8UC1);


}

@Override
public void onCameraViewStopped() {
    //release
    mRgba.release();
}

@Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
    //camera setup

    //get each frame from camera
    mRgba = inputFrame.rgba();

    Vector<Mat> channels = null;

    Core.split(mRgba, channels);

    int rows = mRgba.rows();
    int cols = mRgba.cols();

    double R=0,G=0,B=0;

    for (int i=0; i<rows; i++)
    {
        for (int j=0; j<cols; j++)
        {
            double[] data = mRgba.get(i, j); //Stores element in an array
            R = data[0];
            G= data[1];
            B = data[2];
        }
    }
    Imgproc.putText(mRgba,"R:"+R + "G:"+G +"B:"+B,new Point(10,52),Core.FONT_HERSHEY_COMPLEX,.7,new Scalar(5,255,255), 2,8,false );
    return mRgba;

}

This is my camera class. In the onCameraFrame class you may notice that i have declared rows and columns to break this color space into RGB and this works fine. Except it drastically decrees the FPS of my video.

What am i missing. I would like to have suggestions/direction on how to achieve what im trying to get.

Thank you

edit retag flag offensive close merge delete

Comments

I have been going through the JavaCameraView.java class and found that i can call a method called "enableFpsMeter()" to display the fps on my screen.

miller619 gravatar imagemiller619 ( 2017-09-17 12:29:12 -0500 )edit

simple lesson here: NEVER do anything per pixel from java. if you're writing loops like that, you're doing it wrong.

then, your loop does not make any sense, since you're just overwriting R,G,B, so, in the end, you only get the values for the very last pixel

berak gravatar imageberak ( 2017-09-18 01:31:43 -0500 )edit

please take a look at the docs -- split() tales a List, not a Vector

berak gravatar imageberak ( 2017-09-18 03:45:23 -0500 )edit

i tried splitting the channels like this but the problem is when i log the value or red blue or green i get

    List<Mat> channels = new ArrayList<Mat>();
    Core.split(mRgba, channels);
    red = channels.get(0);
    blue = channels.get(1);
    green = channels.get(2);

    R:Mat [ 240*320*CV_8UC1, isCont=true, isSubmat=false, nativeObj=0xffffffffc61ec770, 
dataAddr=0xffffffffb7942000 ]

I want to display the value of red 0~255 and same for blue or green. I get this only by using the for loop.

I got a better fps by setMaxFrameSize(320, 240); but that is thing is just a band aid and not a real solution. I am still struggling with the problem.

miller619 gravatar imagemiller619 ( 2017-09-18 04:00:53 -0500 )edit

again, DO NOT try to access the pixels. (also, your indexing look way off)

use Mat.dump(), to print it

berak gravatar imageberak ( 2017-09-18 05:11:37 -0500 )edit

thanks for telling me not to access the pixel. i got it the first time you said it.. and i posted this question in the first place because i had a feeling that this is not the way to do it. Having said that i also could not figure out how to solve my issue. and just telling me not to access the pixel does not help. i have also posted in comment what other ways i tried and what result im getting. And i have also mentioned that this im learning opencv (this is my first time). so if you can be a little bit more specific on what i should do that would be great.. Thank you..

miller619 gravatar imagemiller619 ( 2017-09-18 05:25:42 -0500 )edit

so, what IS your issue, now ? you CAN split the image into channels. you also could return one of them (instead of mRgba), if you want to look at it.

berak gravatar imageberak ( 2017-09-18 05:29:18 -0500 )edit

The problem is i just want to display the R,G and B numeric values when i point my camera to certain color. for example if i point to red then 255, 0, 0 or somewhat approximately close to that. the only time i got this is by accessing pixels but the fps dropped to 2 and as you said that its a very wrong approach i started looking for alternatives. I tried to split into channels but each channel still is a Mat object and i dont know how to get the r, g, or b value from it. that is still my issue. i want to get the r, g and b value and print it in log.

miller619 gravatar imagemiller619 ( 2017-09-18 05:45:04 -0500 )edit

apologies for getting impatient, too.

berak gravatar imageberak ( 2017-09-18 10:14:08 -0500 )edit

my apologies... I have been trying to get there for like 3 days now... there is nothing i did not try.. i found some clues and am able to print hex values (got clues form color detection blob example). It converts the rgb to hsv then prints out the hex value (quite accurate) on screen, however when i try to change it to rgb it gives wrong result completely...

miller619 gravatar imagemiller619 ( 2017-09-18 10:26:03 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
1

answered 2017-09-21 15:36:57 -0500

miller619 gravatar image

updated 2017-09-21 15:37:49 -0500

    @Override
public void onCameraViewStarted(int width, int height) {
    //4 channel
    mRgba = new Mat(width, height, CvType.CV_8UC4);
    mHsv = new Mat(width, height, CvType.CV_8UC3);
}

    @Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
    //get each frame from camera
    mRgba = inputFrame.rgba();


    /**********HSV conversion**************/
    //convert mat rgb to mat hsv
    Imgproc.cvtColor(mRgba, mHsv, Imgproc.COLOR_RGB2HSV);

    //find scalar sum of hsv
    Scalar mColorHsv = Core.sumElems(mHsv);

    int pointCount = 320*240;


    //convert each pixel
    for (int i = 0; i < mColorHsv.val.length; i++) {
        mColorHsv.val[i] /= pointCount;
    }

    //convert hsv scalar to rgb scalar
    Scalar mColorRgb = convertScalarHsv2Rgba(mColorHsv);

    /*Log.d("intensity", "Color: #" + String.format("%02X", (int)mColorHsv.val[0])
            + String.format("%02X", (int)mColorHsv.val[1])
            + String.format("%02X", (int)mColorHsv.val[2]) );*/
    //print scalar value
    Log.d("intensity", "R:"+ String.valueOf(mColorRgb.val[0])+" G:"+String.valueOf(mColorRgb.val[1])+" B:"+String.valueOf(mColorRgb.val[2]));


    /*Convert to YUV*/

    int R = (int) mColorRgb.val[0];
    int G = (int) mColorRgb.val[1];
    int B = (int) mColorRgb.val[2];

    int Y = (int) (R *  .299000 + G *  .587000 + B *  .114000);
    int U = (int) (R * -.168736 + G * -.331264 + B *  .500000 + 128);
    int V = (int) (R *  .500000 + G * -.418688 + B * -.081312 + 128);

    //int I = (R+G+B)/3;


    //Log.d("intensity", "I: "+String.valueOf(I));
    Log.d("intensity", "Y:"+ String.valueOf(Y)+" U:"+String.valueOf(U)+" V:"+String.valueOf(V));

    /*calibration*/



    return mRgba;

}



//convert Mat hsv to scalar
private Scalar convertScalarHsv2Rgba(Scalar hsvColor) {
    Mat pointMatRgba = new Mat();
    Mat pointMatHsv = new Mat(1, 1, CvType.CV_8UC3, hsvColor);
    Imgproc.cvtColor(pointMatHsv, pointMatRgba, Imgproc.COLOR_HSV2RGB);

    return new Scalar(pointMatRgba.get(0, 0));
}
edit flag offensive delete link more
0

answered 2017-09-18 10:11:13 -0500

berak gravatar image

updated 2017-09-18 10:13:13 -0500


// print a Mat:
System.out.println(mat.dump())

// or, maybe, only a small part of it:
System.out.println(mat.submat(5,15,5,15).dump())

// sometimes, the mean of a roi is useful:
System.out.println(Core.mean(mat.submat(5,15,5,15)))
edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-09-17 12:09:15 -0500

Seen: 1,199 times

Last updated: Sep 21 '17