Revision history [back]

I am making an Android app to simulate colourblindness. It takes a camera feed and manipulates at runtime.

• My question:

I have an OpenCV Mat object rgba, containing all Pixels of a given frame with red, green, blue and alpha (rgba) values. To speed up processing I'd like to use Renderscript, but when I convert that Mat rgba to Bitmap I lose about 5 fps, making any speedgain by Renderscript pointless. So I would like to know a way of placing the contents of Mat rgba into an Allocation that I can give to Renderscript, where I then just do Matrix multiplication for it all and return it.

• What I tried:

  • Converting Mat to Bitmap and then give to Renderscript, which works but is way too expensive (minus ~5 fps).
  • Allocation aName2 = Allocation.createFromString(mRS, rgba.dump(), 3); // this just shows i am desperate ;(

• In case you're interested what I am doing at the moment:

I based my app on the imagemanipulations example provided by OpenCV, specifically the Sepia filter option.

In onCameraViewStarted{} I set a Kernel such as for example the Sepia Kernel:

        // Fill sepia kernel
        mSepiaKernel = new Mat(4, 4, CvType.CV_32F);
        mSepiaKernel.put(0, 0, /* R */0.189f, 0.769f, 0.393f, 0f);
        mSepiaKernel.put(1, 0, /* G */0.168f, 0.686f, 0.349f, 0f);
        mSepiaKernel.put(2, 0, /* B */0.131f, 0.534f, 0.272f, 0f);
        mSepiaKernel.put(3, 0, /* A */0.000f, 0.000f, 0.000f, 1f);

Then in onCameraFrame OpenCV places the frame into a Mat and the filter is run over it, causing it to have this yellowish tint.

public Mat onCameraFrame(CvCameraViewFrame inputFrame) {  
Mat rgba = inputFrame.rgba(); 
Core.transform(rgba, rgba, mSepiaKernel); 
return rgba;  
}

I have this running for Protan, Deutan and Tritan and on the S7 edge I get 10 fps. This is okayish, but as soon as we put the app on an older phone it becomes incredibly slow. Hence I would like to do what Core.transform() does in Renderscript instead.