franz's profile - activity

Thanks for the reply as well, this is certainly something interesting to try out and see which one's faster. Thanks!

Ahhhh, I never thought of doing a push_back with zeros. I'll test it out and see if it has any significant speed improvements. Thanks!

Hello thanks for the reply, though I'm confused how rotation would help? I already push_back the 2nd Mat to the original one. What I'm looking for is a way to make it faster (and make my code cleaner), and I have a hunch that it might become so if I can simply increase the size of the original Mat, then insert the new data directly to it without needing a 2nd, temporary Mat. That way, the new data is copied only once, compared to my current method where it's first copied into a temporary Mat, then again copied (via push_back) into the original one.

So let's say we have the following code:

// Let's assume imageData represents an image I get as a parameter
unsigned char* imageData;

// Let's also assume that the image represented by imageData is of the exact same size as the parameters we pass to create the mat called x. This means the mat is perfectly filled up.
Mat x(height, width, imageData);

What I want to achieve is to "increase" the numbers of rows in the mat "x", without touching the original data. I tried the resize function but I noticed it stretches the data. So the result is still a mat that is completely filled up. What I want after increasing the number of rows is that the original "height" number of rows still has the imageData untouched, and the newly added rows is empty. I'm experimenting with this due to efficiency. What I have right now is I simply use push_back to append new data to the original Mat to resize it. So I have my original Mat x, then I append a 2nd Mat y (which has the same number of columns as x) to it.

The data I insert into my 2nd Mat y is individually done per cell using the at function, so I was thinking if it was more efficient if I can somehow just increase the number of rows in the original Mat x and use the "at" function on x directly instead of using a temporary Mat y in order to use push_back.

I have also tried this:

// imageData represents an image that has a lower height than the Mat it's being passed to. I assume the bottom set of rows would just be empty, but when I tried to access it near the bottom part I'm getting segmentation fault errors for some reason
Mat x(height * 1.5, width, imageData);

Alright, thank you for your insights, they're very helpful! Anyway, yes I think you're right, that should be 3 and not 4.

Also, if I do indeed get byte arrays for the 3 planes, how would I create a Mat from that? From the examples I saw, using the Mat's put() method accepts a single byte array. Now that I have 3 byte arrays, how do I combine them into a single one in order to pass them to put() ? Or should I create Mats using a different method or constructor? I browsed the docs for the Mat object, but I don't see any method or constructor which accepts 3 byte arrays simultaneously?

Thanks for the reply! That part of the code I actually based it from the official Camera2 example found here. It does initiate it with the 0-plane, but I just assume buffer.getBytes() does the rest of the job as when I run this example, the results are fully colored. Though the difference is the format was set to JPEG, not YUV. So are you saying that since I changed the format from JPEG to YUV, I must retrieve not just the 0-plane?

So I'm getting Image objects from Android's Camera2 API, then I convert them to OpenCV Mat objects via their byte buffers. The YUV_420_888 format is what I set as the output of the camera as recommended by the docs, but when I try converting the Mat from YUV to RGB, all it shows is green.

Following the answers from this thread, this is how I convert the Mat:

Image image = reader.acquireLatestImage();

ByteBuffer buffer = image.getPlanes()[0].getBuffer();
byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);

Mat mat = new Mat(image.getHeight()+image.getHeight()/2, image.getWidth(), CvType.CV_8UC1);
mat.put(0, 0, bytes);
Mat rgb = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC4);
Imgproc.cvtColor(mat, rgb, Imgproc.COLOR_YUV420sp2BGR, 4);

After these lines, all I did next was use imwrite to write the mats to disk. For reference, here's some sample images resulting from the writes:

YUV - http://i.imgur.com/qm765AZ.jpg (straight from the Camera2 API, no processing yet)

RGB - http://i.imgur.com/FzLx2Cc.jpg (the exact same image, but converted from YUV to RGB)

Any insights as to why the RGB image looks the way it does? I've also tried a whole lot of other conversion options besides COLOR_YUV420sp2BGR, but they all seem to have the same effect, which is a green image. Thank you in advance!

EDIT: As has been pointed out in the comments, it seems I need to use all 3 planes of the YUV image, and not just the first one. I know how to convert each plane into a byte array, and now I have 3 byte arrays each representing a plane, but my question is now how do I create a Mat from these 3 byte arrays? The put() method I'm familiar with only accepts a single byte array. Do I concatenate or combine them somehow?

So I'm trying to convert an Image object which I get from the Android Camera2 API into a Mat to do some processing. If I capture the image using the JPEG format, I can easily use the image's getPlanes() method to get a byte array, after which the byte array is easily converted into a Mat using the put() method. I even tried writing the Mat to disk using imwrite, and it stores a perfectly normal jpeg file.

The problem arises when I set the format to YUV_420_888 instead of JPEG. With the only change in the code being I swapped the JPEG format with YUV, the next set of image objects I received are all null for some reason.

Here's how I create Mat objects:

Image image = reader.acquireLatestImage()

ByteBuffer buffer = image.getPlanes()[0].getBuffer();
byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);

Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC1);
mat.put(0, 0, bytes);
image.close()

For reference, the code I'm working with is based from this Camera2 sample. Line 452 in that file shows where I swapped the JPEG format into YUV.