How to pass image data from javascript to opencv c++ webassembly

asked 2019-12-08 12:47:25 -0600

chhaunkar96
1 ●1 ●1

updated 2019-12-09 10:59:57 -0600

I'm trying to pass image bufferArray to c++ function but OpenCV throw an unknown exception(test_wasm.js:2028 Uncaught).

I am beginner.

I'm trying to do the same with the help of this question.

Javascript code:

var openFile = function (e) {
    const fileReader = new FileReader();
    fileReader.onload = (event) => {
        const uint8Arr = new Uint8Array(event.target.result);
        passToWasm(uint8Arr);
    };
    fileReader.readAsArrayBuffer(e.target.files[0]);
};

function passToWasm(uint8ArrData) {
    // copying the uint8ArrData to the heap
    const numBytes = uint8ArrData.length * uint8ArrData.BYTES_PER_ELEMENT;
    const dataPtr = Module._malloc(numBytes);
    const dataOnHeap = new Uint8Array(Module.HEAPU8.buffer, dataPtr, numBytes);
    dataOnHeap.set(uint8ArrData);
    // calling the Wasm function
    console.log(dataOnHeap.byteOffset);
    const res = Module._image_input(dataOnHeap.byteOffset, uint8ArrData.length);

    Module._free(dataPtr);
}

Html code

  <input type='file' id='img' accept='image/*' onchange='openFile(event)'>

C++ code

   int image_input(uint8_t *buffer, size_t size){
                cv::Mat raw_data = cv::Mat(1, size, CV_8UC1, buffer);

                cv::cvtColor(raw_data, raw_data, cv::COLOR_RGB2GRAY);
                return 1;
     }

I also tried different colorspace. like: cv::COLOR_RGBA2GRAY, cv::COLOR_BGR2GRAY but all are not working.

Please help.

edit retag flag offensive close merge delete

Comments

i've no clue about wasm/js but already your c++ code won't work as intended.

cvtColor() will reallocate raw_data , so the result is no more in buffer

please try to explain, what your code is supposed to do, what do you try to read frm disk, exactly ? how do you plan to manage memory ? (aka: who owns what ?)

berak ( 2019-12-09 01:24:55 -0600 )edit

Actually I want to pass an image from the browser to web assembly(compiled OpenCV code to web assembly) and change the colorspace and return the image in javascript.

chhaunkar96 ( 2019-12-09 11:05:22 -0600 )edit

if your FileReader is just reading an image file straight from disk, you will need an imdecode() step (it's NOT an image, so far) inbetween.

berak ( 2019-12-09 11:17:36 -0600 )edit

you probably also cannot reuse the input buffer, because sizes will be different

berak ( 2019-12-09 11:20:23 -0600 )edit

add a comment

answered 2019-12-09 12:16:06 -0600

gpcbitnik
21 ●1 ●4

if you intent to fill buffer with an image file (jpeg, png etc..) you need to use imdecode (probably the best thing to do because in this case you do not have to manage the colorspace of the input image imdecode did it for you if you use it wth IMREAD_COLOR (BGR colospace) or IMREAD_GRAYSCALE flag). then use cvtColor with the image from imdecode.

if you intent to fill buffer with raw pixel data you need more informations to use cvtColor, you need a 2d Mat so you need cols size and rows size. then you need to know the number of channels to use the good opencv datatype Flag (CV_8UC1-2-3-4) finaly you need to know in wich colospace is the input image (RGB? BGR? RGBa?) and manage all possible case.( and there is a lot of case.......)

to construct the image juste use mat constructor as follow Mat Image = Mat(ImageRows, ImageCols, CV_8UC4, buffer);// here image have 4 channels so you have to use CV_8UC4

then to return a result you have 2 options : java own the memory so you have to compute and allocate final image size inside java and give output buffer to c++. c++ own the memory so you have to allocate it in c++ but be very carrefull to memory clear and memory leak.

Comments

Thanks for helping me. Now I'm trying with the following code.

 int image_input(uint8_t *buffer, size_t size) //query image input
    {
        cv::Mat raw_data = cv::Mat(1, size, CV_8UC1, buffer);
        img_object = cv::imdecode(raw_data, cv::IMREAD_GRAYSCALE);
        cout << img_object << endl;
        if (raw_data.empty())
            return 0;
        else
            return 1;
    }

But imdecode returning blank array [].

chhaunkar96 ( 2019-12-09 12:51:52 -0600 )edit

I'm trying to do it with the help of following example:

https://stackoverflow.com/questions/5...

chhaunkar96 ( 2019-12-09 12:58:20 -0600 )edit

in the imdecode documentation : "If the buffer is too short or contains invalid data, the function returns an empty matrix ( Mat::data==NULL )." so thirst make a c++ project in wich you will read an image file as binary, and feed your fonction with the stream. display the image before your fonction (using imread and imshow) and display the image after imdecode.once you have the display, return to your wasm project and try it. if it still not working, store the buffer in a file (from your js) and try to read it only in c++ if it works, you shoul take a look inside your buffer in the c++ code, ( it was probably clean by js ) if it still not work there is something wrong with your file or with your reading function

gpcbitnik ( 2019-12-10 07:28:23 -0600 )edit

can you explain it with help code?

chhaunkar96 ( 2019-12-10 13:22:38 -0600 )edit

add a comment

How to pass image data from javascript to opencv c++ webassembly

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

How to pass image data from javascript to opencv c++ webassembly edit

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

How to pass image data from javascript to opencv c++ webassembly