Ask Your Question

Opencv.js - grabcut and GC_INIT_WITH_MASK

asked 2020-08-26 05:53:13 -0600

knighto05 gravatar image

updated 2020-08-26 21:07:18 -0600

Hello everyone,

I tried to replicate the Interactive Foreground extraction of python here into Javascript.

The first part with GC_INIT_WITH_RECT (which is already in the JS doc) works fine. But then I try to refine the selection with GC_INIT_WITH_MASK.

This is my code so far:

let src = cv.imread("canvasInput");
  cv.cvtColor(src, src, cv.COLOR_RGBA2RGB, 0);
  let newmask = cv.imread("canvasAdjust");
  let mask = new cv.Mat.ones(newmask.rows, newmask.cols, newmask.type());
  src.setTo(new cv.Scalar(0, 0, 255), mask);
  let bgdModel = new cv.Mat();
  let fgdModel = new cv.Mat();
  let rect = new cv.Rect();

// Setting the mask
for (let i = 0; i < newmask.rows; i++) {
    for (let j = 0; j < newmask.cols; j++) {
      if (newmask.ucharPtr(i, j)[1] == 128) {
        // if green, tell the mask that it's foreground
            mask.ucharPtr(i, j)[0] = cv.GC_FGD;
            mask.ucharPtr(i, j)[1] = cv.GC_FGD;
            mask.ucharPtr(i, j)[2] = cv.GC_FGD;
      if (newmask.ucharPtr(i, j)[0] == 255) {
            // if red, tell the mask that it's background
             mask.ucharPtr(i, j)[0] = cv.GC_BGD;
             mask.ucharPtr(i, j)[1] = cv.GC_BGD;
             mask.ucharPtr(i, j)[2] = cv.GC_BGD;

  cv.grabCut(src, mask, rect, bgdModel, fgdModel, 1, cv.GC_INIT_WITH_MASK);

But all it does is so far is some errors with random numbers like Uncaught 6566864 . Pointing out in the line cv.grabCut.

Can you help me to figure out if my approach of setting the mask is wrong and where exactly?

Thank you in advance.

edit retag flag offensive close merge delete

1 answer

Sort by » oldest newest most voted

answered 2020-08-26 09:08:19 -0600

berak gravatar image

first, your mask should have a single CV_8U channel, not 3

then, i think, this is a typo:

if (mask.ucharPtr(i, j)[0] == 255) {

did you mean to check newmask instead ?

and last, i have no idea what you get from cv.imread("canvasAdjust"); , but if it's a real world image, chances are low, that your "exact" conditions will fire

edit flag offensive delete link more



Thank you for replying: first: mmh when you says single CV_8U channel, you mean this line: let mask = new cv.Mat.ones(newmask.rows, newmask.cols, newmask.type());, right? I got it from the official documentation... what should I write there instead then?

second: yeah, it's a typo sorry, but don't worry, it's not in my official code

last: the canvasAdjustis a layer with markers where I define GC_BGD and GC_FGD. It's pretty straightforward with a simulation: Here is a Jsfiddle where I did a test:

knighto05 gravatar imageknighto05 ( 2020-08-26 14:12:34 -0600 )edit

euh I corrected it later, you can see the corrected version in my fiddle, but I will update the post also to make it fair to others.

I tried to do only newmask.rows for you first remark but the issue grew worse, BindingError {name: "BindingError", message: "Cannot pass "581" as a Mat", stack: "BindingError: Cannot pass "581" as a Mat↵ at Bi…seup (http://localhost/OpenCV/poc/index.js:283:7)"} . Maybe I misunderstood. And I tried to just do new cv.Mat() also but nothing changed from the original.

I am confused :/

knighto05 gravatar imageknighto05 ( 2020-08-26 21:06:45 -0600 )edit

look at the tutorial again, the mask MUST be single channel 8bit (and make it Mat.zeros() not Mat.ones(), else you interfere with the GC_XXX flags

berak gravatar imageberak ( 2020-08-27 04:27:42 -0600 )edit

Ah, now I understand, what you're talking about. I'll attempt some stuff and I'll let you know. Thank you already :)

knighto05 gravatar imageknighto05 ( 2020-08-27 06:29:31 -0600 )edit

and maybe start small, by drawing a filled rect with GC_FGD into a

mask = cv.Mat.zeros(height,width,CV_8U);

once you got that running, you can improve your mask generation

berak gravatar imageberak ( 2020-08-27 07:11:53 -0600 )edit

Thank you, for the precious info ! we got it running :D

knighto05 gravatar imageknighto05 ( 2020-08-27 11:58:48 -0600 )edit

Question Tools

1 follower


Asked: 2020-08-26 05:51:21 -0600

Seen: 1,150 times

Last updated: Aug 26 '20