Ask Your Question
0

OpenCV.js unable to track coloured circle and draw circle on target.

asked 2019-05-15 02:09:26 -0600

Adam108 gravatar image

updated 2019-05-16 02:35:21 -0600

berak gravatar image

Hi, I have a code to track a random input target and draw a rectangle on top of my target, but I need to track(Green colored circle and draw a circle on my target, I have searched for it in OpenCV.js but I couldn't find it, help me with any link which can help me, my rectangle code is below

let video = document.getElementById('videoInput');
let cap = new cv.VideoCapture(video);

// take first frame of the video
let frame = new cv.Mat(video.height, video.width, cv.CV_8UC4);
cap.read(frame);

// hardcode the initial location of window
let trackWindow = new cv.Rect(150, 60, 63, 125);

// set up the ROI for tracking
let roi = frame.roi(trackWindow);
let hsvRoi = new cv.Mat();
cv.cvtColor(roi, hsvRoi, cv.COLOR_RGBA2RGB);
cv.cvtColor(hsvRoi, hsvRoi, cv.COLOR_RGB2HSV);
let mask = new cv.Mat();
let lowScalar = new cv.Scalar(30, 30, 0);
let highScalar = new cv.Scalar(180, 180, 180);
let low = new cv.Mat(hsvRoi.rows, hsvRoi.cols, hsvRoi.type(), lowScalar);
let high = new cv.Mat(hsvRoi.rows, hsvRoi.cols, hsvRoi.type(), highScalar);
cv.inRange(hsvRoi, low, high, mask);
let roiHist = new cv.Mat();
let hsvRoiVec = new cv.MatVector();
hsvRoiVec.push_back(hsvRoi);
cv.calcHist(hsvRoiVec, [0], mask, roiHist, [180], [0, 180]);
cv.normalize(roiHist, roiHist, 0, 255, cv.NORM_MINMAX);

// delete useless mats.
roi.delete(); hsvRoi.delete(); mask.delete(); low.delete(); high.delete(); hsvRoiVec.delete();

// Setup the termination criteria, either 10 iteration or move by atleast 1 pt
let termCrit = new cv.TermCriteria(cv.TERM_CRITERIA_EPS | cv.TERM_CRITERIA_COUNT, 10, 1);

let hsv = new cv.Mat(video.height, video.width, cv.CV_8UC3);
let hsvVec = new cv.MatVector();
hsvVec.push_back(hsv);
let dst = new cv.Mat();
let trackBox = null;

const FPS = 30;
function processVideo() {
    try {
        if (!streaming) {
            // clean and stop.
            frame.delete(); dst.delete(); hsvVec.delete(); roiHist.delete(); hsv.delete();
            return;
        }
        let begin = Date.now();

        // start processing.
        cap.read(frame);
        cv.cvtColor(frame, hsv, cv.COLOR_RGBA2RGB);
        cv.cvtColor(hsv, hsv, cv.COLOR_RGB2HSV);
        cv.calcBackProject(hsvVec, [0], roiHist, dst, [0, 180], 1);

        // apply camshift to get the new location
        [trackBox, trackWindow] = cv.CamShift(dst, trackWindow, termCrit);

        // Draw it on image
        let pts = cv.rotatedRectPoints(trackBox);
        cv.line(frame, pts[0], pts[1], [255, 0, 0, 255], 3);
        cv.line(frame, pts[1], pts[2], [255, 0, 0, 255], 3);
        cv.line(frame, pts[2], pts[3], [255, 0, 0, 255], 3);
        cv.line(frame, pts[3], pts[0], [255, 0, 0, 255], 3);
        cv.imshow('canvasOutput', frame);

        // schedule the next one.
        let delay = 1000/FPS - (Date.now() - begin);
        setTimeout(processVideo, delay);
    } catch (err) {
        utils.printError(err);
    }
};

// schedule the first one.
setTimeout(processVideo, 0);
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
1

answered 2019-05-24 03:26:53 -0600

Lucy8 gravatar image

updated 2019-05-24 03:27:49 -0600

If you are able to detect targets and your problem is that you need to filter only green ones then you need to get the average color of each target and see if it's in your color range. To do that you need to you use cv.mean(src, mask). This is documentation for cv.mean.

Also, see this question, it's very close to yours.

Alternatively, you can try this approach

  1. Create a mask in color range that you want, threshold image
  2. Find contours
  3. If you need all the green things, then draw a circle on each contour, if you need objects in specific shapes like a circle than you can use object matching and draw circle only on matched contours

Here is some code on how to make a mask in specific color range, you may need to change the upper and lower color values, also here is used THRESH_BINARY but it may be better to use THRESH_TRUNC.

// segmenting by green color
const greenColorUpper = hue => new cv.Vec(hue, 0.8 * 255, 0.6 * 255); // change this values
const greenColorLower = hue => new cv.Vec(hue, 0.1 * 255, 0.05 * 255); // change this values

const makeColorMask = (img) => {
  // filter by color
  const imgHLS = img.cvtColor(cv.COLOR_BGR2HLS);
  const rangeMask = imgHLS.inRange(greenColorLower(80), greenColorUpper(140)); // change this values

  // remove noise
  const blurred = rangeMask.blur(new cv.Size(10, 10));
  const thresholded = blurred.threshold(
    200,
    255,
    cv.THRESH_BINARY
  );

  return thresholded;
};


// Find Contours
const contours = new cv.MatVector();
const hierarchy = new cv.Mat();
const thresholded = makeColorMask(img)
cv.findContours(thresholded, contours, hierarchy, cv.RETR_LIST, cv.CHAIN_APPROX_SIMPLE);

And here is the documentation for shape matching. If shape matching doesn't work for you, you can write your own circle validator. OpenCV gives you all the vertices of the contour, perimeter, and center of the contour.

Here are very useful posts about color and shape matching with python

https://www.pyimagesearch.com/2014/08...

https://www.pyimagesearch.com/2016/02...

edit flag offensive delete link more

Comments

1

Hi, thank you for your reply, many reference links are available only for python, I like to develop it in javascript, specially opencv.js don't have many functions, which python and java have many functions example (IplImage, Point2f, UByteIndexer, cv.minEnclosingCircle (this one explained in OpenCV web, but not available in opencv.js file), FloatPoint, double, OpenCVFrameConverter, cv.findContours).

Adam108 gravatar imageAdam108 ( 2019-05-24 06:30:53 -0600 )edit

Hi, I know that struggle because I also started OpenCV with javascript and used opencv.js. There are not so many tutorials with JS and that's the reason that I use python tutorials. The first step is to learn and understand what is done and the second step to convert it to Javascript.

Yes, there are some functions that do not exist in the JS version but you don't need every function. The main functions DO exist. I'm not sure that you need plImage, Point2f, or UByteIndexer. And cv.findContours DO exists in opencv.js. I've used opencv.js version 3.4.3 with wasm (opencv_js.wasm) and there was cv.findContours which worked fine.

Could you please specify which version of opencv.js are you using and what is the error message when you try to use cv.findContours?

Lucy8 gravatar imageLucy8 ( 2019-05-28 03:04:12 -0600 )edit

https://docs.opencv.org/master/opencv.js this is the link, I'm using this opencv.js file if I'm not using right one guide me with another one.

Adam108 gravatar imageAdam108 ( 2019-05-29 00:22:46 -0600 )edit

I downloaded that version of opencv.js and tried, it also has cv.findContours. Here is a small example that worked for me. Please try this it should work

    let imgElement = document.querySelector('#imageSrc');
    let src = cv.imread(imgElement);
    let dst = cv.Mat.zeros(src.cols, src.rows, cv.CV_8UC3);
    cv.cvtColor(src, src, cv.COLOR_RGBA2GRAY, 0);
    cv.threshold(src, src, 120, 200, cv.THRESH_BINARY);
    let contours = new cv.MatVector();
    let hierarchy = new cv.Mat();
    cv.findContours(src, contours, hierarchy, cv.RETR_CCOMP, cv.CHAIN_APPROX_SIMPLE);
    const contourColor = [255, 0, 0, 255];
    cv.drawContours(dst, contours, -1, contourColor, 1, cv.LINE_8, hierarchy, 100);
    cv.imshow('canvasOutput', dst);
Lucy8 gravatar imageLucy8 ( 2019-05-29 02:55:56 -0600 )edit

Hi, I have run the above example and i got the output as my original image.

Adam108 gravatar imageAdam108 ( 2019-05-29 03:43:18 -0600 )edit

That doesn't sound right. Seems something is incorrect. Please open developer tools in browser and check in console whether you have an error or not. Or put here both your HTML and JS files and I will try to help.

Lucy8 gravatar imageLucy8 ( 2019-05-29 06:27:51 -0600 )edit
0

answered 2019-05-29 05:01:09 -0600

Adam108 gravatar image

updated 2019-05-29 05:02:56 -0600

package com.emaraic.distance;

import org.bytedeco.javacpp.FloatPointer; import org.bytedeco.javacpp.Loader; import org.bytedeco.javacpp.indexer.UByteIndexer; import org.bytedeco.javacpp.opencv_core.CvContour; import org.bytedeco.javacpp.opencv_core.CvMemStorage; import org.bytedeco.javacpp.opencv_core.CvPoint; import org.bytedeco.javacpp.opencv_core.CvSeq; import org.bytedeco.javacpp.opencv_core.IplImage; import org.bytedeco.javacpp.opencv_core.Mat; import org.bytedeco.javacpp.opencv_core.Point; import org.bytedeco.javacpp.opencv_core.Point2f; import org.bytedeco.javacpp.opencv_core.Scalar; import static org.bytedeco.javacpp.helper.opencv_core.CV_RGB; import static org.bytedeco.javacpp.opencv_core.IPL_DEPTH_8U; import org.bytedeco.javacpp.opencv_core.Size; import static org.bytedeco.javacpp.opencv_core.cvCreateImage; import static org.bytedeco.javacpp.opencv_core.cvCreateMemStorage; import static org.bytedeco.javacpp.opencv_core.cvGetSize; import static org.bytedeco.javacpp.opencv_core.cvPoint; import org.bytedeco.javacpp.opencv_imgproc; import static org.bytedeco.javacpp.opencv_imgproc.COLOR_BGR2HSV; import static org.bytedeco.javacpp.opencv_imgproc.CV_BGR2GRAY; import static org.bytedeco.javacpp.opencv_imgproc.CV_CHAIN_APPROX_SIMPLE; import static org.bytedeco.javacpp.opencv_imgproc.CV_RETR_LIST; import static org.bytedeco.javacpp.opencv_imgproc.CV_THRESH_OTSU; import static org.bytedeco.javacpp.opencv_imgproc.contourArea; import static org.bytedeco.javacpp.opencv_imgproc.cvCircle; import static org.bytedeco.javacpp.opencv_imgproc.cvCvtColor; import static org.bytedeco.javacpp.opencv_imgproc.cvFindContours; import static org.bytedeco.javacpp.opencv_imgproc.cvThreshold; import static org.bytedeco.javacpp.opencv_imgproc.cvtColor; import static org.bytedeco.javacpp.opencv_imgproc.putText; import static org.bytedeco.javacpp.opencv_imgproc.resize; import org.bytedeco.javacv.CanvasFrame; import org.bytedeco.javacv.Frame; import org.bytedeco.javacv.FrameGrabber; import org.bytedeco.javacv.OpenCVFrameConverter; import org.bytedeco.javacv.OpenCVFrameGrabber; public class DistanceMeasurement {

private static final double FOCAL_LENGTH = 906.107;//focal length in pixels obtained from Calibration class
private static final double RADIUS_OF_MARKER = 1.0;
private static final OpenCVFrameConverter.ToIplImage converter = new OpenCVFrameConverter.ToIplImage();

public static boolean isPixelBlue(Mat image, int x, int y) {
    if (x < 0 || y < 0) {
        return false;
    }
    UByteIndexer srcIndexer = image.createIndexer();
    int[] hsv = new int[3];
    boolean blue = false;
    for (int i = 0; i < srcIndexer.rows(); i++) {
        for (int j = 0; j < srcIndexer.cols(); j++) {
            if (j == x && i == y) {
                srcIndexer.get(i, j, hsv);
                if (hsv[0] >= 100 && hsv[0] <= 130 && hsv[1] >= 50 && hsv[1] <= 255 && hsv[2] >= 50 && hsv[2] <= 255) {
                    blue = true;
                }
            }
        }
    }
    return blue;
}

public static void main(String[] args) {
    try {
        OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(0);
        grabber.start();
        CanvasFrame canvas = new CanvasFrame("Distance Measurement - Emaraic");

        canvas.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
        canvas.setCanvasSize(660, 660);
        canvas.setLocationRelativeTo(null);
        while (true) {
            Frame frame = grabber.grab();
            IplImage grabbedImage = converter.convert(frame);

            if (grabbedImage != null && canvas.isVisible()) {
                Mat img = converter.convertToMat(frame);

                /*Resize Image (comment this section if you want to use default resolution)*/
                Size size = new Size(640, 640);
                resize(img, img, size);
                grabbedImage = new IplImage(img);

                /*Invert to HSV color space*/
                Mat imghsv = new Mat();
                cvtColor(img, imghsv, COLOR_BGR2HSV);

                /*Convert image to grayscale*/
                IplImage gray = cvCreateImage(cvGetSize(grabbedImage), IPL_DEPTH_8U, 1);
                cvCvtColor(grabbedImage, gray, CV_BGR2GRAY);


                /*Binarising Image*/
                IplImage binimg = cvCreateImage(cvGetSize(gray), IPL_DEPTH_8U, 1);
                cvThreshold(gray, binimg, 0, 255, CV_THRESH_OTSU);

                /*Find countour */
                CvMemStorage ...
(more)
edit flag offensive delete link more

Comments

Yes but please specify the functions that you can't find. For example, js doesn't have Point2f but you don't need it. Here Point2f just creates a point which can be done in Javascript too. So equivalent of this

Point2f center = new Point2f();

in JS is

const center = { x: 0, y:0 }

But in this case, you don't even need to declare it, remember Javascript is not statically typed language so replace this 3 lines

Point2f center = new Point2f();
FloatPointer radius = new FloatPointer(1f);
opencv_imgproc.minEnclosingCircle(new Mat(ptr), center, radius);

with simply 1 line

const { center, radius } = cv.minEnclosingCircle(contour);
Lucy8 gravatar imageLucy8 ( 2019-05-29 06:44:22 -0600 )edit

wow, this is great thank you very much, I lacking in this work just because of very less document or guidance in opencv.js, I think you are very good at it, how to identify my target in particular color and shape I can find the focal length from a particular distance, but how to show in output about calculating distance from target in javascript using opencv.js, or is it even possible, but I couldn't find any help from anyone one because I contacted may peoples about it but all ready to help in python and they told they have no idea about opencv.js. but you are really helping me thank you @Lucy8

Adam108 gravatar imageAdam108 ( 2019-05-31 00:54:16 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2019-05-15 02:09:26 -0600

Seen: 2,147 times

Last updated: May 24 '19