Ask Your Question
0

Identifying and selecting floor in a room

asked 2018-11-16 07:14:08 -0500

funkycoder gravatar image

I am new to OpenCV and have been reading a lot to help me achieve this. I need to be able to identify the floor of a room. I have looked at a lot of other questions and answers but none of them have been able to help me out.

One of the suggestions said:

  1. Load your image

  2. Sharpen your image by applying a Laplacian convolution filter

  3. Convert your image to grayscale

  4. Threshold the image to filter out noisy components

  5. Morph, dilate then apply watershed to separate planes

  6. Then find contours in the filtered image

Here is the code that I have tried so far:

import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.core.MatOfPoint;
import org.opencv.core.Point;
import org.opencv.core.Scalar;
import org.opencv.core.Size;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;
import org.opencv.photo.Photo;

import java.util.ArrayList;
import java.util.List;
import java.util.Random;

public class IdentifyRoom {
  public static void main(String []args) {
    System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

    Random rng = new Random(12345);

    Mat srcGray = new Mat();
    String filename = args.length > 0 ? args[0] : "../../room.jpg";
    Mat source = Imgcodecs.imread(filename);
    if (source.empty()) {
      System.err.println("Cannot read image: " + filename);
      System.exit(0);
    }

    Mat destination = new Mat(source.rows(), source.cols(), source.type());
    int kernelSize = 9;
    Mat kernel = new Mat(kernelSize,kernelSize, CvType.CV_32F) {
      {
        put(0,0,0);
        put(0,1,-1);
        put(0,2,0);

        put(1,0-1);
        put(1,1,4);
        put(1,2,-1);

        put(2,0,0);
        put(2,1,-1);
        put(2,2,0);
      }
    };

    Imgproc.filter2D(source, destination, -1, kernel);

    /// Convert image to gray and blur it
    Imgproc.cvtColor(source, srcGray, Imgproc.COLOR_BGR2GRAY);
    Imgproc.blur(srcGray, srcGray, new Size(3, 3));

    // histogram equalization
    Imgproc.equalizeHist(srcGray, srcGray);

    // denoise
    Imgproc.threshold(srcGray, srcGray, 25, 255, Imgproc.THRESH_BINARY_INV);

    Mat morphingMatrix = Mat.ones(1,1, CvType.CV_8UC1);
    Imgproc.morphologyEx(srcGray, srcGray, Imgproc.MORPH_OPEN, morphingMatrix);

    // Image denoising
    Photo.fastNlMeansDenoising(srcGray, srcGray);

    // dilate
    int dilation_size = 5;
    Mat element1 = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new  Size(dilation_size, dilation_size));
    Imgproc.dilate(srcGray, srcGray, element1);

    // watershed // does not work
    /* Mat marker_tempo = new Mat();
    Mat rgba = new Mat();
    Mat threeChannel = new Mat();
    Imgproc.cvtColor(rgba, threeChannel, Imgproc.COLOR_RGB2GRAY);
    Imgproc.threshold(threeChannel, threeChannel, 100, 255, Imgproc.THRESH_OTSU);

    Mat markers = new Mat(rgba.size(),CvType.CV_8U, new Scalar(0));

    markers.convertTo(marker_tempo, CvType.CV_32S);
    Imgproc.watershed(rgba, marker_tempo);
    marker_tempo.convertTo(markers,CvType.CV_8U); */


    /// Find contours
    List<MatOfPoint> contours = new ArrayList<>();
    Mat hierarchy = new Mat();
    Imgproc.findContours(srcGray, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);

    Mat drawing = Mat.zeros(srcGray.size(), CvType.CV_8UC3);
    // Scalar color = new Scalar(rng.nextInt(256), rng.nextInt(256), rng.nextInt(256));
    for (int i = 0; i < contours.size(); i++) {
      Scalar color = new Scalar(rng.nextInt(256), rng.nextInt(256), rng.nextInt(256));
      // Imgproc.drawContours(drawing, contours, i, color, Imgproc.LINE_8);
      Imgproc.drawContours(drawing, contours, i, color, 2, Imgproc.LINE_8, hierarchy, 0, new Point());
    }

    Imgcodecs.imwrite("../../output1.jpg", drawing);
  }
}

Source

The source image

Output

The output

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
3

answered 2018-11-16 08:11:52 -0500

berak gravatar image

i can't really answer this, only give more "bad advice" ;)

i'd say, noo, don't throw away the color information, also thresholding will kill everything with shadows on it

and the more "low-level" you make it, the more parameters you will have to handle / tweak.

this is a segmentation problem, so try to use methods, that specialize on that. e.g: graph_segmentation:

image description

(graphsegmentation_demo room.jpg out.png 0.7 400 400)

there's also a hfs module:

image description

you could also use SLIC or SEEDS superpixels for a rough "pre-segmentation", and try to do something with their center colors

(cc @Martian :)

edit flag offensive delete link more

Comments

Oh my! I really did not expect this. Thanks a million ton. It looks like segmentation works quite well for the scenario I am looking into. I had no idea at all which way to go when I had started.

I do not even know the full potential of OpenCV. Is it possible to find the coordinates of the polygon which make up the floor?

funkycoder gravatar imagefunkycoder ( 2018-11-16 08:42:08 -0500 )edit
1

the colors are only for visualization. you can also get an "indexed label" mat from those algos (similar to connectedComponents())

apart from that, your original attempt did not come that far, but itwould have been the same problem: which of the blobs is the floor ? i think, you must do some "educated guess", like: the most bottom one, in the middle.

berak gravatar imageberak ( 2018-11-16 09:16:55 -0500 )edit

@funkycoder -- Please don't forget to upvote berak's answer, as well as mark it as correct by clicking on the check-mark. Pretty cool project!

sjhalayka gravatar imagesjhalayka ( 2018-11-16 13:41:17 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-11-16 07:14:08 -0500

Seen: 38 times

Last updated: Nov 16