Ask Your Question
0

Problems with Augmented Reality

asked 2017-10-08 00:38:18 -0500

kimchiboy03 gravatar image

updated 2017-10-08 01:56:56 -0500

Hello, I am trying to create a program which uses augmented reality. However, I am currently stuck on drawing the object. This is my code so far:

#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/calib3d/calib3d.hpp"

using namespace std;
using namespace cv;

Mat camMat;
Mat distortion;

//Initializes augmented reality
void initAR()
{
    FileStorage fs("out_camera_data.xml", FileStorage::READ);

    fs["Camera_Matrix"] >> camMat;
    fs["Distortion_Coefficients"] >> distortion;
}

//Load 3d model
void loadModel(vector<Mat> squares, Mat colorImg)
{
    vector<Point3f> objectPt = { Point3f(-1, -1, 0), Point3f(-1, 1, 0), Point3f(1, 1, 0), Point3f(1, -1, 0) };
    Mat objectMat(objectPt);

    Mat rvec;
    Mat tvec;
    solvePnP(objectMat, squares[0], camMat, distortion, rvec, tvec);

    vector<Point3f> line3d[4];
    line3d[0] = { { 1, 1, 0 }, { 1, 0, 0 }, { 0, 0, 0 } };
    //line3d[1] = { { -1, 0, 0 },{ 1, 0, 1 } };
    //line3d[2] = { { 1, 0, 0 },{ -1, 0, 1 } };
    //line3d[3] = { { 1, 0, 0 },{ 1, 0, 1 } };

    vector<Point2f> line2d[4];
    projectPoints(line3d[0], rvec, tvec, camMat, distortion, line2d[0]);

    polylines(colorImg, line2d[0][0], true, Scalar(255, 0, 0), 2);
}

//Displays 3d object
void doAR(Mat colorImg)
{
    Mat bwImg;
    Mat blurImg;
    Mat threshImg;
    vector<vector<Point> > cnts;

    if (!colorImg.empty())
    {
        cvtColor(colorImg, bwImg, CV_BGR2GRAY);
        blur(bwImg, blurImg, Size(5, 5));
        threshold(blurImg, threshImg, 128.0, 255.0, THRESH_OTSU);
        findContours(threshImg, cnts, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);

        vector<Mat> squares;
        for (auto contour : cnts)
        {
            vector<Point> approx;
            approxPolyDP(contour, approx, arcLength(Mat(contour), true)*0.02, true);
            if (approx.size() == 4 && fabs(contourArea(Mat(approx))) > 1000 && isContourConvex(Mat(approx)))
            {
                Mat square;
                Mat(approx).convertTo(square, CV_32FC3);
                squares.push_back(square);
            }
        }

        if (squares.size() > 0)
        {
            loadModel(squares, colorImg);
        }

        cvNamedWindow("AR", 0);
        cvSetWindowProperty("AR", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
        imshow("AR", colorImg);
        waitKey(1);
    }
}

I am trying to draw a triangle with the points in line3d. Can someone please tell me why this isn't drawing? If anyone needs to see more of my code, I will post more on my answer.

Thanks in advance.

edit retag flag offensive close merge delete

Comments

that should not even compile !

berak gravatar imageberak ( 2017-10-08 01:46:01 -0500 )edit

@berak That's just part of my code. I'll post more of my code.

kimchiboy03 gravatar imagekimchiboy03 ( 2017-10-08 01:55:26 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2017-10-08 02:19:05 -0500

berak gravatar image

updated 2017-10-08 02:52:20 -0500

polyLines is quite a tricky beast ;)

it expects a vector<vector<Point>> as input (also, integer Points, not float ones !)

here's a simple example:

Mat ocv = Mat(100,100,CV_8UC3,Scalar::all(40));
vector<vector<Point>> line2d(1);
line2d[0] = { { 50, 50 }, { 50, 20 }, { 20, 20 } };
polylines(ocv, line2d, true, Scalar(255, 0, 0), 2);

image description

your case is a bit more complicated, because you have to copy the float points into a new, integer point array, before you can use polyLines():

 vector<Point2f> line2d[4] = ... //projected 
 vector<vector<Point>> newpts(4);
 for(size_t i=0; i<4; i++) {
     for(size_t j=0; j<line2d[i].size(); j++) {
           newpts[i].push_back(line2d[i][j]);
     }
 }
 polylines(colorImg, newpts, true, Scalar(255, 0, 0), 2);

(idk, maybe it's easier, to use many simple line() calls for this ?)

edit flag offensive delete link more

Comments

I'm currently struggling to integrate this into my code. Firstly, I cannot convert vector<point2f> to vector<vector<point2f>>... Can you please help me out a little bit more?

kimchiboy03 gravatar imagekimchiboy03 ( 2017-10-08 02:45:53 -0500 )edit

ah, my bad, should have kept it closer to your example, let me edit ..

better, now ?

berak gravatar imageberak ( 2017-10-08 02:50:12 -0500 )edit

Thank you. My programs seems to work now. Thanks once again.

kimchiboy03 gravatar imagekimchiboy03 ( 2017-10-08 02:56:57 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-10-08 00:38:18 -0500

Seen: 132 times

Last updated: Oct 08 '17