OpenCV UMat conversion from std::vector

asked 2017-01-09 21:07:53 -0500

jpistorino gravatar image

I am using VS2012 C++/CLI with OpenCV 3.1 and Win10.

I am trying to get cameracalibration working and appear to be having an issue with converting the vector of Point2f returned by findChessboardCorners to a UMat either just generally or for use with drawChessboardCorners.

Calling drawChessboardCorners or even just trying to convert the vector to a UMat causes an exception. Checking the pointBuf vector after findChessboardCorners indicates that the vector is not empty. I am not sure why I get this runtime exception and any help or even indication of how better to debug it is appreciated.

MY code is as follows and crashes on either the commented out drawChessboardCorners call or the effort to convert the pointBuf vector to a UMat called tpoints.

void Channel::Calibration(videoInput* CameraVI,bool Enabled,int CameraNumber, unsigned char * LdataBuffer, cvImages* Images,int ChannelNumber,bool MirrorH, bool MirrorV, bool MainWinCond,ImageSource^ ImageEvents)
{
    int totalframes = 0;
    int numCornersHor = CALIBRATE_CORNERS_HORIZONTAL;
    int numCornersVer = CALIBRATE_CORNERS_VERTICAL;
    Size board_sz = Size(numCornersHor,numCornersVer);
    std::vector<std::vector<Point2f> > imagePoints;
    int successes = 0;
    UMat tImage;
    UMat grayImage;
    System::String^ tfname;

    UMat tpoints;

    bool found = false;
    TermCriteria criteria = TermCriteria( CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 30, 0.1 );

    long mDefaultmin,mDefaultmax,mDefaultStep,mDefaultFlags,mDefaultValue;
    long mDefaultfocus,mDefaultzoom;

    Size imageSize;
    std::vector<Point2f> pointBuf;
    int tpointBufsize;

    if(Enabled)
    {
        CameraVI->setVideoSettingCamera(CameraNumber,CameraControl_Focus,mDefaultfocus,CameraControl_Flags_Manual);
        CameraVI->getVideoSettingCamera(CameraNumber,CameraControl_Zoom,mDefaultmin,mDefaultmax,mDefaultStep,mDefaultzoom,mDefaultFlags,mDefaultValue);
        CameraVI->setVideoSettingCamera(CameraNumber,CameraControl_Zoom,mDefaultzoom,CameraControl_Flags_Manual);

        while(successes<CALIBRATE_NUMBER_OF_BOARDS_TO_MATCH && totalframes<CALIBRATE_TOTAL_FRAMES)
        {
            pointBuf.clear();

            // check for new image
            if(CameraVI->isFrameNew(CameraNumber))
            {
                totalframes++;

                // load the image from the buffer
                CameraVI->getPixels(CameraNumber, LdataBuffer, false, true);

                // convert it to a mat
                cv::Mat ttImage(Images->Height, Images->Width, CV_8UC3, LdataBuffer, UMat::AUTO_STEP);
                cv::UMat tImage = ttImage.getUMat(ACCESS_RW);

                found = findChessboardCorners(tImage, board_sz, pointBuf, CV_CALIB_CB_ADAPTIVE_THRESH | CV_CALIB_CB_FAST_CHECK| CV_CALIB_CB_NORMALIZE_IMAGE);

                if(found)
                {
                    cvtColor(tImage,grayImage,CV_BGR2GRAY);
                    cornerSubPix( grayImage, pointBuf, Size(11,11), Size(-1,-1), criteria);
                    imagePoints.push_back(pointBuf);
                    successes++;
                    putText(tImage,msclr::interop::marshal_as<std::string>("Calibrate Successes: " + successes.ToString() + " TotalFrames: " + totalframes.ToString()),Point(40,40),FONT_HERSHEY_SIMPLEX,1.5,OPENCV_WHITE,3);
                    tpointBufsize = pointBuf.size();
//                  drawChessboardCorners( tImage, board_sz, UMat(pointBuf), found );
                    Sleep(2000);
                }
                else
                    putText(tImage,msclr::interop::marshal_as<std::string>("Calibrate Successes: " + successes.ToString() + " TotalFrames: " + totalframes.ToString()),Point(40,40),FONT_HERSHEY_SIMPLEX,1.5,OPENCV_WHITE,3);


                tpoints = UMat(pointBuf,false);

                ImageEvents->FireCVImage(ChannelNumber,MainWinCond,tImage);
            }
            Sleep(200);
        }
        CameraVI->setVideoSettingCamera(CameraNumber,CameraControl_Zoom,mDefaultzoom,CameraControl_Flags_Auto);

        imageSize = tImage.size();

        std::vector<UMat> rvecs, tvecs;
        std::vector<float> reprojErrs;

        Calibrated = runCalibration(tImage, CALIBRATE_SQUARE_SIZE , grayImage.size(), imagePoints, rvecs, tvecs, reprojErrs);

        ImageEvents->FireCVCalibrationDetails(ChannelNumber,Calibrated,*intrinsic,*distCoeffs,RMS,totalAVGErr);

        initUndistortRectifyMap(*intrinsic, *distCoeffs,cv::UMat(),*intrinsic,imageSize,CV_16SC2,*map1,*map2);

        saveCameraCalibration(gcnew System::String(CameraPath) + Images->Width.ToString() + "x" + Images->Height.ToString() + "y.xml");
    }
    else
        ImageEvents->FireCVStatus(ChannelNumber,"Calibrate Camera Not Enabled!");
}
edit retag flag offensive close merge delete

Comments

no idea if it helps, but try: UMat(pointBuf, true) , so points really get copied to gpu memory.

http://docs.opencv.org/master/d7/d45/...

berak gravatar imageberak ( 2017-01-10 02:04:19 -0500 )edit

Thanks Berak. That seems to be the answer. I was following the sample cameracalibration code so that may need to be changed or it may be just a bug overall.

jpistorino gravatar imagejpistorino ( 2017-01-10 11:29:40 -0500 )edit

though i'm happy to hear this, - which tutorial do you mean exactly ?

(i can't see anthing there, using UMat (or lord have mercy - CLI))

berak gravatar imageberak ( 2017-01-10 11:43:13 -0500 )edit

Unfortunately, I think it is too late for a higher power to intervene on my platform choice.

Here is the code I generally followed: http://docs.opencv.org/2.4/doc/tutori...

It is true that this sample uses Mat but I thought UMat was supposed to be a drop in replacement for Mat (whether you get any performance improvement is a different issue). .

jpistorino gravatar imagejpistorino ( 2017-01-10 12:49:24 -0500 )edit

"for a higher power to intervene on my platform choice." yea, i'm just pulling your leg (think "schadenfreude")

using UMat, again you want to keep all of your data on the GPU, as long as you can, no ?

opencv2.4 does not use UMat at all, but - i'd think - maybe you you should use your experiences there, to add another sample/tutorial with opencv3.x, - to help other folks trying that path ?

berak gravatar imageberak ( 2017-01-10 13:00:16 -0500 )edit