Ask Your Question
0

Quickly capture N frames and continue with live feed

asked 2015-07-02 04:30:51 -0500

Prem gravatar image

I would like to capture N number of images process them and do something with live feed from the camera. I start camera, capture 30 frames and store them in a Mat vector. Now when I try to access or process the vector I am getting runtime error. I am using pointgray camera. I think I am missing something really basic. Kindly guide me what could be wrong?

      #include <opencv2\highgui\highgui.hpp>
      #include <opencv2\core\core.hpp>
      #include <opencvlibpath.h>
      #include <FlyCapture2.h>
      #include <vector>

      using namespace cv;
      using namespace FlyCapture2;
      using namespace std;

        const static int NUMBER_OF_FRAME_CAPTURE = 30;
       const static int SENSITIVITY_VALUE = 60;
       const static int BLUR_SIZE = 50;


       //I would like to execute this function after quickly capturing 30 frames
        void getThresholdImage(vector<Mat>  &framesToProcess, vector<Mat> &thresholdImages)
       {
      vector<Mat> grayImage;

for (int i = 0; i < framesToProcess.size(); i++)
{
    Mat tempMatImage, tempGrayImage;

    resize(framesToProcess[i], tempMatImage, Size(600, 800));
    cvtColor(tempMatImage, tempGrayImage, COLOR_BGR2GRAY);
    grayImage.push_back(tempGrayImage);

    if (i > 0)
    {
        Mat tempDifferenceImage, tempThresholdImage;
        absdiff(grayImage[i - 1], grayImage[i], tempDifferenceImage);
        threshold(tempDifferenceImage, tempThresholdImage, SENSITIVITY_VALUE, 255, THRESH_BINARY);
        blur(tempThresholdImage, tempThresholdImage, Size(BLUR_SIZE, BLUR_SIZE));
        thresholdImages.push_back(tempThresholdImage);
    }
}
 }

  int main()
 {
Mat cameraFeed;

vector<Image> rawImages;
vector<Mat> framesToProcess;
vector<Mat> thresholdImages;
//vector<point> contourCenters;
vector<Point> LEDPoints;

Camera camera;
Error error;

error = camera.Connect(0);
if (error != PGRERROR_OK)
{
    cout << "Failed to connect to camera" << endl;
    getchar();
    exit(1);
}
error = camera.StartCapture();
if (error == PGRERROR_ISOCH_BANDWIDTH_EXCEEDED)
{
    cout << "Bandwidth exceeded" << endl;
    getchar();
    exit(1);
}
else if (error != PGRERROR_OK)
{
    cout << "Failed to start image capture" << endl;
    getchar();
    exit(1);
}

while (1){

    framesToProcess.clear();
    thresholdImages.clear();
    rawImages.clear();

    //quickly capture 30 images
    for (int i = 0; i < NUMBER_OF_FRAME_CAPTURE; i++)
    {
        Mat currentFrame;
        Image rawImage;
        Error error = camera.RetrieveBuffer(&rawImage);
        if (error != PGRERROR_OK)
        {
            cout << "capture error" << endl;
        }

        // convert to rgb
        Image rgbImage;
        rawImage.Convert(FlyCapture2::PIXEL_FORMAT_BGR, &rgbImage);

        // convert to OpenCV Mat
        unsigned int rowBytes = (double)rgbImage.GetReceivedDataSize() / (double)rgbImage.GetRows();
        currentFrame = Mat(rgbImage.GetRows(), rgbImage.GetCols(), CV_8UC3, rgbImage.GetData(), rowBytes);
        framesToProcess.push_back(currentFrame);
        //cvtColor(currentFrame, currentFrame, COLOR_BGR2GRAY);
        //imshow("GRAY", currentFrame);
    }

    // this line returns 30
    cout << "Frames to process" << framesToProcess.size() << endl;



    // I get runtime error while calling this function  
    getThresholdImage(framesToProcess, thresholdImages);

    //Then I just tried displaying image from captured Mat Vector above with
    //imshow("frame", framesToProcess[0]);
    // Still I get runtime error which means it populates the vector with images but it is not accessible

    // Following section tries to continue with live feed. 
    //Already tried commenting this section but doesn't have any effect
    Image rawImage;
    Error error = camera.RetrieveBuffer(&rawImage);
    if (error != PGRERROR_OK)
    {
        cout << "capture error" << endl;
    }

    // convert to rgb
    Image rgbImage;
    rawImage.Convert(FlyCapture2::PIXEL_FORMAT_BGR, &rgbImage);

    // convert to OpenCV Mat
    unsigned int rowBytes = (double)rgbImage.GetReceivedDataSize() / (double)rgbImage.GetRows();
    cameraFeed = Mat(rgbImage.GetRows(), rgbImage.GetCols(), CV_8UC3, rgbImage.GetData(), rowBytes);
    resize(cameraFeed, cameraFeed, Size(800, 600));

    imshow("camera feed", cameraFeed);
    waitKey(10);
}

}
edit retag flag offensive close merge delete

Comments

Here is the runtime error I get First-chance exception at 0x00007FFF1EB42262 (opencv_core2410d.dll) in LearnCPP11.exe: 0xC0000005: Access violation reading location 0x00000028F52A2840. Unhandled exception at 0x00007FFF1EB42262 (opencv_core2410d.dll) in LearnCPP11.exe: 0xC0000005: Access violation reading location 0x00000028F52A2840.

Prem gravatar imagePrem ( 2015-07-02 04:34:46 -0500 )edit

I think there is a problem with this line :

currentFrame = Mat(rgbImage.GetRows(), rgbImage.GetCols(), CV_8UC3, rgbImage.GetData(), rowBytes);

if you look this
data – Pointer to the user data. Matrix constructors that take data and step parameters do not allocate matrix data. Instead, they just initialize the matrix header that points to the specified data, which means that no data is copied. This operation is very efficient and can be used to process external data using OpenCV functions. The external data is not automatically deallocated, so you should take care of it.

It seems that you are using same memory for different images

LBerger gravatar imageLBerger ( 2015-07-02 04:37:44 -0500 )edit

currentFrame is added to the vector. does the vector only contain the reference as well? This works perfectly fine if I include my code to process each frame within for loop. but this will delay the image capture. What can be the workaround for this?

Prem gravatar imagePrem ( 2015-07-02 04:49:01 -0500 )edit

Yes you are right but problem is in push_back. Push_back does not make a copy of your Mat. If you watch in memory currentFrame and framesToProcess you can see that it is same Mat object. Push_back make a data field copy. I think you have to use Clone

LBerger gravatar imageLBerger ( 2015-07-02 05:07:50 -0500 )edit

...remeber that cv::Mat has its own memory management and Mat::operator = and .. is an O(1) operation. This means that no data is copied....

You have to preallocate your std::vector so to have one cv::Mat for each elements than use Mat::copyTo. For example, instead of grayImage.push_back(tempGrayImage) you could use grayImage.resize( framesToProcess.size()) and after tempGrayImage.copyTo(grayImage[i]).

pklab gravatar imagepklab ( 2015-07-02 06:16:26 -0500 )edit

@LBerger thanks for saving me clone is working. But I am getting real performance issue now. Any tips on optimization of this code? @pklab thanks for tips.

Prem gravatar imagePrem ( 2015-07-02 06:35:14 -0500 )edit

Tips is good/pre allocation. You can create and allocate your vector of N grey scale images 800x600 on 1 shoot with std::vector<cv::Mat> images(N,cv::Mat(cv::Size(800,600),CV_C8U1)); than work directly on mages[i], or if you like, create a shortname for it like tmpImage = images[i];

pklab gravatar imagepklab ( 2015-07-02 08:51:14 -0500 )edit

Thanks a lot guys. @LBerger as your solution of clone worked perfectly. Would you please add it as answer to this question. If needed I would post a new question.

Prem gravatar imagePrem ( 2015-07-02 09:23:46 -0500 )edit

1 answer

Sort by » oldest newest most voted
1

answered 2015-07-02 06:54:24 -0500

LBerger gravatar image

updated 2015-07-02 09:30:31 -0500

If you know you need thirty images why don't you allocate 30 images before the loop and copy acquire image in image i (i ranging 0 to 29)

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2015-07-02 04:30:51 -0500

Seen: 189 times

Last updated: Jul 02 '15