Ask Your Question
0

calcOpticalFlowPyrLK and goodFeaturesToTrack doesn't work properly in Ros

asked 2015-04-20 16:02:58 -0500

215 gravatar image

updated 2015-04-20 16:04:03 -0500

I trying to port some code i written in C++ non ROS into a Ros environment, which causing some problems.. The purpose of this code is to track the movement of a persons face, a bit like the pi_face_tracker (why not use the existing => my uses haar cascades when the point.size() drops below a threshold) The problem though, is that the opticalFlow isn't working properly.. The imshow screen doesn't say where the new points are, but just keep the old points. Which makes me suspect if it even tracking the movements of the point or not?..

here is the code:

#include "image_converter.h"
#include <termios.h>
int getch()
{
  static struct termios oldt, newt;
  tcgetattr( STDIN_FILENO, &oldt);           // save old settings
  newt = oldt;
  newt.c_cc[VMIN] = 0; newt.c_cc[VTIME] = 0;
  newt.c_lflag &= ~(ICANON);                 // disable buffering      
  tcsetattr( STDIN_FILENO, TCSANOW, &newt);  // apply new settings

  int c = getchar();  // read character (non-blocking)

  tcsetattr( STDIN_FILENO, TCSANOW, &oldt);  // restore old settings
  return c;
}


ImageConverter::ImageConverter()
    : it_(nh_),pos(2),  vel(2) 
  {
    // Subscrive to input video feed and publish output video feed

        image_sub_ = it_.subscribe("/images/left/image_rect_color", 1, 
        &ImageConverter::imageDetect, this);
        image_pub_ = it_.advertise("/image_converter/output_video", 1);
        state = false;
        reinit = false;
        redetect = false;
        frames = 0;
        cout << "start over" << endl;
  }

 ImageConverter::~ImageConverter()
  {
    //cv::destroyAllWindows();
  }

  void ImageConverter::imageDetect(const sensor_msgs::ImageConstPtr& msg)
  {  
     facedetect_pub = nh_.advertise<sensor_msgs::JointState>("/ptu/cmd", 1);
     sub = nh_.subscribe("/joint_states",1,&ImageConverter::chatterCallback,this); 
     cv_bridge::CvImagePtr cv_ptr;
     sensor_msgs::JointState ms;
    try
    {
      cv_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::BGR8);
    }
    catch (cv_bridge::Exception& e)
    {
      ROS_ERROR("cv_bridge exception: %s", e.what());
      return;
    }
    if(!(state))
    {
        vector<Rect> faces(1);
        Point center_of_frame(cv_ptr->image.size().width/2,cv_ptr->image.size().height/2);
        pair<Point, Point> corners;
        pair<double,double> displacement;
        double displacement_pixel_x;
        double displacement_pixel_y;
        pair<Rect, bool> response;

            if (cv_ptr->image.type()!= 8) {
                cvtColor(cv_ptr->image, cv_ptr->image, CV_8U);
            }

    //-- 1. Load the cascades
            if( !face_cascade.load( face_cascade_XML ) ){
                cout << "Cascade Error" << endl;
            };

            circle(cv_ptr->image, center_of_frame, 1, CV_RGB(0,255,255),8,8,0);

    //-- Detect faces
            face_cascade.detectMultiScale( cv_ptr->image, faces, 1.1, 2, 0, Size(100, 100) );
            if(faces.empty())
            {
                if( !face_cascade.load( face_cascade_XML1 ) )
                {
                    cout << "Cascade Error" << endl;
                };
                face_cascade.detectMultiScale( cv_ptr->image, faces, 1.1, 2, 0, Size(100, 100) );


                if( !face_cascade.load( nose_xml ) )
                {
                    cout << "Cascade Error" << endl;
                };
                face_cascade.detectMultiScale( cv_ptr->image, faces, 1.1, 2, 0, Size(100, 100) );

            }
            for( int i = 0; i<faces.size() ; i++)
            {
                Point center_position_of_face((faces[i].br().x+faces[i].tl().x)/2,(faces[i].tl().y+faces[i].br().y)/2);
                Point corner_1(faces[i].br().x,faces[i].tl().y);
                Point corner_2 = faces[i].tl();
                Point corner_3 = faces[i].br();
                Point corner_4(faces[i].tl().x,faces[i].br().y);
                rectangle(cv_ptr->image, faces[i], CV_RGB(0,255,0),4,8,0);
                circle(cv_ptr->image, center_position_of_face, 8, CV_RGB(128,128,128),8,8,0);
                circle(cv_ptr->image, corner_1, 1, CV_RGB(128,128,128),8,8,0);
                circle(cv_ptr->image, corner_2, 1, CV_RGB(128,128,128),8,8,0);
                circle(cv_ptr->image, corner_3, 1, CV_RGB(128,128,128),8,8 ...
(more)
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
1

answered 2015-04-21 11:50:59 -0500

dtmoodie gravatar image

prevGrey's scope is within the imageDetect scope, thus it is destroyed on every iteration of the function. Either define prevGrey as static or make it a member variable that is updated at each iteration.

edit flag offensive delete link more

Comments

thank you very much :)

215 gravatar image215 ( 2015-04-22 02:48:35 -0500 )edit

Any suggestion on improvements?

215 gravatar image215 ( 2015-04-22 02:49:16 -0500 )edit
1

"Any suggestion on improvements?" - yes. do not load a cascade file for each image you process, but only once in the class constructor

berak gravatar imageberak ( 2015-04-22 03:42:16 -0500 )edit

Cool.. => indeed a CPU killer.. I added the cascade classifier, to compensate the issue that lukas kanade wasn't that good to focus on a certain area.. The cascade classier creates a ROI of the face i want to track.. problem is though when the head moves, outside of this ROI it becomes harder to track the face since the points become more spread.. but cascade classifier are only able to detect the face when they are at a certain pose, Any idea to overcome this problem?

215 gravatar image215 ( 2015-04-22 04:02:24 -0500 )edit

You could consider using something like CMT (http://www.gnebehay.com/cmt/) to tracking the face once your cascade classifier has locked on.

dtmoodie gravatar imagedtmoodie ( 2015-04-29 16:05:39 -0500 )edit

Good idea... Though CMT don't make it possible to track 360 degrees. which kanade does since it is just a within mask it detects movements.

215 gravatar image215 ( 2015-04-29 20:13:24 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2015-04-20 16:02:58 -0500

Seen: 619 times

Last updated: Apr 21 '15