Ask Your Question
0

calibration between mouse move & pupil move (eye writer)

asked 2016-08-25 23:35:58 -0600

eshahnazi gravatar image

updated 2016-08-26 10:53:36 -0600

hello i write program for detect face & then detect Right eye & then detect pupil (circle hough) Finally move mouse.

The problem I have is that ‌Move the mouse range is low. the range of motion mouse is 20 x 20 pixels. Mouse cursor moves only in a very small area of the desktop.

I need Mouse cursor moves in all area of the desktop.

This code get the coordinates of the center of the circular Hough and then cursor is moved .

......
......
.....

 void mousemove(int x_pos, int y_pos)
{
    ///Strings that will contain the conversions
    string xcord; string ycord;

    ///These are buffers or something? I don't really know... lol.
    stringstream sstr; stringstream sstr2;

    ///Conversion to regular string happens here
    sstr<<5*x_pos;
    xcord = sstr.str();
    sstr2<<5*y_pos;
    ycord = sstr2.str();

    ///Getting the command string
    string command = "xdotool mousemove " + xcord + " " + ycord;

    ///Converting command string to a form that system() accepts.
    const char *com = command.c_str();
    system(com);
}


int main()
{
    CascadeClassifier face_cascade("/home/elinux/opencv-3.1.0/data/haarcascades_cuda/haarcascade_frontalface_alt2.xml");
    CascadeClassifier eye_cascade("/home/elinux/opencv-3.1.0/data/haarcascades_cuda

/haarcascade_righteye_2splits.xml");
    ....
    ....
    ...
morphologyEx(framepupil, morph_grad, MORPH_GRADIENT,se,Point(-1,-1),1);
HoughCircles(morph_grad,  circles,  HOUGH_GRADIENT, 1,  1000,  100,15, 10,20);
if(circles.size()!=0){
        (circles[t][0])=(circles[t][0])*((circles[t][0])/10);
        (circles[t][1])=(circles[t][1])*((circles[t][1])/10);
        mousemove((circles[t][0]),(circles[t][1]));
        }
....
.....
....

Here are the all codes: https://github.com/Ehsan-Shahnazi/EyeWriter/blob/master/22-22.cpp

How should calibration between eye positions and cursor positions?? Do you have code samples?

(Perhaps the problem of how to track pupil or codes to move the mouse or...???)

thanks

edit retag flag offensive close merge delete

Comments

Please, update your question and make the actual problem more clear. For now I see that

  • You have working code for all parts
  • You have an output

So what do you want? A better performance? Is the registration to poor? Also a eye motion will never be larger than around 40x40 pixels if you simply look at the pupil region. Are you sure you reprojected it correctly to screen coordinates?

StevenPuttemans gravatar imageStevenPuttemans ( 2016-08-26 04:59:38 -0600 )edit

Please add the relevant code, that translates the movement of the eye into screen coordinates. You are aware that you will need to do some calibration/mapping between both right?

StevenPuttemans gravatar imageStevenPuttemans ( 2016-08-26 06:03:21 -0600 )edit

2 answers

Sort by » oldest newest most voted
1

answered 2016-08-28 08:21:34 -0600

eshahnazi gravatar image

updated 2016-08-29 06:07:44 -0600

hello MR @StevenPuttemans i can Coordinate mouse move & pupil move...it very easy..!!! just i change this code:

sstr<<1.2*x_pos;
xcord = sstr.str();
sstr2<<0.9*y_pos;
ycord = sstr2.str();

1.2 & 0.9 is Multiple desktop to start the coordinate . (google translate!!!) before this Multiple is 5, As a result of moving the mouse was missing. with my code i dont need calibration!!! just use this code for control mouse move:

int xx=(circles[i][1])-x;
x=(circles[i][1]);
xx=abs(xx);
int yy=(circles[i][1])-y;
y=(circles[i][1]);
yy=abs(yy);

if(circles.size()!=0 ){ //اگر دایره ای در تصویر تشخیص داده شد شرط اجرا شود
    if (xx>20 || yy>20){
        (circles[i][0])=(circles[i][0])*(2);
        (circles[i][1])=(circles[i][1])*(2);
         mousemove((circles[i][0]),(circles[i][1]));}}

Unfortunately, my English is not good otherwise I wrote a fuller explanation.

my project code ; https://github.com/Ehsan-Shahnazi/ video of my project: https://www.youtube.com/user/eshahnazi very thanks mr@StevenPuttemans

edit flag offensive delete link more

Comments

Well good for you that you got it working. However, this will be very dependent on the person itself and thus require you to set those values for each individual.

StevenPuttemans gravatar imageStevenPuttemans ( 2016-08-29 04:12:00 -0600 )edit

dont need change value...for anyone sstr<<1.2*x_pos; & sstr2<<0.9*y_pos; &

 if (xx>20 || yy>20){
        (circles[i][0])=(circles[i][0])*(2);
        (circles[i][1])=(circles[i][1])*(2);

is Fixed.

eshahnazi gravatar imageeshahnazi ( 2016-08-29 06:06:55 -0600 )edit

Okay but lets be fair, it works on your setup. There is absolutely no relation at all now. Your pointer in your demo video jumps around like crazy. I do not see how this is usefull?

StevenPuttemans gravatar imageStevenPuttemans ( 2016-08-29 07:02:04 -0600 )edit

this jumps because of bad pupil detection and not related jumps mouse. i use houghcircles for detection pupil that It seems not suitable. In your opinion, what is the proper way to identify pupils in the photo below? /* pupil image */ thanks

eshahnazi gravatar imageeshahnazi ( 2016-08-29 07:24:34 -0600 )edit
1

answered 2016-08-26 06:26:17 -0600

Wait you are forgetting the most essential part of the solution. Once you have found the coordinates of the eyes, the iris inside can only move in a small region. Then you will need a calibration between those positions and the image positions, like what is being done with eyetracker camera's.

Take a look here: http://www.tobiipro.com/learn-and-sup...

edit flag offensive delete link more

Comments

yes. you are right. How should I calibration between those positions and the image positions. Do you have code samples? thanks

eshahnazi gravatar imageeshahnazi ( 2016-08-26 06:52:11 -0600 )edit

i have coordinates of right eye area (not pupil area). The following code

croppedRectangle2=Rect(eyes[t].x,eyes[t].y+40,eyes[t].width ,eyes[t].height-40);

can i use this coordinates (right eye area) for calibration?? example coordinates right eye area is A rectangle with a length and width 80*100px.

eshahnazi gravatar imageeshahnazi ( 2016-08-26 07:01:56 -0600 )edit

This is not so straightforward as you think, nor do I have sample codes for this, since this is a very specific problem. You will have to google the web for that.

StevenPuttemans gravatar imageStevenPuttemans ( 2016-08-26 07:18:59 -0600 )edit

with change this code;

if(circles.size()!=0){
        (circles[t][0])=(circles[t][0])*((circles[t][0])/10);
        (circles[t][1])=(circles[t][1])*((circles[t][1])/10);
        mousemove((circles[t][0]),(circles[t][1]));
        }

I multiplied a constant factor ( this code: ( *((circles[t][0])/10); )) with center eye coordinates. in this case, the range of mouse move is large...It is true?

However, moving the mouse will be very fast and With noise.

if i can reduce noise, in this case Program is complete (Approximately)...!!

eshahnazi gravatar imageeshahnazi ( 2016-08-26 07:32:40 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2016-08-25 23:35:58 -0600

Seen: 1,993 times

Last updated: Aug 29 '16