Seltymar's profile - activity

I have asked this question on stack overflow and has been answered : http://stackoverflow.com/questions/14...

Here is the copy of the answer given in stackoverflow:

As I understand you have two separate GMMs for the foreground and background part of the image.The total probability of a sample pixel 'x' in the test image when evaluated in the foreground GMM is

P_fg(x) = sum_over_j_1_to_k ( Wj_fg * Pj_fg( x ))
where 
k = number of clusters in foreground GMM
x = test sample
Pj_fg(x) = probability that sample x is in j-th  cluster according to the foreground GMM
Wj_fg = weight of the j-th cluster in foreground GMM
also, sum of all weights should be 1 for each GMM.`

We can do a similar calculation for the background GMM.

From looking at the EM code in opencv, it looks like the first part of the 2 values that EM returns is the log likelihood. For the foreground GMM this is

 log(P_fg(x_i))

I implemented your algorithm and for each pixel in the test image, I compared the log-likelihoods returned for each of the two GMM-s and classified the pixel with the GMM with higher value. I got decent results.

In that respect, yes this value is an indication of the pixel to be belonging to the entire GMM.

2) In my implementation of your problem, I always got the log likelihoods of all GMMS of all test-sample pixels under 0.

@CalumMurray I haven't discover any solution about that. I also post the question in stackoverflow http://stackoverflow.com/questions/14230377/opencv-output-of-the-predict-function-of-expectation-maximization But didn't get any success... If you find the solution, please post it, thank's.

Yes, you're right. Just I was confused that a dot product of matrices is a matrix multiplication which gives a matrix as output. And the dot product from openCV gives a scalar (as it treats matrices as vectors). But as OP is asking for cvDotProduct, there is no ambiguity.

For me, dot for matrices is matrix multiplication (I may wrong). Mat::dot is for vector in a Mat. Is the OP asking for a matrix multiplication or a dot between 2 vectors ?

I try to find the propability of a pixel from one image (ImageCurrent.tif, see at the end) to belongs of the foreground and background of another image (ImageFormer.tif).

Therefore, I create 2 GMMs (Gaussian Mixture Model) from one image with seperate foreground and background (using a mask). Then I used the predict function of the Expectation Maximization from OpenCV.

When I get back the logarithm likelihood of the predict function, I have sometimes values >0 which is for me strange as the probability should be between 0 and 1 (so the log likelihood should be <=0).

Is someone have experiment this kind of problem ? And how can I resolve it ?

Here is a test program showing the problem:

Mat mask = imread("mask.tiff", 0);
Mat formerImage = imread("ImageFormer.tiff");
Mat currentImage = imread("ImageCurrent.tiff");

// number of cluster in the GMM 
int nClusters = 5;

int countB=0, countF=0;

Vec3b color;

Vec2d probFg, probBg; // probabilities to belong to the foreground or background from GMMs

//count the number of pixels for each training data
for(int c=0; c<=40;c++) {
    for(int l=0; l<=40;l++) {
        if(mask.at<BYTE>(l, c)==255) {
            countF++;
        } else if(mask.at<BYTE>(l, c)==0) {
            countB++;
        }
    }
}
printf("countB %d countF %d \n", countB, countF);

Mat samplesForeground = Mat(countF,3, CV_64F);

Mat samplesBackground = Mat(countB,3, CV_64F);


// Expectation-Maximization able to resolve the GMM and to predict the probability for a pixel to belong to the GMM.
EM em_foreground= EM(nClusters);
EM em_background= EM(nClusters);

countB=0;
countF=0;

// fill the training data from the former image depending of the mask
for(int c=0; c<=40;c++) {
    for(int l=0; l<=40;l++) {
        if(mask.at<BYTE>(l, c)==255) {
            color = formerImage.at<Vec3b>(l, c);
            samplesForeground.at<double>(countF,0)=color[0];
            samplesForeground.at<double>(countF,1)=color[1];
            samplesForeground.at<double>(countF,2)=color[2];
            countF++;
        } else if(mask.at<BYTE>(l, c)==0) {
            color = formerImage.at<Vec3b>(l, c);
            samplesBackground.at<double>(countB, 0)=color[0];
            samplesBackground.at<double>(countB, 1)=color[1];
            samplesBackground.at<double>(countB, 2)=color[2];
            countB++;
        }
    }
}

printf("countB %d countF %d \n", countB, countF);
em_foreground.train(samplesForeground);
em_background.train(samplesBackground);

Mat sample(1, 3, CV_64F);

// try every pixel of the current image and get the log likelihood
for(int c=0; c<=40;c++) {
    for(int l=0; l<=40;l++) {
        color = currentImage.at<Vec3b>(l,c);
        sample.at<double>(0)=color[0];
        sample.at<double>(1)=color[1];
        sample.at<double>(2)=color[2];
        probFg=em_foreground.predict(sample);
        probBg=em_background.predict(sample);
        if(probFg[0]>0 || probBg[0]>0)
            printf("probFg[0] %f probBg[0] %f \n", probFg[0], probBg[0]);
    }
}

I tried the program with the followings images:

: http://postimage.org/image/eatsc71lj/

: http://postimage.org/image/sz4ona21j/

:http://postimage.org/image/xjquwak0v/

EDIT: Is this can be a bug from OpenCV ? As I'm not sure if it is normal or not, I don't know if I should open a ticket.