# Expectation Maximization: logarithm likelihood > 0

I try to find the propability of a pixel from one image (ImageCurrent.tif, see at the end) to belongs of the foreground and background of another image (ImageFormer.tif).

Therefore, I create 2 GMMs (Gaussian Mixture Model) from one image with seperate foreground and background (using a mask). Then I used the predict function of the Expectation Maximization from OpenCV.

When I get back the logarithm likelihood of the predict function, I have sometimes values >0 which is for me strange as the probability should be between 0 and 1 (so the log likelihood should be <=0).

Is someone have experiment this kind of problem ? And how can I resolve it ?

Here is a test program showing the problem:

Mat mask = imread("mask.tiff", 0);

// number of cluster in the GMM
int nClusters = 5;

int countB=0, countF=0;

Vec3b color;

Vec2d probFg, probBg; // probabilities to belong to the foreground or background from GMMs

//count the number of pixels for each training data
for(int c=0; c<=40;c++) {
for(int l=0; l<=40;l++) {
countF++;
countB++;
}
}
}
printf("countB %d countF %d \n", countB, countF);

Mat samplesForeground = Mat(countF,3, CV_64F);

Mat samplesBackground = Mat(countB,3, CV_64F);

// Expectation-Maximization able to resolve the GMM and to predict the probability for a pixel to belong to the GMM.
EM em_foreground= EM(nClusters);
EM em_background= EM(nClusters);

countB=0;
countF=0;

// fill the training data from the former image depending of the mask
for(int c=0; c<=40;c++) {
for(int l=0; l<=40;l++) {
color = formerImage.at<Vec3b>(l, c);
samplesForeground.at<double>(countF,0)=color[0];
samplesForeground.at<double>(countF,1)=color[1];
samplesForeground.at<double>(countF,2)=color[2];
countF++;
color = formerImage.at<Vec3b>(l, c);
samplesBackground.at<double>(countB, 0)=color[0];
samplesBackground.at<double>(countB, 1)=color[1];
samplesBackground.at<double>(countB, 2)=color[2];
countB++;
}
}
}

printf("countB %d countF %d \n", countB, countF);
em_foreground.train(samplesForeground);
em_background.train(samplesBackground);

Mat sample(1, 3, CV_64F);

// try every pixel of the current image and get the log likelihood
for(int c=0; c<=40;c++) {
for(int l=0; l<=40;l++) {
color = currentImage.at<Vec3b>(l,c);
sample.at<double>(0)=color[0];
sample.at<double>(1)=color[1];
sample.at<double>(2)=color[2];
probFg=em_foreground.predict(sample);
probBg=em_background.predict(sample);
if(probFg[0]>0 || probBg[0]>0)
printf("probFg[0] %f probBg[0] %f \n", probFg[0], probBg[0]);
}
}


I tried the program with the followings images:

EDIT: Is this can be a bug from OpenCV ? As I'm not sure if it is normal or not, I don't know if I should open a ticket.

edit retag close merge delete

Did you ever discover the solution to this? I have the identical problem.

( 2014-02-18 09:28:21 -0500 )edit

@CalumMurray I haven't discover any solution about that. I also post the question in stackoverflow http://stackoverflow.com/questions/14230377/opencv-output-of-the-predict-function-of-expectation-maximization But didn't get any success... If you find the solution, please post it, thank's.

( 2014-09-22 02:38:12 -0500 )edit

Sort by » oldest newest most voted

I have asked this question on stack overflow and has been answered : http://stackoverflow.com/questions/14...

Here is the copy of the answer given in stackoverflow:

As I understand you have two separate GMMs for the foreground and background part of the image.The total probability of a sample pixel 'x' in the test image when evaluated in the foreground GMM is

P_fg(x) = sum_over_j_1_to_k ( Wj_fg * Pj_fg( x ))
where
k = number of clusters in foreground GMM
x = test sample
Pj_fg(x) = probability that sample x is in j-th  cluster according to the foreground GMM
Wj_fg = weight of the j-th cluster in foreground GMM
also, sum of all weights should be 1 for each GMM.


We can do a similar calculation for the background GMM.

From looking at the EM code in opencv, it looks like the first part of the 2 values that EM returns is the log likelihood. For the foreground GMM this is

 log(P_fg(x_i))`

I implemented your algorithm and for each pixel in the test image, I compared the log-likelihoods returned for each of the two GMM-s and classified the pixel with the GMM with higher value. I got decent results.

In that respect, yes this value is an indication of the pixel to be belonging to the entire GMM.

2) In my implementation of your problem, I always got the log likelihoods of all GMMS of all test-sample pixels under 0.

more

Maybe you could also summarize the Stack Overflow answer here to avoid switching on another page and in case the link is no more accessible?

( 2016-06-17 11:09:49 -0500 )edit

Official site

GitHub

Wiki

Documentation