# Precision-recall evaluation

Is this

http://i39.tinypic.com/2n0cw2.jpg">

a possibile outcome for a precision-recall graph or you think i'm doing something wrong? (The blue line is relative to a one-step classifier, the other line refers to the same classifier with the difference that this time a second step is added in the classification process).

edit retag close merge delete

Sort by ยป oldest newest most voted

Your precision seems to improve slightly as the recall improves. Typically, the precision will go down as the recall improves. So, yes, there seems to be something suspect. And the precision seems to be very high to begin with while recall is still varying over a good portion albeit with very high value.

The ideal situation will be where precision and recall are both 1.0 and in that case, you will not have a graph like the one where recall varies with constant precision.

more

@unxnut Hi unxnut. Thank you for your comment. I was wondering if you could take a quick look to my confusion matrixes. You can find a screenshot of my spreadsheet here http://tinypic.com/r/qssdxw/5

( 2013-05-05 12:12:33 -0500 )edit

I looked at your numbers but cannot tell how you are computing precision and recall. Could you tell me how you did it so that I can see if there is any issue in your definition of those two terms.

( 2013-05-05 12:58:18 -0500 )edit

@unxnut: Precision = TP/(TP+FP), Recall = TP/(TP+FN)

( 2013-05-05 13:07:20 -0500 )edit

Recall definition is not correct. I assume TP is the "Total Positives". Recall will be TP + Missed Positives. In other words, the total number of positives in the system.

( 2013-05-05 13:27:19 -0500 )edit

I'm actually a little bit confused. Could you elaborate more or make an example? If i'm following your reasoning, you mean Recall = # True Positives / # Total Positives, but total should indeed be equal to (#True Positives + #False Negatives), which corresponds to my definition from earlier.

( 2013-05-05 13:47:32 -0500 )edit

You are right. The double negatives confused me :-(

( 2013-05-05 13:51:12 -0500 )edit

I just went back and looked at your image again. I eyeballed a few values and they seem correct. Now, the next issue will be how you have plotted those values in your original image (the one on this page). Could you elaborate on that image?

( 2013-05-05 13:56:49 -0500 )edit

For the "apple" classifier (a 1-vs-all classifier) i take precision and recall for different k values. I then plot them like in http://i40.tinypic.com/3145rme.jpg

( 2013-05-05 14:05:23 -0500 )edit

Define k. Is this the number of instances?

( 2013-05-05 14:34:57 -0500 )edit

Number of clusters (first step is a Bag-of-words classification approach)

( 2013-05-05 15:07:16 -0500 )edit

I think there is something strange: you are plotting recall and precision for different kmeans parameter, but, this parameters doesn't seem related to your precision.

I think you should plot your decision parameter (the one used in confusion matrix), eg. if you want to recognize an apple, and every time you "see" something you say: "this is an apple", your recall is 100% but your precision should be quite low (depending on how many apples you have in your test).

A contrario, if you almost never said "this is an apple", you probably have a good precision, but a low recall. T

he idea of this graph, as I understand it, is to show "how good is your estimation". If I use the analogy of biometric:

How many bad guys are entering in my house if I want to be sure that all good guys are able to enter?

And help the "user" to find a good compromise... This is the basic goal of ROC curves, see the Wikipedia page here.

more

Official site

GitHub

Wiki

Documentation