Basically, we can calculate precision and recall easily. For example, we have total 1000 cases. We know that there are 100 cases which are positive. Then, you want system to predict the positive. For example, you get 200 positive cases in testing and then you record the ids of our predictions. After that, sum up how many times to get right and wrong. There are 4 ways to determine right of wrong:
1. True Negative (TN): case negative and system can predict as negative.
2. False Negative (FN): case positive but system predicted as negative.
3. False Positive (FP): case negative but system predicted as positive.
4. True Positive (TP): case positive and system can predict as positive.
Then, we found that got 80 true positives of 200 cases. Total cases (N) = 1000
TP = 80
FN = 20 -> (100-80)
FP = 120 -> (200-80)
TN = 780 -> ((1000-100)-(200-80)) -> (900-120)
Note that: TP + FN + FP + TN = N,
TP + FN = Number of positive labels,
TN + FP = Number of negative labels
Finally, we can do calculation.
Accuracy = (TP + TN) / N = (80 + 780) / 1000 = 0.86 => 86%
Recall = TP / (TP + FN) = 80 / (80 + 20) = 0.8 => 80%
Precision = TP / (TP + FP) = 80 / (80 + 120) = 0.4 => 40%
Are you sure this is correct, Veerapat? I think
ReplyDeleteFalse Positive: predicted as Positive, but actually negative(test labels).
False Negative: predicted as Negative, but actually positive(test labels).
Yeah, you're right. Based on An introduction to Information Retrieval (http://nlp.stanford.edu/IR-book/), I found the same words as you said. Thanks, I've confused for a long time (mistake for a year -_-").
ReplyDeleteI edited my post already. I think my calculation is still the same.
ReplyDelete