mAP (mean Average Precision) for Object Detection

mAP is the metric to measure the accuracy of object detectors. It is the average of the maximum precisions at different recall values.

Precision & recall

Precision measures how accurate is your predictions. i.e. the percentage of your positive predictions are correct. If you have a precision score of close to 1.0 then there is a high likelihood that whatever the classifier predicts as a positive detection is in fact a correct prediction.
Recall measures how good you find all the positives. For example, we can find 80% of the possible positive cases in our top K predictions. If you have a recall score close to 1.0 then almost all objects that are in your dataset will be positively detected by the model.
Here are their mathematical definitions:

IoU (Intersection over union)

IoU measures how much overlap between 2 regions, This measures how good is our prediction in the object detector with the ground truth.


AP (Average Precision)

AP calculation with precision-recall curve

Precision-Recall curve for an example detector.

The calculation if the AP score is to take the average value of the precision across all recall values (see explanation in section 4.2 of the Pascal Challenge paper).
From section 4.2 in paper

However the actual calculation is a little different.

precision-recall curve

We set the precision for recall r to the maximum precision obtained for any recall r' > r. Then we compute the AP as the area under this curve (shown in light blue) by numerical integration. No approximation is involved since the curve is piecewise constant.

Code for AP calculation

def compute_ap(recall, precision):
    """ Compute the average precision, given the recall and precision curves.
    Code originally from https://github.com/rbgirshick/py-faster-rcnn.

    # Arguments
        recall:    The recall curve (list).
        precision: The precision curve (list).
    # Returns
        The average precision as computed in py-faster-rcnn.
    """
    # correct AP calculation
    # first append sentinel values at the end
    mrec = np.concatenate(([0.], recall, [1.]))
    mpre = np.concatenate(([0.], precision, [0.]))

    # compute the precision envelope
    for i in range(mpre.size - 1, 0, -1):
        mpre[i - 1] = np.maximum(mpre[i - 1], mpre[i])

    # to calculate area under PR curve, look for points
    # where X axis (recall) changes value
    i = np.where(mrec[1:] != mrec[:-1])[0]

    # and sum (\Delta recall) * prec
    ap = np.sum((mrec[i + 1] - mrec[i]) * mpre[i + 1])
    return ap
import numpy as np

recall = [0.2,0.4,0.4,0.4,0.4,0.6,0.8,0.8,0.8,1.0]
precision = [1.0,1.0,0.67,0.5,0.4,0.5,0.57,0.5,0.44,0.5]
AP = compute_ap(recall, precision)
print('AP = {}'.format(AP))
# AP = 0.728
# If we calculate using 11 spaced recall values, 
# AP = (5 × 1.0 + 4 × 0.57 + 2 × 0.5)/11 = 0.753. 
# Check the this blog: mAP for Object Detection

# Intermediate results
def compute_ap(recall, precision):
    # correct AP calculation
    # first append sentinel values at the end
    mrec = np.concatenate(([0.], recall, [1.]))
    mpre = np.concatenate(([0.], precision, [0.]))
    # mrec = [0.  0.2 0.4 0.4 0.4 0.4 0.6 0.8 0.8 0.8 1.  1. ]
    # mpre = [0.   1.   1.   0.67 0.5  0.4  0.5  0.57 0.5  0.44 0.5  0.  ]
    
    # compute the precision envelope
    for i in range(mpre.size - 1, 0, -1):
        mpre[i - 1] = np.maximum(mpre[i - 1], mpre[i])
    # mpre = [1.   1.   1.   0.67 0.57 0.57 0.57 0.57 0.5  0.5  0.5  0.  ]

    # to calculate area under PR curve, look for points
    # where X axis (recall) changes value
    i = np.where(mrec[1:] != mrec[:-1])[0]
    # i = [0 1 5 6 9]
    # mrec[1:] = [0.2 0.4 0.4 0.4 0.4 0.6 0.8 0.8 0.8 1.  1. ]
    # mrec[:-1] = [0.  0.2 0.4 0.4 0.4 0.4 0.6 0.8 0.8 0.8 1. ]

    # and sum (\Delta recall) * prec
    ap = np.sum((mrec[i + 1] - mrec[i]) * mpre[i + 1])
    return ap     

Reference

mean Average Precision-github
mAP for Object Detection
Evaluation 12: mean average precision
Understanding the mAP Evaluation Metric for Object Detection
Discussion on fast.ai

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書(shū)系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

  • rljs by sennchi Timeline of History Part One The Cognitiv...
    sennchi閱讀 7,847評(píng)論 0 10
  • 汪峰和章子怡,每隔1小時(shí),會(huì)進(jìn)行一次視頻或者通話(huà)。 魯豫說(shuō),兩個(gè)人之間有時(shí)候這種查崗查哨吧,也是一種甜蜜的束縛。 ...
    百合的香氣閱讀 1,289評(píng)論 0 2
  • 空白的人生
    哈嘍小迷弟閱讀 350評(píng)論 1 0

友情鏈接更多精彩內(nèi)容