Evaluation Metrics for Computer Vision

Duration: 7 min

This module delves into the critical evaluation metrics used in computer vision tasks such as object detection, segmentation, and classification. Understanding these metrics is essential for assessing the performance of computer vision models and making informed decisions in model development and deployment.

Intersection over Union (IoU)

Intersection over Union (IoU) is a crucial metric for evaluating the accuracy of bounding boxes in object detection. It measures the overlap between the predicted bounding box and the ground truth bounding box. A higher IoU indicates better alignment between the predicted and actual object locations.

def calculate_iou(boxA, boxB):
    # Determine the coordinates of the intersection rectangle
    xA = max(boxA[0], boxB[0])
    yA = max(boxA[1], boxB[1])
    xB = min(boxA[2], boxB[2])
    yB = min(boxA[3], boxB[3])

    # Compute the area of intersection rectangle
    interArea = max(0, xB - xA + 1) * max(0, yB - yA + 1)

    # Compute the area of both the prediction and ground-truth rectangles
    boxAArea = (boxA[2] - boxA[0] + 1) * (boxA[3] - boxA[1] + 1)
    boxBArea = (boxB[2] - boxB[0] + 1) * (boxB[3] - boxB[1] + 1)

    # Compute the intersection over union by taking the intersection
    # area and dividing it by the sum of prediction + ground-truth
    # areas - the intersection area
    iou = interArea / float(boxAArea + boxBArea - interArea)

    return iou

# Example usage
boxA = [100, 100, 200, 200]
boxB = [150, 150, 250, 250]
print(calculate_iou(boxA, boxB))

Try it in Google Colab:

0.4444444444444444

Mean Average Precision (mAP)

Mean Average Precision (mAP) is a comprehensive metric used to evaluate object detection models. It considers both the precision and recall of the model across different Intersection over Union (IoU) thresholds. mAP provides a single scalar value that summarizes the model's performance, making it easier to compare different models.

from sklearn.metrics import average_precision_score

def calculate_map(gt, pred):
    # Flatten the ground truth and predictions
    gt_flat = [item for sublist in gt for item in sublist]
    pred_flat = [item for sublist in pred for item in sublist]

    # Calculate average precision for each class
    aps = [average_precision_score(gt_flat, pred_flat) for gt, pred in zip(gt, pred)]

    # Calculate mean average precision
    map_score = sum(aps) / len(aps)

    return map_score

# Example usage
ground_truth = [[1, 0, 1], [0, 1, 0]]
predictions = [[0.9, 0.1, 0.8], [0.2, 0.9, 0.3]]
print(calculate_map(ground_truth, predictions))

💡 Tip: When evaluating object detection models, ensure that the IoU threshold is appropriately set to match the specific requirements of your application.