Mean Average Precision

Install the metrics extra before using this API:

pip install "supervision[metrics]"

MeanAveragePrecision scores detection models on sv.Detections (boxes, masks or oriented boxes) matched by IoU. KeyPointMeanAveragePrecision scores pose estimation models on sv.KeyPoints matched by Object Keypoint Similarity (OKS); see Pose estimation. Both follow COCO evaluation and return results with the same map50_95, map50, map75, plot and to_pandas interface.

Evaluate targets from a PyTorch DataLoader

A DataLoader can return any annotation format defined by its dataset and collate function. Convert each image's annotations to a separate sv.Detections object, then pair it with that image's predictions. Ground truth needs xyxy boxes and integer class_id values; only predictions need confidence scores.

For Hugging Face DETR, Detections.from_transformers accepts post-processed predictions with boxes, labels, and scores. Training targets instead have boxes and class_labels. With annotation conversion enabled, their boxes use normalized (center_x, center_y, width, height) coordinates. Convert them with sv.xcycwh_to_xyxy and scale to the same image dimensions used to post-process predictions. sv.xywh_to_xyxy is for boxes whose first two coordinates describe the top-left corner, so it is not the right converter here.

Convert normalized targets

The following helper converts image-normalized boxes into pixel coordinates. image_size_hw is (height, width), while the scale applied to boxes is (width, height, width, height).

import numpy as np
import numpy.typing as npt
import supervision as sv


def normalized_targets_to_detections(
    boxes: npt.NDArray[np.float32],
    class_ids: npt.NDArray[np.int64],
    image_size_hw: tuple[int, int],
) -> sv.Detections:
    """Convert normalized center-based targets to pixel corner coordinates."""
    height, width = image_size_hw
    xyxy = sv.xcycwh_to_xyxy(boxes)
    xyxy *= np.array([width, height, width, height], dtype=xyxy.dtype)
    return sv.Detections(xyxy=xyxy, class_id=class_ids)


target = normalized_targets_to_detections(
    boxes=np.array([[0.5, 0.5, 0.25, 0.5]], dtype=np.float32),
    class_ids=np.array([0], dtype=np.int64),
    image_size_hw=(480, 640),
)
print(target.xyxy)
# [[240. 120. 400. 360.]]

An image without objects should still have one target entry: pass boxes with shape (0, 4) and class IDs with shape (0,). The helper preserves these shapes and does not modify the input arrays.

Run evaluation

This example assumes you already have a DETR model, its image_processor, and TEST_DATALOADER. Each batch contains pixel_values, pixel_mask, and a list of per-image labels dictionaries. classes lists class names in class-ID order, read here from the model's id2label mapping; both predictions and targets must use IDs from 0 to len(classes) - 1. Remap both sides first if your dataset uses sparse category IDs.

Coordinate assumptions: the targets below are normalized relative to each image before batch padding, and evaluation preprocessing only resizes images. Under these assumptions, orig_size restores original-image pixel coordinates, as in the Hugging Face evaluation example. Check your processor and collate function: processing multiple images together can update annotations to the padded canvas. Such targets need padding and resize transforms reversed before using orig_size; multiplying by orig_size alone does not undo padding. Likewise, crops or other geometric augmentations require their transforms to be accounted for. Keep validation preprocessing deterministic.

import torch
from supervision.metrics import MeanAveragePrecision

device = next(model.parameters()).device
classes = [
    model.config.id2label[class_id] for class_id in sorted(model.config.id2label)
]
model.eval()
predictions: list[sv.Detections] = []
targets: list[sv.Detections] = []

with torch.no_grad():
    for batch in TEST_DATALOADER:
        labels = batch["labels"]
        outputs = model(
            pixel_values=batch["pixel_values"].to(device),
            pixel_mask=batch["pixel_mask"].to(device),
        )
        target_sizes = torch.stack([label["orig_size"] for label in labels]).to(device)
        results = image_processor.post_process_object_detection(
            outputs,
            target_sizes=target_sizes,
            threshold=0.0,
        )

        for result, label in zip(results, labels, strict=True):
            height, width = label["orig_size"].detach().cpu().tolist()
            targets.append(
                normalized_targets_to_detections(
                    boxes=label["boxes"].detach().cpu().numpy().astype(np.float32),
                    class_ids=label["class_labels"]
                    .detach()
                    .cpu()
                    .numpy()
                    .astype(np.int64),
                    image_size_hw=(height, width),
                )
            )
            predictions.append(sv.Detections.from_transformers(result))

map_result = MeanAveragePrecision().update(predictions, targets).compute()
print(map_result.map50_95)

confusion_matrix = sv.ConfusionMatrix.from_detections(
    predictions=predictions,
    targets=targets,
    classes=classes,
    conf_threshold=0.5,
)
print(confusion_matrix.matrix)

Keep low-confidence predictions when computing mAP: filtering at 0.5 before evaluation discards part of the precision-recall curve. The confusion matrix uses its own confidence threshold. Iterate over the complete loader, preserving one prediction/target pair per image even when either side is empty.

Dense scenes

Detection AP uses COCO's default prediction limits (1, 10, 100). The highest limit caps the highest-confidence predictions per image and category, after any class mapping or class-agnostic merging. Ground truths are not capped. A perfect prediction of 1,000 objects of one class therefore reaches about 0.109 AP@100. The same objects spread evenly over ten categories can reach 1.0.

Use an explicit larger limit when evaluating dense scenes:

from supervision.metrics import MeanAveragePrecision

metric = MeanAveragePrecision(max_detection_thresholds=(1, 10, 1000))
result = metric.update(predictions, targets).compute()
print(result.max_detection_thresholds)  # (1, 10, 1000)

The sequence must contain exactly three strictly increasing positive integers. AP uses its largest value; the smaller values preserve the evaluator's COCO limit structure. The default remains unchanged. Higher limits also increase matching work and memory use, especially for masks.

Custom-limit AP is a different metric from standard COCO AP@100: report the cap when comparing results. Text summaries and custom-limit plot titles display it; results, including each object-size result, retain max_detection_thresholds. result.to_pandas().attrs["max_detection_thresholds"] retains the limits in memory without changing the score columns. CSV export does not retain DataFrame attributes, so store the configuration alongside exported scores.

Pose estimation

KeyPointMeanAveragePrecision scores pose estimation models the way COCO keypoint evaluation does: predictions are matched to targets by Object Keypoint Similarity (OKS) instead of IoU. Pass one sv.KeyPoints per image for predictions and targets. COCO visibility v=1 (occluded) and v=2 (visible) both map to KeyPoints.visible=True and v=0 to False, since pycocotools counts every keypoint with v>0. To reproduce COCO numbers, store each target's annotated area in targets.data["area"]; otherwise the area of the box spanning its visible keypoints is used. Targets with no visible keypoint are COCO ignore regions only when their boxes are given as targets.data["xyxy"] in (x_min, y_min, x_max, y_max); without boxes they are skipped, so a prediction on one counts as a false positive, and a class whose only targets are skipped scores -1. A prediction box in predictions.data["xyxy"] sets its object-size bucket, as the bbox of a COCO result entry does in pycocotools. Mark COCO crowd targets with targets.data["iscrowd"]; like pycocotools, they are ignore regions that any number of predictions may match. Predictions are ranked by detection_confidence, not keypoint confidence. Non-finite (NaN or infinite) coordinates are not rejected: a non-finite target keypoint counts as unlabelled, like visible=False, and a non-finite prediction keypoint adds nothing to OKS while its finite keypoints still count; pycocotools instead returns a NaN OKS when a labelled keypoint of the pair is NaN. A target area must be finite and non-negative; a zero area is accepted but leaves only exact-match keypoints counting. Clean such labels first.

import supervision as sv
from supervision.metrics import KeyPointMeanAveragePrecision

metric = KeyPointMeanAveragePrecision()  # COCO sigmas for 17-point skeletons
for predictions, targets in zip(PREDICTIONS, TARGETS):
    metric.update(predictions, targets)

result = metric.compute()
print(result.map50_95)

API reference

MeanAveragePrecision

supervision.metrics.mean_average_precision.MeanAveragePrecision

Bases: Metric[MeanAveragePrecisionResult]

Mean Average Precision (mAP) is a metric used to evaluate object detection models. It is the average of the precision-recall curves at different IoU thresholds.

Examples:

>>> import numpy as np
>>> import supervision as sv
>>> from supervision.metrics import MeanAveragePrecision
>>> predictions = sv.Detections(
...     xyxy=np.array([[0, 0, 10, 10]]),
...     class_id=np.array([0]),
...     confidence=np.array([0.9])
... )
>>> targets = sv.Detections(
...     xyxy=np.array([[0, 0, 10, 10]]),
...     class_id=np.array([0])
... )
>>> map_metric = MeanAveragePrecision()
>>> map_result = map_metric.update(predictions, targets).compute()
>>> round(float(map_result.map50), 2)
1.0
>>> print(map_result)
Average Precision (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 1.000
Average Precision (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 1.000
Average Precision (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

Source code in src/supervision/metrics/mean_average_precision.py

class MeanAveragePrecision(Metric[MeanAveragePrecisionResult]):
    """Mean Average Precision (mAP) is a metric used to evaluate object detection
    models. It is the average of the precision-recall curves at different IoU
    thresholds.

    Examples:
        ```pycon
        >>> import numpy as np
        >>> import supervision as sv
        >>> from supervision.metrics import MeanAveragePrecision
        >>> predictions = sv.Detections(
        ...     xyxy=np.array([[0, 0, 10, 10]]),
        ...     class_id=np.array([0]),
        ...     confidence=np.array([0.9])
        ... )
        >>> targets = sv.Detections(
        ...     xyxy=np.array([[0, 0, 10, 10]]),
        ...     class_id=np.array([0])
        ... )
        >>> map_metric = MeanAveragePrecision()
        >>> map_result = map_metric.update(predictions, targets).compute()
        >>> round(float(map_result.map50), 2)
        1.0
        >>> print(map_result)
        Average Precision (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 1.000
        Average Precision (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 1.000
        Average Precision (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 1.000
        Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 1.000
        Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
        Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

        ```

    ![example_plot](
        https://media.roboflow.com/supervision-docs/metrics/mAP_plot_example.png
    ){ align=center width="800" }
    """

    def __init__(
        self,
        metric_target: MetricTarget = MetricTarget.BOXES,
        class_agnostic: bool = False,
        class_mapping: dict[int, int] | None = None,
        image_indices: list[int] | None = None,
        *,
        max_detection_thresholds: Sequence[int] = (1, 10, 100),
    ) -> None:
        """Initialize the Mean Average Precision metric.

        Args:
            metric_target: The type of detection data to use.
            class_agnostic: Whether to treat all data as a single class with ID `-1`.
                Detections without class IDs join that class when any input has
                class IDs, and otherwise keep the default class `0`.
            class_mapping: A dictionary to map class IDs to new IDs. With
                `class_agnostic=True` and any input that has class IDs, it must
                contain key `-1`, which maps the single merged class. It is not
                used when no input has class IDs.
            image_indices: The indices of the images to use.
            max_detection_thresholds: Exactly three strictly increasing positive
                integer limits, defaulting to COCO's `(1, 10, 100)`. AP uses the
                largest limit: the highest-confidence predictions per image and
                category after class mapping or class-agnostic merging. Ground
                truths are not capped. For dense scenes, use `(1, 10, 1000)`.
                Results using different limits are different metrics and should
                not be compared as standard COCO AP@100.

        Raises:
            ValueError: If `max_detection_thresholds` is not a sequence of three
                strictly increasing positive integers.

        Example:
            ```python
            from supervision.metrics import MeanAveragePrecision

            metric = MeanAveragePrecision(max_detection_thresholds=(1, 10, 1000))
            ```
        """
        thresholds = tuple(max_detection_thresholds)
        if (
            len(thresholds) != 3
            or any(type(limit) is not int or limit <= 0 for limit in thresholds)
            or not thresholds[0] < thresholds[1] < thresholds[2]
        ):
            raise ValueError(
                "max_detection_thresholds must contain exactly three strictly "
                "increasing positive integers."
            )
        self._max_detection_thresholds = thresholds
        self._metric_target = metric_target
        self._class_agnostic = class_agnostic

        self._predictions_list: list[Detections] = []
        self._targets_list: list[Detections] = []
        self._class_mapping = class_mapping
        self._image_indices = image_indices

    def reset(self) -> None:
        """Reset the metric to its initial state, clearing all stored data."""
        self._predictions_list = []
        self._targets_list = []

    def update(
        self,
        predictions: Detections | list[Detections],
        targets: Detections | list[Detections],
    ) -> MeanAveragePrecision:
        """Add new predictions and targets to the metric, but do not compute the result.

        Args:
            predictions: The predicted detections.
            targets: The ground-truth detections.

        Returns:
            The updated metric instance.
        """
        if not isinstance(predictions, list):
            predictions = [predictions]
        if not isinstance(targets, list):
            targets = [targets]

        if len(predictions) != len(targets):
            raise ValueError(
                f"The number of predictions ({len(predictions)}) and"
                f" targets ({len(targets)}) during the update must be the same."
            )

        if self._class_agnostic:
            # Relabel private copies so caller-owned detections stay as passed in.
            predictions = deepcopy(predictions)
            targets = deepcopy(targets)

            for prediction in predictions:
                if prediction.class_id is not None:
                    prediction.class_id = _agnostic_class_ids(len(prediction))
            for target in targets:
                if target.class_id is not None:
                    target.class_id = _agnostic_class_ids(len(target))

        self._predictions_list.extend(predictions)
        self._targets_list.extend(targets)

        return self

    def _detections_content(self, detections: Detections) -> npt.NDArray[Any] | None:
        """Return per-detection masks or oriented boxes for the metric target, or `None`
        for the box target and for empty detections.

        Preserve oriented-box precision so IoU can translate large coordinates before
        converting them to its local float32 geometry.
        """
        if self._metric_target == MetricTarget.BOXES or len(detections) == 0:
            return None
        if self._metric_target == MetricTarget.MASKS:
            if detections.mask is None:
                raise ValueError(
                    "MeanAveragePrecision with `MetricTarget.MASKS` requires"
                    " masks on both predictions and targets."
                )
            return np.asarray(detections.mask).astype(bool)
        if self._metric_target == MetricTarget.ORIENTED_BOUNDING_BOXES:
            obb = detections.data.get(ORIENTED_BOX_COORDINATES)
            if obb is None:
                raise ValueError(
                    "MeanAveragePrecision with"
                    " `MetricTarget.ORIENTED_BOUNDING_BOXES` requires"
                    f" `{ORIENTED_BOX_COORDINATES}` in `data` on both"
                    " predictions and targets."
                )
            return np.asarray(obb).reshape(-1, 4, 2)
        raise ValueError(f"Invalid metric target: {self._metric_target}")

    def _content_area(
        self, xywh: list[float], content: npt.NDArray[Any] | None, idx: int
    ) -> float:
        """Compute the default annotation area for the metric target: bbox area for
        boxes, pixel count for masks, polygon area for oriented boxes."""
        if content is None:
            return float(xywh[2] * xywh[3])
        if self._metric_target == MetricTarget.MASKS:
            return float(np.count_nonzero(content[idx]))
        return float(_polygon_areas(content[idx : idx + 1])[0])

    def _prepare_targets(
        self, targets: list[Detections]
    ) -> dict[str, list[_TypeCocoDict]]:
        """Transform targets into a dictionary that can be used by the COCO
        evaluator."""
        images: list[_TypeCocoDict] = [{"id": img_id} for img_id in range(len(targets))]
        if self._image_indices is not None:
            images = [{"id": self._image_indices[img["id"]]} for img in images]
        # Annotations list
        annotations: list[_TypeCocoDict] = []
        for image_id, image_targets in enumerate(targets):
            if self._image_indices is not None:
                image_id = self._image_indices[image_id]

            # Ensure xyxy is not None
            if image_targets.xyxy is None:
                continue

            content = self._detections_content(image_targets)
            for target_idx, xyxy in enumerate(image_targets.xyxy):
                xywh = [xyxy[0], xyxy[1], xyxy[2] - xyxy[0], xyxy[3] - xyxy[1]]

                # Default values
                category_id = 0

                if image_targets.class_id is not None:
                    cls_id = image_targets.class_id[target_idx]
                    if self._class_mapping is not None:
                        category_id = self._class_mapping[int(cls_id)]
                    else:
                        category_id = int(cls_id)

                # Use area from data if available, otherwise calculate from the
                # metric target content (bbox, mask or oriented box)
                area = None
                if (
                    image_targets.data is not None
                    and AREA_DATA_FIELD in image_targets.data
                ):
                    area_data: npt.NDArray[np.float32] = np.asarray(
                        image_targets.data[AREA_DATA_FIELD], dtype=np.float32
                    )
                    area = float(area_data[target_idx])

                if area is None:
                    area = self._content_area(xywh, content, target_idx)

                iscrowd = 0
                if (
                    image_targets.data is not None
                    and ISCROWD_DATA_FIELD in image_targets.data
                ):
                    iscrowd_data: npt.NDArray[np.int64] = np.asarray(
                        image_targets.data[ISCROWD_DATA_FIELD], dtype=np.int64
                    )
                    iscrowd = int(iscrowd_data[target_idx])

                ignore = 0
                if image_targets.data is not None and "ignore" in image_targets.data:
                    ignore_data: npt.NDArray[np.int64] = np.asarray(
                        image_targets.data["ignore"], dtype=np.int64
                    )
                    ignore = int(ignore_data[target_idx])

                dict_annotation: _TypeCocoDict = {
                    "area": area,
                    "iscrowd": iscrowd,
                    "image_id": image_id,
                    "bbox": xywh,
                    "category_id": category_id,
                    "id": len(annotations) + 1,  # Start IDs from 1 (0 means no match)
                    "ignore": ignore,
                }
                if content is not None:
                    dict_annotation["content"] = content[target_idx]
                annotations.append(dict_annotation)
        # Category list
        all_cat_ids = {annotation["category_id"] for annotation in annotations}
        categories: list[_TypeCocoDict] = [{"id": cat_id} for cat_id in all_cat_ids]
        # Create coco dictionary
        return {
            "images": images,
            "annotations": annotations,
            "categories": categories,
        }

    def _prepare_predictions(
        self, predictions: list[Detections]
    ) -> list[_TypeCocoDict]:
        """Transform predictions into a list of predictions that can be used by the COCO
        evaluator."""
        coco_predictions: list[_TypeCocoDict] = []
        for image_id, image_predictions in enumerate(predictions):
            if self._image_indices is not None:
                image_id = self._image_indices[image_id]

            if image_predictions.xyxy is None:
                continue

            content = self._detections_content(image_predictions)
            for pred_idx, xyxy in enumerate(image_predictions.xyxy):
                xywh = [xyxy[0], xyxy[1], xyxy[2] - xyxy[0], xyxy[3] - xyxy[1]]

                category_id = 0
                score = 0.0

                if image_predictions.class_id is not None:
                    cls_id = image_predictions.class_id[pred_idx]
                    if self._class_mapping is not None:
                        category_id = self._class_mapping[int(cls_id)]
                    else:
                        category_id = int(cls_id)

                if image_predictions.confidence is not None:
                    score = float(image_predictions.confidence[pred_idx])

                # Use area from data if available, otherwise calculate from the
                # metric target content (bbox, mask or oriented box)
                area = None
                if (
                    image_predictions.data is not None
                    and AREA_DATA_FIELD in image_predictions.data
                ):
                    area_data: npt.NDArray[np.float32] = np.asarray(
                        image_predictions.data[AREA_DATA_FIELD], dtype=np.float32
                    )
                    area = float(area_data[pred_idx])

                if area is None:
                    area = self._content_area(xywh, content, pred_idx)

                dict_prediction: _TypeCocoDict = {
                    "image_id": image_id,
                    "bbox": xywh,
                    "score": score,
                    "category_id": category_id,
                    "area": area,
                    "id": len(coco_predictions) + 1,
                }
                if content is not None:
                    dict_prediction["content"] = content[pred_idx]
                coco_predictions.append(dict_prediction)
        return coco_predictions

    def compute(self) -> MeanAveragePrecisionResult:
        """
        Calculate Mean Average Precision based on predicted and ground-truth
        detections at different thresholds using the COCO evaluation metrics.
        Source: https://github.com/rafaelpadilla/review_object_detection_metrics

        Returns:
            The Mean Average Precision result.
        """
        total_images_predictions = len(self._predictions_list)
        total_images_targets = len(self._targets_list)

        if total_images_predictions != total_images_targets:
            raise ValueError(
                f"The number of predictions ({total_images_predictions}) and"
                f" targets ({total_images_targets}) during the evaluation must be"
                " the same."
            )

        predictions_list = self._predictions_list
        targets_list = self._targets_list
        if self._class_agnostic:
            detections = predictions_list + targets_list
            # All-unlabeled inputs keep their default category; mixed inputs
            # must join the class assigned to labeled detections in update.
            # Relabeled shallow copies keep the stored detections unmodified.
            if any(d.class_id is not None and len(d) > 0 for d in detections):
                predictions_list = [
                    _with_agnostic_class_id(d) for d in predictions_list
                ]
                targets_list = [_with_agnostic_class_id(d) for d in targets_list]

        dict_targets = self._prepare_targets(targets_list)
        lst_predictions = self._prepare_predictions(predictions_list)
        # Create a coco object with the targets
        coco_gt = EvaluationDataset(targets=dict_targets)
        # Include the predictions to coco object
        coco_det = coco_gt.load_predictions(lst_predictions)
        # Create a coco evaluator with the predictions
        cocoEval = COCOEvaluator(coco_gt, coco_det, metric_target=self._metric_target)

        cocoEval.params.max_dets = list(self._max_detection_thresholds)

        # Evaluate on all images
        cocoEval.evaluate()

        # Create MeanAveragePrecisionResult object for small objects
        mAP_small = MeanAveragePrecisionResult(
            metric_target=self._metric_target,
            is_class_agnostic=self._class_agnostic,
            max_detection_thresholds=self._max_detection_thresholds,
            mAP_scores=np.asarray(
                cocoEval.results["mAP_scores_small"], dtype=np.float64
            ),
            ap_per_class=np.asarray(
                cocoEval.results["ap_per_class_small"], dtype=np.float64
            ),
            iou_thresholds=np.asarray(cocoEval.params.iou_thrs, dtype=np.float64),
            matched_classes=np.asarray(cocoEval.params.cat_ids, dtype=np.int32),
        )
        # Create MeanAveragePrecisionResult object for medium objects
        mAP_medium = MeanAveragePrecisionResult(
            metric_target=self._metric_target,
            is_class_agnostic=self._class_agnostic,
            max_detection_thresholds=self._max_detection_thresholds,
            mAP_scores=np.asarray(
                cocoEval.results["mAP_scores_medium"], dtype=np.float64
            ),
            ap_per_class=np.asarray(
                cocoEval.results["ap_per_class_medium"], dtype=np.float64
            ),
            iou_thresholds=np.asarray(cocoEval.params.iou_thrs, dtype=np.float64),
            matched_classes=np.asarray(cocoEval.params.cat_ids, dtype=np.int32),
        )
        # Create MeanAveragePrecisionResult object for large objects
        mAP_large = MeanAveragePrecisionResult(
            metric_target=self._metric_target,
            is_class_agnostic=self._class_agnostic,
            max_detection_thresholds=self._max_detection_thresholds,
            mAP_scores=np.asarray(
                cocoEval.results["mAP_scores_large"], dtype=np.float64
            ),
            ap_per_class=np.asarray(
                cocoEval.results["ap_per_class_large"], dtype=np.float64
            ),
            iou_thresholds=np.asarray(cocoEval.params.iou_thrs, dtype=np.float64),
            matched_classes=np.asarray(cocoEval.params.cat_ids, dtype=np.int32),
        )

        # Create the final MeanAveragePrecisionResult object
        mAP_result = MeanAveragePrecisionResult(
            metric_target=self._metric_target,
            is_class_agnostic=self._class_agnostic,
            max_detection_thresholds=self._max_detection_thresholds,
            mAP_scores=np.asarray(
                cocoEval.results["mAP_scores_all_sizes"], dtype=np.float64
            ),
            ap_per_class=np.asarray(
                cocoEval.results["ap_per_class_all_sizes"], dtype=np.float64
            ),
            iou_thresholds=np.asarray(cocoEval.params.iou_thrs, dtype=np.float64),
            matched_classes=np.asarray(cocoEval.params.cat_ids, dtype=np.int32),
            small_objects=mAP_small,
            medium_objects=mAP_medium,
            large_objects=mAP_large,
        )
        return mAP_result

Methods:

__init__(metric_target: MetricTarget = MetricTarget.BOXES, class_agnostic: bool = False, class_mapping: dict[int, int] | None = None, image_indices: list[int] | None = None, *, max_detection_thresholds: Sequence[int] = (1, 10, 100)) -> None

Initialize the Mean Average Precision metric.

Parameters:

Name	Type	Description	Default
metric_target	MetricTarget	The type of detection data to use.	BOXES
class_agnostic	bool	Whether to treat all data as a single class with ID -1. Detections without class IDs join that class when any input has class IDs, and otherwise keep the default class 0.	False
class_mapping	dict[int, int] | None	A dictionary to map class IDs to new IDs. With class_agnostic=True and any input that has class IDs, it must contain key -1, which maps the single merged class. It is not used when no input has class IDs.	None
image_indices	list[int] | None	The indices of the images to use.	None
max_detection_thresholds	Sequence[int]	Exactly three strictly increasing positive integer limits, defaulting to COCO's (1, 10, 100). AP uses the largest limit: the highest-confidence predictions per image and category after class mapping or class-agnostic merging. Ground truths are not capped. For dense scenes, use (1, 10, 1000). Results using different limits are different metrics and should not be compared as standard COCO AP@100.	(1, 10, 100)

Raises:

Type	Description
ValueError	If max_detection_thresholds is not a sequence of three strictly increasing positive integers.

Example

from supervision.metrics import MeanAveragePrecision

metric = MeanAveragePrecision(max_detection_thresholds=(1, 10, 1000))

Source code in src/supervision/metrics/mean_average_precision.py

def __init__(
    self,
    metric_target: MetricTarget = MetricTarget.BOXES,
    class_agnostic: bool = False,
    class_mapping: dict[int, int] | None = None,
    image_indices: list[int] | None = None,
    *,
    max_detection_thresholds: Sequence[int] = (1, 10, 100),
) -> None:
    """Initialize the Mean Average Precision metric.

    Args:
        metric_target: The type of detection data to use.
        class_agnostic: Whether to treat all data as a single class with ID `-1`.
            Detections without class IDs join that class when any input has
            class IDs, and otherwise keep the default class `0`.
        class_mapping: A dictionary to map class IDs to new IDs. With
            `class_agnostic=True` and any input that has class IDs, it must
            contain key `-1`, which maps the single merged class. It is not
            used when no input has class IDs.
        image_indices: The indices of the images to use.
        max_detection_thresholds: Exactly three strictly increasing positive
            integer limits, defaulting to COCO's `(1, 10, 100)`. AP uses the
            largest limit: the highest-confidence predictions per image and
            category after class mapping or class-agnostic merging. Ground
            truths are not capped. For dense scenes, use `(1, 10, 1000)`.
            Results using different limits are different metrics and should
            not be compared as standard COCO AP@100.

    Raises:
        ValueError: If `max_detection_thresholds` is not a sequence of three
            strictly increasing positive integers.

    Example:
        ```python
        from supervision.metrics import MeanAveragePrecision

        metric = MeanAveragePrecision(max_detection_thresholds=(1, 10, 1000))
        ```
    """
    thresholds = tuple(max_detection_thresholds)
    if (
        len(thresholds) != 3
        or any(type(limit) is not int or limit <= 0 for limit in thresholds)
        or not thresholds[0] < thresholds[1] < thresholds[2]
    ):
        raise ValueError(
            "max_detection_thresholds must contain exactly three strictly "
            "increasing positive integers."
        )
    self._max_detection_thresholds = thresholds
    self._metric_target = metric_target
    self._class_agnostic = class_agnostic

    self._predictions_list: list[Detections] = []
    self._targets_list: list[Detections] = []
    self._class_mapping = class_mapping
    self._image_indices = image_indices

compute() -> MeanAveragePrecisionResult

Calculate Mean Average Precision based on predicted and ground-truth detections at different thresholds using the COCO evaluation metrics. Source: https://github.com/rafaelpadilla/review_object_detection_metrics

Returns:

Type	Description
MeanAveragePrecisionResult	The Mean Average Precision result.

Source code in src/supervision/metrics/mean_average_precision.py

def compute(self) -> MeanAveragePrecisionResult:
    """
    Calculate Mean Average Precision based on predicted and ground-truth
    detections at different thresholds using the COCO evaluation metrics.
    Source: https://github.com/rafaelpadilla/review_object_detection_metrics

    Returns:
        The Mean Average Precision result.
    """
    total_images_predictions = len(self._predictions_list)
    total_images_targets = len(self._targets_list)

    if total_images_predictions != total_images_targets:
        raise ValueError(
            f"The number of predictions ({total_images_predictions}) and"
            f" targets ({total_images_targets}) during the evaluation must be"
            " the same."
        )

    predictions_list = self._predictions_list
    targets_list = self._targets_list
    if self._class_agnostic:
        detections = predictions_list + targets_list
        # All-unlabeled inputs keep their default category; mixed inputs
        # must join the class assigned to labeled detections in update.
        # Relabeled shallow copies keep the stored detections unmodified.
        if any(d.class_id is not None and len(d) > 0 for d in detections):
            predictions_list = [
                _with_agnostic_class_id(d) for d in predictions_list
            ]
            targets_list = [_with_agnostic_class_id(d) for d in targets_list]

    dict_targets = self._prepare_targets(targets_list)
    lst_predictions = self._prepare_predictions(predictions_list)
    # Create a coco object with the targets
    coco_gt = EvaluationDataset(targets=dict_targets)
    # Include the predictions to coco object
    coco_det = coco_gt.load_predictions(lst_predictions)
    # Create a coco evaluator with the predictions
    cocoEval = COCOEvaluator(coco_gt, coco_det, metric_target=self._metric_target)

    cocoEval.params.max_dets = list(self._max_detection_thresholds)

    # Evaluate on all images
    cocoEval.evaluate()

    # Create MeanAveragePrecisionResult object for small objects
    mAP_small = MeanAveragePrecisionResult(
        metric_target=self._metric_target,
        is_class_agnostic=self._class_agnostic,
        max_detection_thresholds=self._max_detection_thresholds,
        mAP_scores=np.asarray(
            cocoEval.results["mAP_scores_small"], dtype=np.float64
        ),
        ap_per_class=np.asarray(
            cocoEval.results["ap_per_class_small"], dtype=np.float64
        ),
        iou_thresholds=np.asarray(cocoEval.params.iou_thrs, dtype=np.float64),
        matched_classes=np.asarray(cocoEval.params.cat_ids, dtype=np.int32),
    )
    # Create MeanAveragePrecisionResult object for medium objects
    mAP_medium = MeanAveragePrecisionResult(
        metric_target=self._metric_target,
        is_class_agnostic=self._class_agnostic,
        max_detection_thresholds=self._max_detection_thresholds,
        mAP_scores=np.asarray(
            cocoEval.results["mAP_scores_medium"], dtype=np.float64
        ),
        ap_per_class=np.asarray(
            cocoEval.results["ap_per_class_medium"], dtype=np.float64
        ),
        iou_thresholds=np.asarray(cocoEval.params.iou_thrs, dtype=np.float64),
        matched_classes=np.asarray(cocoEval.params.cat_ids, dtype=np.int32),
    )
    # Create MeanAveragePrecisionResult object for large objects
    mAP_large = MeanAveragePrecisionResult(
        metric_target=self._metric_target,
        is_class_agnostic=self._class_agnostic,
        max_detection_thresholds=self._max_detection_thresholds,
        mAP_scores=np.asarray(
            cocoEval.results["mAP_scores_large"], dtype=np.float64
        ),
        ap_per_class=np.asarray(
            cocoEval.results["ap_per_class_large"], dtype=np.float64
        ),
        iou_thresholds=np.asarray(cocoEval.params.iou_thrs, dtype=np.float64),
        matched_classes=np.asarray(cocoEval.params.cat_ids, dtype=np.int32),
    )

    # Create the final MeanAveragePrecisionResult object
    mAP_result = MeanAveragePrecisionResult(
        metric_target=self._metric_target,
        is_class_agnostic=self._class_agnostic,
        max_detection_thresholds=self._max_detection_thresholds,
        mAP_scores=np.asarray(
            cocoEval.results["mAP_scores_all_sizes"], dtype=np.float64
        ),
        ap_per_class=np.asarray(
            cocoEval.results["ap_per_class_all_sizes"], dtype=np.float64
        ),
        iou_thresholds=np.asarray(cocoEval.params.iou_thrs, dtype=np.float64),
        matched_classes=np.asarray(cocoEval.params.cat_ids, dtype=np.int32),
        small_objects=mAP_small,
        medium_objects=mAP_medium,
        large_objects=mAP_large,
    )
    return mAP_result

reset() -> None

Reset the metric to its initial state, clearing all stored data.

Source code in src/supervision/metrics/mean_average_precision.py

def reset(self) -> None:
    """Reset the metric to its initial state, clearing all stored data."""
    self._predictions_list = []
    self._targets_list = []

update(predictions: Detections | list[Detections], targets: Detections | list[Detections]) -> MeanAveragePrecision

Add new predictions and targets to the metric, but do not compute the result.

Parameters:

Name	Type	Description	Default
predictions	Detections | list[Detections]	The predicted detections.	required
targets	Detections | list[Detections]	The ground-truth detections.	required

Returns:

Type	Description
MeanAveragePrecision	The updated metric instance.

Source code in src/supervision/metrics/mean_average_precision.py

def update(
    self,
    predictions: Detections | list[Detections],
    targets: Detections | list[Detections],
) -> MeanAveragePrecision:
    """Add new predictions and targets to the metric, but do not compute the result.

    Args:
        predictions: The predicted detections.
        targets: The ground-truth detections.

    Returns:
        The updated metric instance.
    """
    if not isinstance(predictions, list):
        predictions = [predictions]
    if not isinstance(targets, list):
        targets = [targets]

    if len(predictions) != len(targets):
        raise ValueError(
            f"The number of predictions ({len(predictions)}) and"
            f" targets ({len(targets)}) during the update must be the same."
        )

    if self._class_agnostic:
        # Relabel private copies so caller-owned detections stay as passed in.
        predictions = deepcopy(predictions)
        targets = deepcopy(targets)

        for prediction in predictions:
            if prediction.class_id is not None:
                prediction.class_id = _agnostic_class_ids(len(prediction))
        for target in targets:
            if target.class_id is not None:
                target.class_id = _agnostic_class_ids(len(target))

    self._predictions_list.extend(predictions)
    self._targets_list.extend(targets)

    return self

MeanAveragePrecisionResult

supervision.metrics.mean_average_precision.MeanAveragePrecisionResult dataclass

Bases: MetricResult

The result of the Mean Average Precision calculation.

Returns -1 sentinel scores when no detections or targets are present.

Attributes:

Name	Type	Description
metric_target	MetricTarget	the type of data used for the metric - boxes, masks or oriented bounding boxes.
is_class_agnostic	bool	When computing class-agnostic results, the class ID of every labeled detection is set to -1. Detections without class IDs join that class when any input has class IDs, and otherwise keep the default class 0.
mAP_scores	NDArray[float64]	the mAP scores at each IoU threshold. Shape: (num_iou_thresholds,)
ap_per_class	NDArray[float64]	the average precision scores per class and IoU threshold. Shape: (num_target_classes, num_iou_thresholds)
iou_thresholds	NDArray[float64]	the IoU thresholds used in the calculations.
matched_classes	NDArray[int32]	the class IDs of all matched classes. Corresponds to the rows of ap_per_class.
small_objects	MeanAveragePrecisionResult | None	the mAP results for small objects (area < 32²).
medium_objects	MeanAveragePrecisionResult | None	the mAP results for medium objects (32² ≤ area < 96²).
large_objects	MeanAveragePrecisionResult | None	the mAP results for large objects (area ≥ 96²).
max_detection_thresholds	tuple[int, ...]	The three evaluation limits. AP uses the last limit, per image and category after any class mapping.

Source code in src/supervision/metrics/mean_average_precision.py

@dataclass
class MeanAveragePrecisionResult(MetricResult):
    """The result of the Mean Average Precision calculation.

    Returns `-1` sentinel scores when no detections or targets are present.

    Attributes:
        metric_target: the type of data used for the metric -
            boxes, masks or oriented bounding boxes.
        is_class_agnostic: When computing class-agnostic results, the class ID of
            every labeled detection is set to `-1`. Detections without class IDs
            join that class when any input has class IDs, and otherwise keep the
            default class `0`.
        mAP_scores: the mAP scores at each IoU threshold.
            Shape: `(num_iou_thresholds,)`
        ap_per_class: the average precision scores per
            class and IoU threshold. Shape: `(num_target_classes, num_iou_thresholds)`
        iou_thresholds: the IoU thresholds used in the calculations.
        matched_classes: the class IDs of all matched classes.
            Corresponds to the rows of `ap_per_class`.
        small_objects: the mAP results
            for small objects (area < 32²).
        medium_objects: the mAP results
            for medium objects (32² ≤ area < 96²).
        large_objects: the mAP results
            for large objects (area ≥ 96²).
        max_detection_thresholds: The three evaluation limits. AP uses the last
            limit, per image and category after any class mapping.
    """

    metric_target: MetricTarget
    is_class_agnostic: bool

    @property
    def map50_95(self) -> float:
        """The mAP score at IoU thresholds from `0.5` to `0.95`."""
        return _mean_valid_score(self.mAP_scores)

    @property
    def map50(self) -> float:
        """The mAP score at IoU threshold of `0.5`."""
        return float(self.mAP_scores[0])

    @property
    def map75(self) -> float:
        """The mAP score at IoU threshold of `0.75`."""
        return float(self.mAP_scores[5])

    mAP_scores: npt.NDArray[np.float64]
    ap_per_class: npt.NDArray[np.float64]
    iou_thresholds: npt.NDArray[np.float64]
    matched_classes: npt.NDArray[np.int32]
    small_objects: MeanAveragePrecisionResult | None = None
    medium_objects: MeanAveragePrecisionResult | None = None
    large_objects: MeanAveragePrecisionResult | None = None
    max_detection_thresholds: tuple[int, ...] = (1, 10, 100)

    def __str__(self) -> str:
        """Formats the evaluation output metrics to match the structure used by
        pycocotools.

        Example:
           ```pycon
           >>> import numpy as np
           >>> import supervision as sv
           >>> from supervision.metrics import MeanAveragePrecision
           >>> predictions = sv.Detections(
           ...     xyxy=np.array([[0, 0, 10, 10]]),
           ...     class_id=np.array([0]),
           ...     confidence=np.array([0.9])
           ... )
           >>> targets = sv.Detections(
           ...     xyxy=np.array([[0, 0, 10, 10]]),
           ...     class_id=np.array([0])
           ... )
           >>> map_metric = MeanAveragePrecision()
           >>> map_result = map_metric.update(predictions, targets).compute()
           >>> print(map_result)  # doctest: +ELLIPSIS
           Average Precision (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = ...
           Average Precision (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = ...
           Average Precision (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = ...
           Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = ...
           Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = ...
           Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = ...

           ```
        """
        max_dets = self.max_detection_thresholds[-1]
        if (
            self.small_objects is None
            or self.medium_objects is None
            or self.large_objects is None
        ):
            return (
                f"Average Precision (AP) @[ IoU=0.50:0.95 | area=   all | "
                f"maxDets={max_dets} ] = {self.map50_95:.3f}\n"
                f"Average Precision (AP) @[ IoU=0.50      | area=   all | "
                f"maxDets={max_dets} ] = {self.map50:.3f}\n"
                f"Average Precision (AP) @[ IoU=0.75      | area=   all | "
                f"maxDets={max_dets} ] = {self.map75:.3f}"
            )

        return (
            f"Average Precision (AP) @[ IoU=0.50:0.95 | area=   all | "
            f"maxDets={max_dets} ] = {self.map50_95:.3f}\n"
            f"Average Precision (AP) @[ IoU=0.50      | area=   all | "
            f"maxDets={max_dets} ] = {self.map50:.3f}\n"
            f"Average Precision (AP) @[ IoU=0.75      | area=   all | "
            f"maxDets={max_dets} ] = {self.map75:.3f}\n"
            f"Average Precision (AP) @[ IoU=0.50:0.95 | area= small | "
            f"maxDets={max_dets} ] = {self.small_objects.map50_95:.3f}\n"
            f"Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | "
            f"maxDets={max_dets} ] = {self.medium_objects.map50_95:.3f}\n"
            f"Average Precision (AP) @[ IoU=0.50:0.95 | area= large | "
            f"maxDets={max_dets} ] = {self.large_objects.map50_95:.3f}"
        )

    def to_pandas(self) -> pd.DataFrame:
        """Convert the result to a pandas DataFrame.

        Returns:
            The result as a DataFrame. Its `attrs["max_detection_thresholds"]`
            records the limits without changing the score columns. DataFrame
            attributes are not preserved by CSV export.
        """
        # Average precisions are currently not included in the DataFrame.
        dataframe = _scores_to_pandas(
            {"mAP@50:95": self.map50_95, "mAP@50": self.map50, "mAP@75": self.map75},
            [
                ("small_objects", self.small_objects),
                ("medium_objects", self.medium_objects),
                ("large_objects", self.large_objects),
            ],
        )

        dataframe.attrs["max_detection_thresholds"] = self.max_detection_thresholds
        return dataframe

    def _get_plot_details(self, include_object_sizes: bool = True) -> PlotDetails:
        """Return bar-chart data for mAP scores.

        Args:
            include_object_sizes: When ``True``, include bars for
                small / medium / large object-size categories.
        """
        labels = ["mAP@50:95", "mAP@50", "mAP@75"]
        values = [self.map50_95, self.map50, self.map75]
        colors = [LEGACY_COLOR_PALETTE[0]] * 3
        _append_object_size_plot_details(
            labels,
            values,
            colors,
            include_object_sizes=include_object_sizes,
            metric_labels=["mAP@50:95", "mAP@50", "mAP@75"],
            small_objects=self.small_objects,
            medium_objects=self.medium_objects,
            large_objects=self.large_objects,
            value_getter=lambda result: [
                result.map50_95,
                result.map50,
                result.map75,
            ],
        )

        title = (
            "Mean Average Precision\n"
            f"(target: {self.metric_target.value}, "
            f"class agnostic: {self.is_class_agnostic})"
        )
        if self.max_detection_thresholds[-1] != 100:
            title += f"\nmaxDets={self.max_detection_thresholds[-1]}"
        return PlotDetails(labels=labels, values=values, colors=colors, title=title)

    def plot(self) -> None:
        """Plot the mAP results.

        ![example_plot](
        https://media.roboflow.com/supervision-docs/metrics/mAP_plot_example.png
        ){ align=center width="800" }
        """
        _show_bar_plot(self._get_plot_details())

Attributes

map50: float property

The mAP score at IoU threshold of 0.5.

map50_95: float property

The mAP score at IoU thresholds from 0.5 to 0.95.

map75: float property

The mAP score at IoU threshold of 0.75.

Methods:

__str__() -> str

Formats the evaluation output metrics to match the structure used by pycocotools.

Example

>>> import numpy as np
>>> import supervision as sv
>>> from supervision.metrics import MeanAveragePrecision
>>> predictions = sv.Detections(
...     xyxy=np.array([[0, 0, 10, 10]]),
...     class_id=np.array([0]),
...     confidence=np.array([0.9])
... )
>>> targets = sv.Detections(
...     xyxy=np.array([[0, 0, 10, 10]]),
...     class_id=np.array([0])
... )
>>> map_metric = MeanAveragePrecision()
>>> map_result = map_metric.update(predictions, targets).compute()
>>> print(map_result)  # doctest: +ELLIPSIS
Average Precision (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = ...
Average Precision (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = ...
Average Precision (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = ...
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = ...
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = ...
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = ...

Source code in src/supervision/metrics/mean_average_precision.py

def __str__(self) -> str:
    """Formats the evaluation output metrics to match the structure used by
    pycocotools.

    Example:
       ```pycon
       >>> import numpy as np
       >>> import supervision as sv
       >>> from supervision.metrics import MeanAveragePrecision
       >>> predictions = sv.Detections(
       ...     xyxy=np.array([[0, 0, 10, 10]]),
       ...     class_id=np.array([0]),
       ...     confidence=np.array([0.9])
       ... )
       >>> targets = sv.Detections(
       ...     xyxy=np.array([[0, 0, 10, 10]]),
       ...     class_id=np.array([0])
       ... )
       >>> map_metric = MeanAveragePrecision()
       >>> map_result = map_metric.update(predictions, targets).compute()
       >>> print(map_result)  # doctest: +ELLIPSIS
       Average Precision (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = ...
       Average Precision (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = ...
       Average Precision (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = ...
       Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = ...
       Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = ...
       Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = ...

       ```
    """
    max_dets = self.max_detection_thresholds[-1]
    if (
        self.small_objects is None
        or self.medium_objects is None
        or self.large_objects is None
    ):
        return (
            f"Average Precision (AP) @[ IoU=0.50:0.95 | area=   all | "
            f"maxDets={max_dets} ] = {self.map50_95:.3f}\n"
            f"Average Precision (AP) @[ IoU=0.50      | area=   all | "
            f"maxDets={max_dets} ] = {self.map50:.3f}\n"
            f"Average Precision (AP) @[ IoU=0.75      | area=   all | "
            f"maxDets={max_dets} ] = {self.map75:.3f}"
        )

    return (
        f"Average Precision (AP) @[ IoU=0.50:0.95 | area=   all | "
        f"maxDets={max_dets} ] = {self.map50_95:.3f}\n"
        f"Average Precision (AP) @[ IoU=0.50      | area=   all | "
        f"maxDets={max_dets} ] = {self.map50:.3f}\n"
        f"Average Precision (AP) @[ IoU=0.75      | area=   all | "
        f"maxDets={max_dets} ] = {self.map75:.3f}\n"
        f"Average Precision (AP) @[ IoU=0.50:0.95 | area= small | "
        f"maxDets={max_dets} ] = {self.small_objects.map50_95:.3f}\n"
        f"Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | "
        f"maxDets={max_dets} ] = {self.medium_objects.map50_95:.3f}\n"
        f"Average Precision (AP) @[ IoU=0.50:0.95 | area= large | "
        f"maxDets={max_dets} ] = {self.large_objects.map50_95:.3f}"
    )

plot() -> None

Plot the mAP results.

Source code in src/supervision/metrics/mean_average_precision.py

def plot(self) -> None:
    """Plot the mAP results.

    ![example_plot](
    https://media.roboflow.com/supervision-docs/metrics/mAP_plot_example.png
    ){ align=center width="800" }
    """
    _show_bar_plot(self._get_plot_details())

to_pandas() -> pd.DataFrame

Convert the result to a pandas DataFrame.

Returns:

Type	Description
DataFrame	The result as a DataFrame. Its attrs["max_detection_thresholds"]
DataFrame	records the limits without changing the score columns. DataFrame
DataFrame	attributes are not preserved by CSV export.

Source code in src/supervision/metrics/mean_average_precision.py

def to_pandas(self) -> pd.DataFrame:
    """Convert the result to a pandas DataFrame.

    Returns:
        The result as a DataFrame. Its `attrs["max_detection_thresholds"]`
        records the limits without changing the score columns. DataFrame
        attributes are not preserved by CSV export.
    """
    # Average precisions are currently not included in the DataFrame.
    dataframe = _scores_to_pandas(
        {"mAP@50:95": self.map50_95, "mAP@50": self.map50, "mAP@75": self.map75},
        [
            ("small_objects", self.small_objects),
            ("medium_objects", self.medium_objects),
            ("large_objects", self.large_objects),
        ],
    )

    dataframe.attrs["max_detection_thresholds"] = self.max_detection_thresholds
    return dataframe

get_coco_class_index_mapping

supervision.dataset.formats.coco.get_coco_class_index_mapping(annotations_path: str) -> dict[int, int]

Generates a mapping from sequential class indices to original COCO class ids.

This function is essential when working with models that expect class ids to be zero-indexed and sequential (0 to 79), as opposed to the original COCO dataset where category ids are non-contiguous ranging from 1 to 90 but skipping some ids.

Use Cases

• Evaluating models trained with COCO-style annotations where class ids are sequential ranging from 0 to 79.
• Ensuring consistent class indexing across training, inference and evaluation, when using different tools or datasets with COCO format.
• Reproducing results from models that assume sequential class ids (0 to 79).

How it Works

• Reads the COCO annotation file in its original format (annotations_path).
• Extracts and sorts all class names by their original COCO id (1 to 90).
• Builds a mapping from COCO class ids (not sequential with skipped ids) to new class ids (sequential ranging from 0 to 79).
• Returns a dictionary mapping: {new_class_id: original_COCO_class_id}.

Parameters:

Name	Type	Description	Default
annotations_path	str	Path to COCO JSON annotations file	required

Returns:

Type	Description
dict[int, int]	A mapping from new class id (sequential ranging from 0 to 79)
dict[int, int]	to original COCO class id (1 to 90 with skipped ids).

Examples:

>>> import json
>>> import os
>>> import tempfile
>>> from supervision.dataset.formats.coco import get_coco_class_index_mapping
>>> coco_data = {
...     "categories": [
...         {"id": 1, "name": "person"},
...         {"id": 3, "name": "car"},
...     ],
...     "images": [],
...     "annotations": [],
... }
>>> annotations_path = None
>>> try:
...     with tempfile.NamedTemporaryFile(
...         mode="w", suffix=".json", delete=False
...     ) as f:
...         annotations_path = f.name
...         json.dump(coco_data, f)
...     mapping = get_coco_class_index_mapping(annotations_path)
...     print(mapping)
... finally:
...     if annotations_path is not None:
...         os.remove(annotations_path)
{0: 1, 1: 3}
>>> os.path.exists(annotations_path)
False

Source code in src/supervision/dataset/formats/coco.py

def get_coco_class_index_mapping(annotations_path: str) -> dict[int, int]:
    """Generates a mapping from sequential class indices to original COCO class ids.

    This function is essential when working with models that expect class ids to be
    zero-indexed and sequential (0 to 79), as opposed to the original COCO
    dataset where category ids are non-contiguous ranging from 1 to 90 but skipping some
    ids.

    Use Cases:
        - Evaluating models trained with COCO-style annotations where class ids
          are sequential ranging from 0 to 79.
        - Ensuring consistent class indexing across training, inference and evaluation,
          when using different tools or datasets with COCO format.
        - Reproducing results from models that assume sequential class ids (0 to 79).

    How it Works:
        - Reads the COCO annotation file in its original format (`annotations_path`).
        - Extracts and sorts all class names by their original COCO id (1 to 90).
        - Builds a mapping from COCO class ids (not sequential with skipped ids) to
          new class ids (sequential ranging from 0 to 79).
        - Returns a dictionary mapping: `{new_class_id: original_COCO_class_id}`.

    Args:
        annotations_path: Path to COCO JSON annotations file
        (e.g., `instances_val2017.json`).

    Returns:
        A mapping from new class id (sequential ranging from 0 to 79)
        to original COCO class id (1 to 90 with skipped ids).

    Examples:
        ```pycon
        >>> import json
        >>> import os
        >>> import tempfile
        >>> from supervision.dataset.formats.coco import get_coco_class_index_mapping
        >>> coco_data = {
        ...     "categories": [
        ...         {"id": 1, "name": "person"},
        ...         {"id": 3, "name": "car"},
        ...     ],
        ...     "images": [],
        ...     "annotations": [],
        ... }
        >>> annotations_path = None
        >>> try:
        ...     with tempfile.NamedTemporaryFile(
        ...         mode="w", suffix=".json", delete=False
        ...     ) as f:
        ...         annotations_path = f.name
        ...         json.dump(coco_data, f)
        ...     mapping = get_coco_class_index_mapping(annotations_path)
        ...     print(mapping)
        ... finally:
        ...     if annotations_path is not None:
        ...         os.remove(annotations_path)
        {0: 1, 1: 3}
        >>> os.path.exists(annotations_path)
        False

        ```
    """
    coco_data = read_json_file(annotations_path)
    classes = coco_categories_to_classes(coco_categories=coco_data["categories"])
    class_mapping = build_coco_class_index_mapping(
        coco_categories=coco_data["categories"], target_classes=classes
    )
    return {v: k for k, v in class_mapping.items()}

KeyPointMeanAveragePrecision

supervision.metrics.keypoint_mean_average_precision.KeyPointMeanAveragePrecision

Bases: Metric[KeyPointMeanAveragePrecisionResult]

Keypoint Mean Average Precision based on Object Keypoint Similarity (OKS).

This is the COCO keypoint metric: predictions are matched to targets by OKS instead of IoU, at 10 OKS thresholds from 0.5 to 0.95, with 101-point recall interpolation and at most 20 predictions per image and class. The Inputs are one :class:~supervision.key_points.core.KeyPoints per image for predictions and targets. Matching and accumulation are shared with :class:~supervision.metrics.mean_average_precision.MeanAveragePrecision. The result matches pycocotools COCOeval(..., iouType="keypoints") when the target areas match.

The following behavior details matter for pycocotools parity:

• Visibility. Target keypoints with visible set to False are unlabelled, like COCO v=0, and do not enter the OKS. COCO v=1 (labelled but occluded) and v=2 (labelled and visible) both map to visible=True, because pycocotools counts every keypoint with v>0. visible=None means every keypoint is labelled. Prediction visibility is not used.
• Targets without visible keypoints (COCO num_keypoints == 0). When targets.data["xyxy"] holds the target boxes as (N, 4) (x_min, y_min, x_max, y_max), such a target is kept as an ignore region, as in pycocotools: OKS is measured to its box expanded by its own width and height on each side, a prediction matched to it is ignored rather than counted, and it is never a miss, in every size bucket. Its area is targets.data["area"] or, if absent, the box area. Without boxes the target is skipped, so a prediction that lands on it counts as a false positive; if it was the class's only target, the class has no targets and scores -1. Boxes of targets with visible keypoints are not used.
• Area. OKS is normalized by the target's object area. Pass it as targets.data["area"] (the same key :class:~supervision.metrics.mean_average_precision.MeanAveragePrecision reads) to reproduce COCO, which uses the annotated segmentation area. Without it, the area of the box spanning the target's visible keypoints is used. That box is usually tighter than the person's outline, so each pair's OKS differs from COCO; mAP can move either way, since matches change too. A target with a single visible keypoint gets area 0. The fallback keeps the metric usable on keypoint-only labels.
• Scores. Predictions are ranked by detection_confidence; keypoint confidence is not used. When detection_confidence is None, every prediction of that image scores 0, as in :class:~supervision.metrics.mean_average_precision.MeanAveragePrecision.
• Object size. Targets fall into the medium and large buckets by that same area. Predictions fall in by the area of their box in predictions.data["xyxy"] when given, else of the box spanning all their keypoints, as pycocotools does with and without a result bbox. Boxes never change the OKS of a prediction.
• Crowd. targets.data["iscrowd"] marks crowd targets, as in COCO iscrowd=1. A crowd target is an ignore region: it is never a miss, and any number of predictions may match it and are then ignored. OKS against it is computed like any other target, using its box when it has no visible keypoint. Without the key, every target is a regular instance.
• Invalid values. Non-finite (NaN or infinite) coordinates are not rejected. A non-finite target keypoint counts as unlabelled, like visible=False, so a target with only non-finite keypoints is a target without visible keypoints. A non-finite prediction keypoint adds 0 to the OKS while its finite keypoints still count. pycocotools instead returns a NaN OKS when a labelled keypoint of the pair is NaN. A target area must be finite and non-negative. With an area of 0 only keypoints at exactly zero distance add to the OKS, so such pairs score low or 0. Clean such labels beforehand.

Examples:

>>> import numpy as np
>>> import supervision as sv
>>> from supervision.metrics import KeyPointMeanAveragePrecision
>>> targets = sv.KeyPoints(
...     xy=np.array([[[10, 10], [60, 10], [35, 80]]], dtype=np.float32),
...     class_id=np.array([0]),
...     data={"area": np.array([4000.0])},
... )
>>> predictions = sv.KeyPoints(
...     xy=np.array([[[11, 10], [60, 12], [35, 79]]], dtype=np.float32),
...     class_id=np.array([0]),
...     detection_confidence=np.array([0.9]),
... )
>>> metric = KeyPointMeanAveragePrecision(sigmas=[0.25, 0.25, 0.25])
>>> result = metric.update(predictions, targets).compute()
>>> round(result.map50, 2)
1.0

Source code in src/supervision/metrics/keypoint_mean_average_precision.py

class KeyPointMeanAveragePrecision(Metric[KeyPointMeanAveragePrecisionResult]):
    """Keypoint Mean Average Precision based on Object Keypoint Similarity (OKS).

    This is the COCO keypoint metric: predictions are matched to targets by OKS
    instead of IoU, at 10 OKS thresholds from `0.5` to `0.95`, with 101-point
    recall interpolation and at most 20 predictions per image and class. The
    Inputs are one :class:`~supervision.key_points.core.KeyPoints` per image for
    predictions and targets. Matching and accumulation are shared with
    :class:`~supervision.metrics.mean_average_precision.MeanAveragePrecision`.
    The result matches `pycocotools` `COCOeval(..., iouType="keypoints")` when the
    target areas match.

    The following behavior details matter for `pycocotools` parity:

    - **Visibility.** Target keypoints with `visible` set to `False` are
      unlabelled, like COCO `v=0`, and do not enter the OKS. COCO `v=1`
      (labelled but occluded) and `v=2` (labelled and visible) both map to
      `visible=True`, because `pycocotools` counts every keypoint with `v>0`.
      `visible=None` means every keypoint is labelled. Prediction visibility
      is not used.
    - **Targets without visible keypoints** (COCO `num_keypoints == 0`). When
      `targets.data["xyxy"]` holds the target boxes as `(N, 4)`
      `(x_min, y_min, x_max, y_max)`, such a target is kept as an ignore
      region, as in `pycocotools`: OKS is measured to its box expanded by its
      own width and height on each side, a prediction matched to it is ignored
      rather than counted, and it is never a miss, in every size bucket. Its
      area is `targets.data["area"]` or, if absent, the box area. Without
      boxes the target is skipped, so a prediction that lands on it counts as
      a false positive; if it was the class's only target, the class has no
      targets and scores `-1`. Boxes of targets with visible keypoints are not used.
    - **Area.** OKS is normalized by the target's object area. Pass it as
      `targets.data["area"]` (the same key
      :class:`~supervision.metrics.mean_average_precision.MeanAveragePrecision`
      reads) to
      reproduce COCO, which uses the annotated segmentation area. Without it,
      the area of the box spanning the target's visible keypoints is used. That
      box is usually tighter than the person's outline, so each pair's OKS
      differs from COCO; mAP can move either way, since matches change too. A
      target with a single visible keypoint gets area `0`. The fallback keeps
      the metric usable on keypoint-only labels.
    - **Scores.** Predictions are ranked by `detection_confidence`; keypoint
      `confidence` is not used. When `detection_confidence` is `None`, every
      prediction of that image scores `0`, as in
      :class:`~supervision.metrics.mean_average_precision.MeanAveragePrecision`.
    - **Object size.** Targets fall into the medium and large buckets by that
      same area. Predictions fall in by the area of their box in
      `predictions.data["xyxy"]` when given, else of the box spanning all
      their keypoints, as `pycocotools` does with and without a result
      `bbox`. Boxes never change the OKS of a prediction.
    - **Crowd.** `targets.data["iscrowd"]` marks crowd targets, as in COCO
      `iscrowd=1`. A crowd target is an ignore region: it is never a miss, and
      any number of predictions may match it and are then ignored. OKS against
      it is computed like any other target, using its box when it has no
      visible keypoint. Without the key, every target is a regular instance.
    - **Invalid values.** Non-finite (NaN or infinite) coordinates are not
      rejected. A non-finite target keypoint counts as unlabelled, like
      `visible=False`, so a target with only non-finite keypoints is a target
      without visible keypoints. A non-finite prediction keypoint adds `0` to
      the OKS while its finite keypoints still count. `pycocotools` instead
      returns a NaN OKS when a labelled keypoint of the pair is NaN. A target
      area must be finite and non-negative. With an area of `0` only
      keypoints at exactly zero distance add to the OKS, so such pairs score
      low or `0`. Clean such labels beforehand.

    Examples:
        ```pycon
        >>> import numpy as np
        >>> import supervision as sv
        >>> from supervision.metrics import KeyPointMeanAveragePrecision
        >>> targets = sv.KeyPoints(
        ...     xy=np.array([[[10, 10], [60, 10], [35, 80]]], dtype=np.float32),
        ...     class_id=np.array([0]),
        ...     data={"area": np.array([4000.0])},
        ... )
        >>> predictions = sv.KeyPoints(
        ...     xy=np.array([[[11, 10], [60, 12], [35, 79]]], dtype=np.float32),
        ...     class_id=np.array([0]),
        ...     detection_confidence=np.array([0.9]),
        ... )
        >>> metric = KeyPointMeanAveragePrecision(sigmas=[0.25, 0.25, 0.25])
        >>> result = metric.update(predictions, targets).compute()
        >>> round(result.map50, 2)
        1.0

        ```
    """

    def __init__(
        self,
        sigmas: npt.ArrayLike | None = None,
        class_agnostic: bool = False,
    ) -> None:
        """Initialize the keypoint Mean Average Precision metric.

        Args:
            sigmas: Per-keypoint OKS sigmas of shape `(K,)`. Required unless the
                skeleton has 17 keypoints, in which case it defaults to
                the COCO 17-point sigmas. There is no generic default: sigmas
                encode how precisely each keypoint can be annotated, so COCO
                values are wrong for other skeletons.
            class_agnostic: Whether to treat all data as a single class with ID
                `-1`. When no input has class IDs, that class keeps the
                default ID `0`.

        Raises:
            ValueError: If `sigmas` holds a value that is not positive and
                finite. The number of sigmas must match the skeleton's keypoint
                count when it is added to the metric.
        """
        self._sigmas = None if sigmas is None else np.asarray(sigmas, np.float64)
        if self._sigmas is not None:
            _validate_keypoint_sigmas(self._sigmas)
        self._class_agnostic = class_agnostic
        self._num_keypoints: int | None = None
        self._predictions_list: list[KeyPoints] = []
        self._targets_list: list[KeyPoints] = []

    def reset(self) -> None:
        """Reset the metric to its initial state, clearing all stored data."""
        self._num_keypoints = None
        self._predictions_list = []
        self._targets_list = []

    def update(
        self,
        predictions: KeyPoints | list[KeyPoints],
        targets: KeyPoints | list[KeyPoints],
    ) -> KeyPointMeanAveragePrecision:
        """Add predictions and targets of one or more images to the metric.

        Args:
            predictions: The predicted key points, one
                :class:`~supervision.key_points.core.KeyPoints` per image.
            targets: The ground-truth key points, one
                :class:`~supervision.key_points.core.KeyPoints` per image.

        Returns:
            The updated metric instance.

        Raises:
            ValueError: If the numbers of predictions and targets differ, a
                skeleton has no keypoints, a skeleton's number of keypoints
                differs from earlier ones or from the length of `sigmas`,
                `sigmas` is missing for a skeleton that is not 17 points long,
                `data["xyxy"]` of predictions or targets is not of shape
                `(N, 4)`, `targets.data["iscrowd"]` is not of shape `(N,)`, or
                `targets.data["area"]` is not of shape `(N,)` or holds a
                negative or non-finite area. A rejected update leaves the
                metric unchanged.
        """
        if not isinstance(predictions, list):
            predictions = [predictions]
        if not isinstance(targets, list):
            targets = [targets]

        if len(predictions) != len(targets):
            raise ValueError(
                f"The number of predictions ({len(predictions)}) and"
                f" targets ({len(targets)}) during the update must be the same."
            )

        num_keypoints = self._num_keypoints
        for key_points in [*predictions, *targets]:
            num_keypoints = self._check_num_keypoints(key_points, num_keypoints)
        for name, key_points_list in (
            ("predictions", predictions),
            ("targets", targets),
        ):
            for key_points in key_points_list:
                boxes = key_points.data.get(XYXY_DATA_FIELD)
                if boxes is not None and np.shape(boxes) != (len(key_points), 4):
                    raise ValueError(
                        f"`{name}.data['{XYXY_DATA_FIELD}']` must have shape "
                        f"({len(key_points)}, 4); got {np.shape(boxes)}."
                    )
        for key_points in targets:
            iscrowd = key_points.data.get(ISCROWD_DATA_FIELD)
            if iscrowd is not None and np.shape(iscrowd) != (len(key_points),):
                raise ValueError(
                    f"`targets.data['{ISCROWD_DATA_FIELD}']` must have shape "
                    f"({len(key_points)},); got {np.shape(iscrowd)}."
                )
            _check_target_areas(key_points)

        # Store anything only once every check passed, so that a rejected update
        # does not pin the skeleton size for later updates.
        self._num_keypoints = num_keypoints
        self._predictions_list.extend(predictions)
        self._targets_list.extend(targets)
        return self

    def _check_num_keypoints(
        self, key_points: KeyPoints, num_keypoints: int | None
    ) -> int | None:
        """Check that a skeleton matches earlier ones and the given sigmas.

        Args:
            key_points: The skeletons of one image.
            num_keypoints: The number of keypoints of earlier skeletons, or
                `None` if there were none.

        Returns:
            The number of keypoints of these and earlier skeletons, or `None`
            while there are none.

        Raises:
            ValueError: If the skeletons have no keypoints, the number of
                keypoints differs from earlier skeletons or from the length of
                `sigmas`, or `sigmas` is missing for a skeleton that is not 17
                points long.
        """
        if len(key_points) == 0:
            return num_keypoints
        skeleton_size = int(key_points.xy.shape[1])
        if skeleton_size == 0:
            raise ValueError(
                "Skeletons must have at least one keypoint; got `xy` of shape "
                f"{key_points.xy.shape}."
            )
        if num_keypoints is None:
            _resolve_keypoint_sigmas(self._sigmas, skeleton_size)
            return skeleton_size
        if skeleton_size != num_keypoints:
            raise ValueError(
                f"Skeletons have {skeleton_size} keypoints, but earlier skeletons "
                f"passed to `update` since the last `reset` have "
                f"{num_keypoints}; all skeletons must have the same "
                f"number of keypoints."
            )
        return num_keypoints

    def _agnostic_category_id(self) -> int:
        """Return the single category of all skeletons when class agnostic.

        As in
        :class:`~supervision.metrics.mean_average_precision.MeanAveragePrecision`,
        it is `-1` when any stored skeleton has a
        class ID, and otherwise the default class `0`.
        """
        has_class_ids = any(
            key_points.class_id is not None and len(key_points) > 0
            for key_points in [*self._predictions_list, *self._targets_list]
        )
        return -1 if has_class_ids else 0

    def _category_id(self, key_points: KeyPoints, index: int, agnostic_id: int) -> int:
        """Return the evaluation category of one skeleton.

        Args:
            key_points: The skeletons of one image.
            index: The skeleton's index in `key_points`.
            agnostic_id: The category of every skeleton when class agnostic.

        Returns:
            `agnostic_id` when class agnostic, `0` without `class_id`, else its
            class.
        """
        if self._class_agnostic:
            return agnostic_id
        if key_points.class_id is None:
            return 0
        return int(key_points.class_id[index])

    def _prepare_targets(
        self, agnostic_id: int
    ) -> tuple[dict[str, list[_TypeCocoDict]], dict[int, npt.NDArray[np.float64]]]:
        """Transform targets into the COCO dictionary used by the evaluator.

        Args:
            agnostic_id: The category of every target when class agnostic.

        Returns:
            The COCO dataset dictionary, and the xyxy box by annotation id of
            each kept target without visible keypoints.
        """
        target_boxes: dict[int, npt.NDArray[np.float64]] = {}
        images: list[_TypeCocoDict] = [
            {"id": image_id} for image_id in range(len(self._targets_list))
        ]
        annotations: list[_TypeCocoDict] = []
        for image_id, image_targets in enumerate(self._targets_list):
            if len(image_targets) == 0:
                continue
            # `xy` may carry a third (z) column; OKS is planar.
            xy = image_targets.xy[..., :2].astype(np.float64)
            visible = (
                np.ones(xy.shape[:2], dtype=bool)
                if image_targets.visible is None
                else np.asarray(image_targets.visible, dtype=bool)
            )
            # A non-finite target keypoint is unlabelled, like COCO `v=0`, so it
            # stays out of the OKS and the fallback area. Not in place: `visible`
            # may be the caller's array.
            visible = visible & np.isfinite(xy).all(axis=-1)
            areas = image_targets.data.get(AREA_DATA_FIELD)
            boxes = image_targets.data.get(XYXY_DATA_FIELD)
            if boxes is not None:
                boxes = np.asarray(boxes, dtype=np.float64)
            iscrowd = image_targets.data.get(ISCROWD_DATA_FIELD)
            # Per-image arrays, so that the loop below only picks rows. Rows of
            # targets without visible keypoints are infinite and never read.
            span_boxes = _keypoints_xywh(xy, visible)
            span_areas = span_boxes[:, 2] * span_boxes[:, 3]
            contents = np.concatenate([xy, visible[..., None]], axis=2)
            for index in range(len(image_targets)):
                is_ignored = not visible[index].any()
                if is_ignored and boxes is None:
                    continue
                xyxy = boxes[index] if is_ignored and boxes is not None else None
                if areas is not None:
                    area = float(areas[index])
                elif xyxy is not None:
                    area = float((xyxy[2] - xyxy[0]) * (xyxy[3] - xyxy[1]))
                else:
                    area = float(span_areas[index])
                content = contents[index]
                annotation_id = len(annotations) + 1  # 0 means no match
                if xyxy is not None:
                    target_boxes[annotation_id] = xyxy
                is_crowd = iscrowd is not None and bool(iscrowd[index])
                annotations.append(
                    {
                        "id": annotation_id,
                        "image_id": image_id,
                        "category_id": self._category_id(
                            image_targets, index, agnostic_id
                        ),
                        "area": area,
                        "iscrowd": int(is_crowd),
                        "ignore": int(is_ignored),
                        "content": content,
                    }
                )
        category_ids = sorted({annotation["category_id"] for annotation in annotations})
        categories: list[_TypeCocoDict] = [{"id": cat_id} for cat_id in category_ids]
        dataset = {
            "images": images,
            "annotations": annotations,
            "categories": categories,
        }
        return dataset, target_boxes

    def _prepare_predictions(self, agnostic_id: int) -> list[_TypeCocoDict]:
        """Transform predictions into the COCO result list used by the evaluator.

        Args:
            agnostic_id: The category of every prediction when class agnostic.
        """
        coco_predictions: list[_TypeCocoDict] = []
        for image_id, image_predictions in enumerate(self._predictions_list):
            if len(image_predictions) == 0:
                continue
            xy = image_predictions.xy[..., :2].astype(np.float64)
            confidence = image_predictions.detection_confidence
            boxes = image_predictions.data.get(XYXY_DATA_FIELD)
            # Per-image arrays, so that the loop below only picks rows.
            if boxes is None:
                bboxes = _keypoints_xywh(xy)
            else:
                xyxy = np.asarray(boxes, dtype=np.float64)
                bboxes = np.concatenate([xyxy[:, :2], xyxy[:, 2:] - xyxy[:, :2]], 1)
            scores = (
                np.zeros(len(image_predictions))
                if confidence is None
                else np.asarray(confidence, dtype=np.float64)
            )
            areas = bboxes[:, 2] * bboxes[:, 3]
            contents = np.concatenate([xy, np.ones((*xy.shape[:2], 1))], axis=2)
            rows = zip(bboxes.tolist(), areas.tolist(), scores.tolist())
            for index, (bbox, area, score) in enumerate(rows):
                coco_predictions.append(
                    {
                        "image_id": image_id,
                        "category_id": self._category_id(
                            image_predictions, index, agnostic_id
                        ),
                        "bbox": bbox,
                        "area": area,
                        "score": score,
                        "content": contents[index],
                    }
                )
        return coco_predictions

    def compute(self) -> KeyPointMeanAveragePrecisionResult:
        """Compute keypoint mAP from the stored predictions and targets.

        Returns:
            The keypoint Mean Average Precision result.
        """
        sigmas = (
            _resolve_keypoint_sigmas(self._sigmas, self._num_keypoints)
            if self._num_keypoints is not None
            else np.empty(0, dtype=np.float64)
        )
        agnostic_id = self._agnostic_category_id()
        dataset, target_boxes = self._prepare_targets(agnostic_id)
        coco_targets = EvaluationDataset(targets=dataset)
        coco_predictions = coco_targets.load_predictions(
            self._prepare_predictions(agnostic_id)
        )
        evaluator = _KeyPointCOCOEvaluator(
            coco_targets, coco_predictions, sigmas, target_boxes
        )
        evaluator.evaluate()

        def make_result(
            size: str,
        ) -> KeyPointMeanAveragePrecisionResult:
            """Build the result for one object-size bucket."""
            return KeyPointMeanAveragePrecisionResult(
                is_class_agnostic=self._class_agnostic,
                mAP_scores=np.asarray(
                    evaluator.results[f"mAP_scores_{size}"], dtype=np.float64
                ),
                ap_per_class=np.asarray(
                    evaluator.results[f"ap_per_class_{size}"], dtype=np.float64
                ),
                oks_thresholds=np.asarray(evaluator.params.iou_thrs, dtype=np.float64),
                matched_classes=np.asarray(evaluator.params.cat_ids, dtype=np.int32),
            )

        result = make_result("all_sizes")
        result.medium_objects = make_result("medium")
        result.large_objects = make_result("large")
        return result

Methods:

__init__(sigmas: npt.ArrayLike | None = None, class_agnostic: bool = False) -> None

Initialize the keypoint Mean Average Precision metric.

Parameters:

Name	Type	Description	Default
sigmas	ArrayLike | None	Per-keypoint OKS sigmas of shape (K,). Required unless the skeleton has 17 keypoints, in which case it defaults to the COCO 17-point sigmas. There is no generic default: sigmas encode how precisely each keypoint can be annotated, so COCO values are wrong for other skeletons.	None
class_agnostic	bool	Whether to treat all data as a single class with ID -1. When no input has class IDs, that class keeps the default ID 0.	False

Raises:

Type	Description
ValueError	If sigmas holds a value that is not positive and finite. The number of sigmas must match the skeleton's keypoint count when it is added to the metric.

Source code in src/supervision/metrics/keypoint_mean_average_precision.py

def __init__(
    self,
    sigmas: npt.ArrayLike | None = None,
    class_agnostic: bool = False,
) -> None:
    """Initialize the keypoint Mean Average Precision metric.

    Args:
        sigmas: Per-keypoint OKS sigmas of shape `(K,)`. Required unless the
            skeleton has 17 keypoints, in which case it defaults to
            the COCO 17-point sigmas. There is no generic default: sigmas
            encode how precisely each keypoint can be annotated, so COCO
            values are wrong for other skeletons.
        class_agnostic: Whether to treat all data as a single class with ID
            `-1`. When no input has class IDs, that class keeps the
            default ID `0`.

    Raises:
        ValueError: If `sigmas` holds a value that is not positive and
            finite. The number of sigmas must match the skeleton's keypoint
            count when it is added to the metric.
    """
    self._sigmas = None if sigmas is None else np.asarray(sigmas, np.float64)
    if self._sigmas is not None:
        _validate_keypoint_sigmas(self._sigmas)
    self._class_agnostic = class_agnostic
    self._num_keypoints: int | None = None
    self._predictions_list: list[KeyPoints] = []
    self._targets_list: list[KeyPoints] = []

compute() -> KeyPointMeanAveragePrecisionResult

Compute keypoint mAP from the stored predictions and targets.

Returns:

Type	Description
KeyPointMeanAveragePrecisionResult	The keypoint Mean Average Precision result.

Source code in src/supervision/metrics/keypoint_mean_average_precision.py

def compute(self) -> KeyPointMeanAveragePrecisionResult:
    """Compute keypoint mAP from the stored predictions and targets.

    Returns:
        The keypoint Mean Average Precision result.
    """
    sigmas = (
        _resolve_keypoint_sigmas(self._sigmas, self._num_keypoints)
        if self._num_keypoints is not None
        else np.empty(0, dtype=np.float64)
    )
    agnostic_id = self._agnostic_category_id()
    dataset, target_boxes = self._prepare_targets(agnostic_id)
    coco_targets = EvaluationDataset(targets=dataset)
    coco_predictions = coco_targets.load_predictions(
        self._prepare_predictions(agnostic_id)
    )
    evaluator = _KeyPointCOCOEvaluator(
        coco_targets, coco_predictions, sigmas, target_boxes
    )
    evaluator.evaluate()

    def make_result(
        size: str,
    ) -> KeyPointMeanAveragePrecisionResult:
        """Build the result for one object-size bucket."""
        return KeyPointMeanAveragePrecisionResult(
            is_class_agnostic=self._class_agnostic,
            mAP_scores=np.asarray(
                evaluator.results[f"mAP_scores_{size}"], dtype=np.float64
            ),
            ap_per_class=np.asarray(
                evaluator.results[f"ap_per_class_{size}"], dtype=np.float64
            ),
            oks_thresholds=np.asarray(evaluator.params.iou_thrs, dtype=np.float64),
            matched_classes=np.asarray(evaluator.params.cat_ids, dtype=np.int32),
        )

    result = make_result("all_sizes")
    result.medium_objects = make_result("medium")
    result.large_objects = make_result("large")
    return result

reset() -> None

Reset the metric to its initial state, clearing all stored data.

Source code in src/supervision/metrics/keypoint_mean_average_precision.py

def reset(self) -> None:
    """Reset the metric to its initial state, clearing all stored data."""
    self._num_keypoints = None
    self._predictions_list = []
    self._targets_list = []

update(predictions: KeyPoints | list[KeyPoints], targets: KeyPoints | list[KeyPoints]) -> KeyPointMeanAveragePrecision

Add predictions and targets of one or more images to the metric.

Parameters:

Name	Type	Description	Default
predictions	KeyPoints | list[KeyPoints]	The predicted key points, one :class:~supervision.key_points.core.KeyPoints per image.	required
targets	KeyPoints | list[KeyPoints]	The ground-truth key points, one :class:~supervision.key_points.core.KeyPoints per image.	required

Returns:

Type	Description
KeyPointMeanAveragePrecision	The updated metric instance.

Raises:

Type

Description

ValueError

If the numbers of predictions and targets differ, a skeleton has no keypoints, a skeleton's number of keypoints differs from earlier ones or from the length of sigmas, sigmas is missing for a skeleton that is not 17 points long, data["xyxy"] of predictions or targets is not of shape (N, 4), targets.data["iscrowd"] is not of shape (N,), or targets.data["area"] is not of shape (N,) or holds a negative or non-finite area. A rejected update leaves the metric unchanged.

Source code in src/supervision/metrics/keypoint_mean_average_precision.py

def update(
    self,
    predictions: KeyPoints | list[KeyPoints],
    targets: KeyPoints | list[KeyPoints],
) -> KeyPointMeanAveragePrecision:
    """Add predictions and targets of one or more images to the metric.

    Args:
        predictions: The predicted key points, one
            :class:`~supervision.key_points.core.KeyPoints` per image.
        targets: The ground-truth key points, one
            :class:`~supervision.key_points.core.KeyPoints` per image.

    Returns:
        The updated metric instance.

    Raises:
        ValueError: If the numbers of predictions and targets differ, a
            skeleton has no keypoints, a skeleton's number of keypoints
            differs from earlier ones or from the length of `sigmas`,
            `sigmas` is missing for a skeleton that is not 17 points long,
            `data["xyxy"]` of predictions or targets is not of shape
            `(N, 4)`, `targets.data["iscrowd"]` is not of shape `(N,)`, or
            `targets.data["area"]` is not of shape `(N,)` or holds a
            negative or non-finite area. A rejected update leaves the
            metric unchanged.
    """
    if not isinstance(predictions, list):
        predictions = [predictions]
    if not isinstance(targets, list):
        targets = [targets]

    if len(predictions) != len(targets):
        raise ValueError(
            f"The number of predictions ({len(predictions)}) and"
            f" targets ({len(targets)}) during the update must be the same."
        )

    num_keypoints = self._num_keypoints
    for key_points in [*predictions, *targets]:
        num_keypoints = self._check_num_keypoints(key_points, num_keypoints)
    for name, key_points_list in (
        ("predictions", predictions),
        ("targets", targets),
    ):
        for key_points in key_points_list:
            boxes = key_points.data.get(XYXY_DATA_FIELD)
            if boxes is not None and np.shape(boxes) != (len(key_points), 4):
                raise ValueError(
                    f"`{name}.data['{XYXY_DATA_FIELD}']` must have shape "
                    f"({len(key_points)}, 4); got {np.shape(boxes)}."
                )
    for key_points in targets:
        iscrowd = key_points.data.get(ISCROWD_DATA_FIELD)
        if iscrowd is not None and np.shape(iscrowd) != (len(key_points),):
            raise ValueError(
                f"`targets.data['{ISCROWD_DATA_FIELD}']` must have shape "
                f"({len(key_points)},); got {np.shape(iscrowd)}."
            )
        _check_target_areas(key_points)

    # Store anything only once every check passed, so that a rejected update
    # does not pin the skeleton size for later updates.
    self._num_keypoints = num_keypoints
    self._predictions_list.extend(predictions)
    self._targets_list.extend(targets)
    return self

KeyPointMeanAveragePrecisionResult

supervision.metrics.keypoint_mean_average_precision.KeyPointMeanAveragePrecisionResult dataclass

Bases: MetricResult

The result of the keypoint Mean Average Precision calculation.

Scores are -1 when there are no targets to evaluate. COCO keypoint evaluation reports no small-object bucket, because small people are not annotated with keypoints in COCO, so only medium and large results exist.

Attributes:

Name	Type	Description
is_class_agnostic	bool	When computing class-agnostic results, every skeleton gets class ID -1 when any input has class IDs, and otherwise keeps the default class 0, as in :class:~supervision.metrics.mean_average_precision.MeanAveragePrecision.
mAP_scores	NDArray[float64]	the mAP scores at each OKS threshold. Shape: (num_oks_thresholds,)
ap_per_class	NDArray[float64]	the average precision scores per class and OKS threshold. Shape: (num_target_classes, num_oks_thresholds)
oks_thresholds	NDArray[float64]	the OKS thresholds used in the calculations.
matched_classes	NDArray[int32]	the class IDs of all target classes. Corresponds to the rows of ap_per_class.
medium_objects	KeyPointMeanAveragePrecisionResult | None	the mAP results for medium objects (32² ≤ area < 96²).
large_objects	KeyPointMeanAveragePrecisionResult | None	the mAP results for large objects (area ≥ 96²).

Source code in src/supervision/metrics/keypoint_mean_average_precision.py

@dataclass
class KeyPointMeanAveragePrecisionResult(MetricResult):
    """The result of the keypoint Mean Average Precision calculation.

    Scores are `-1` when there are no targets to evaluate. COCO keypoint
    evaluation reports no small-object bucket, because small people are not
    annotated with keypoints in COCO, so only medium and large results exist.

    Attributes:
        is_class_agnostic: When computing class-agnostic results, every
            skeleton gets class ID `-1` when any input has class IDs, and
            otherwise keeps the default class `0`, as in
            :class:`~supervision.metrics.mean_average_precision.MeanAveragePrecision`.
        mAP_scores: the mAP scores at each OKS threshold.
            Shape: `(num_oks_thresholds,)`
        ap_per_class: the average precision scores per class and OKS threshold.
            Shape: `(num_target_classes, num_oks_thresholds)`
        oks_thresholds: the OKS thresholds used in the calculations.
        matched_classes: the class IDs of all target classes.
            Corresponds to the rows of `ap_per_class`.
        medium_objects: the mAP results for medium objects
            (32² ≤ area < 96²).
        large_objects: the mAP results for large objects (area ≥ 96²).
    """

    is_class_agnostic: bool

    @property
    def map50_95(self) -> float:
        """The mAP score at OKS thresholds from `0.5` to `0.95`."""
        return _mean_valid_score(self.mAP_scores)

    @property
    def map50(self) -> float:
        """The mAP score at OKS threshold of `0.5`."""
        return float(self.mAP_scores[0])

    @property
    def map75(self) -> float:
        """The mAP score at OKS threshold of `0.75`."""
        return float(self.mAP_scores[5])

    mAP_scores: npt.NDArray[np.float64]
    ap_per_class: npt.NDArray[np.float64]
    oks_thresholds: npt.NDArray[np.float64]
    matched_classes: npt.NDArray[np.int32]
    medium_objects: KeyPointMeanAveragePrecisionResult | None = None
    large_objects: KeyPointMeanAveragePrecisionResult | None = None

    def __str__(self) -> str:
        """Format the scores like the `pycocotools` keypoint summary.

        Example:
            ```pycon
            >>> import numpy as np
            >>> import supervision as sv
            >>> from supervision.metrics import KeyPointMeanAveragePrecision
            >>> xy = np.array([[[10, 10], [60, 10], [35, 80]]], dtype=np.float32)
            >>> targets = sv.KeyPoints(xy=xy, class_id=np.array([0]))
            >>> predictions = sv.KeyPoints(
            ...     xy=xy, class_id=np.array([0]), detection_confidence=np.array([0.9])
            ... )
            >>> metric = KeyPointMeanAveragePrecision(sigmas=[0.5, 0.5, 0.5])
            >>> print(metric.update(predictions, targets).compute())
            Average Precision (AP) @[ OKS=0.50:0.95 | area=   all | ... ] = 1.000
            Average Precision (AP) @[ OKS=0.50      | area=   all | ... ] = 1.000
            Average Precision (AP) @[ OKS=0.75      | area=   all | ... ] = 1.000
            Average Precision (AP) @[ OKS=0.50:0.95 | area=medium | ... ] = 1.000
            Average Precision (AP) @[ OKS=0.50:0.95 | area= large | ... ] = -1.000

            ```
        """
        prefix = "Average Precision (AP) @[ OKS="
        suffix = f"maxDets={_KEYPOINT_MAX_DETECTIONS:>3d} ] = "
        lines = [
            f"{prefix}0.50:0.95 | area=   all | {suffix}{self.map50_95:.3f}",
            f"{prefix}0.50      | area=   all | {suffix}{self.map50:.3f}",
            f"{prefix}0.75      | area=   all | {suffix}{self.map75:.3f}",
        ]
        if self.medium_objects is not None:
            medium = self.medium_objects.map50_95
            lines.append(f"{prefix}0.50:0.95 | area=medium | {suffix}{medium:.3f}")
        if self.large_objects is not None:
            large = self.large_objects.map50_95
            lines.append(f"{prefix}0.50:0.95 | area= large | {suffix}{large:.3f}")
        return "\n".join(lines)

    def to_pandas(self) -> pd.DataFrame:
        """Convert the result to a pandas DataFrame.

        Returns:
            The result as a DataFrame.
        """
        return _scores_to_pandas(
            {"mAP@50:95": self.map50_95, "mAP@50": self.map50, "mAP@75": self.map75},
            [
                ("medium_objects", self.medium_objects),
                ("large_objects", self.large_objects),
            ],
        )

    def _get_plot_details(self, include_object_sizes: bool = True) -> PlotDetails:
        """Return bar-chart data for keypoint mAP scores.

        Args:
            include_object_sizes: When ``True``, include bars for the medium and
                large object-size categories.
        """
        metric_labels = ["mAP@50:95", "mAP@50", "mAP@75"]
        labels = list(metric_labels)
        values = [self.map50_95, self.map50, self.map75]
        colors = [LEGACY_COLOR_PALETTE[0]] * 3
        _append_object_size_plot_details(
            labels,
            values,
            colors,
            include_object_sizes=include_object_sizes,
            metric_labels=metric_labels,
            small_objects=None,
            medium_objects=self.medium_objects,
            large_objects=self.large_objects,
            value_getter=lambda result: [
                result.map50_95,
                result.map50,
                result.map75,
            ],
        )
        title = (
            "Keypoint Mean Average Precision\n"
            f"(class agnostic: {self.is_class_agnostic})"
        )
        return PlotDetails(labels=labels, values=values, colors=colors, title=title)

    def plot(self) -> None:
        """Plot the keypoint mAP results as a bar chart."""
        _show_bar_plot(self._get_plot_details())

Attributes

map50: float property

The mAP score at OKS threshold of 0.5.

map50_95: float property

The mAP score at OKS thresholds from 0.5 to 0.95.

map75: float property

The mAP score at OKS threshold of 0.75.

Methods:

__str__() -> str

Format the scores like the pycocotools keypoint summary.

Example

>>> import numpy as np
>>> import supervision as sv
>>> from supervision.metrics import KeyPointMeanAveragePrecision
>>> xy = np.array([[[10, 10], [60, 10], [35, 80]]], dtype=np.float32)
>>> targets = sv.KeyPoints(xy=xy, class_id=np.array([0]))
>>> predictions = sv.KeyPoints(
...     xy=xy, class_id=np.array([0]), detection_confidence=np.array([0.9])
... )
>>> metric = KeyPointMeanAveragePrecision(sigmas=[0.5, 0.5, 0.5])
>>> print(metric.update(predictions, targets).compute())
Average Precision (AP) @[ OKS=0.50:0.95 | area=   all | ... ] = 1.000
Average Precision (AP) @[ OKS=0.50      | area=   all | ... ] = 1.000
Average Precision (AP) @[ OKS=0.75      | area=   all | ... ] = 1.000
Average Precision (AP) @[ OKS=0.50:0.95 | area=medium | ... ] = 1.000
Average Precision (AP) @[ OKS=0.50:0.95 | area= large | ... ] = -1.000

Source code in src/supervision/metrics/keypoint_mean_average_precision.py

def __str__(self) -> str:
    """Format the scores like the `pycocotools` keypoint summary.

    Example:
        ```pycon
        >>> import numpy as np
        >>> import supervision as sv
        >>> from supervision.metrics import KeyPointMeanAveragePrecision
        >>> xy = np.array([[[10, 10], [60, 10], [35, 80]]], dtype=np.float32)
        >>> targets = sv.KeyPoints(xy=xy, class_id=np.array([0]))
        >>> predictions = sv.KeyPoints(
        ...     xy=xy, class_id=np.array([0]), detection_confidence=np.array([0.9])
        ... )
        >>> metric = KeyPointMeanAveragePrecision(sigmas=[0.5, 0.5, 0.5])
        >>> print(metric.update(predictions, targets).compute())
        Average Precision (AP) @[ OKS=0.50:0.95 | area=   all | ... ] = 1.000
        Average Precision (AP) @[ OKS=0.50      | area=   all | ... ] = 1.000
        Average Precision (AP) @[ OKS=0.75      | area=   all | ... ] = 1.000
        Average Precision (AP) @[ OKS=0.50:0.95 | area=medium | ... ] = 1.000
        Average Precision (AP) @[ OKS=0.50:0.95 | area= large | ... ] = -1.000

        ```
    """
    prefix = "Average Precision (AP) @[ OKS="
    suffix = f"maxDets={_KEYPOINT_MAX_DETECTIONS:>3d} ] = "
    lines = [
        f"{prefix}0.50:0.95 | area=   all | {suffix}{self.map50_95:.3f}",
        f"{prefix}0.50      | area=   all | {suffix}{self.map50:.3f}",
        f"{prefix}0.75      | area=   all | {suffix}{self.map75:.3f}",
    ]
    if self.medium_objects is not None:
        medium = self.medium_objects.map50_95
        lines.append(f"{prefix}0.50:0.95 | area=medium | {suffix}{medium:.3f}")
    if self.large_objects is not None:
        large = self.large_objects.map50_95
        lines.append(f"{prefix}0.50:0.95 | area= large | {suffix}{large:.3f}")
    return "\n".join(lines)

plot() -> None

Plot the keypoint mAP results as a bar chart.

Source code in src/supervision/metrics/keypoint_mean_average_precision.py

def plot(self) -> None:
    """Plot the keypoint mAP results as a bar chart."""
    _show_bar_plot(self._get_plot_details())

to_pandas() -> pd.DataFrame

Convert the result to a pandas DataFrame.

Returns:

Type	Description
DataFrame	The result as a DataFrame.

Source code in src/supervision/metrics/keypoint_mean_average_precision.py

def to_pandas(self) -> pd.DataFrame:
    """Convert the result to a pandas DataFrame.

    Returns:
        The result as a DataFrame.
    """
    return _scores_to_pandas(
        {"mAP@50:95": self.map50_95, "mAP@50": self.map50, "mAP@75": self.map75},
        [
            ("medium_objects", self.medium_objects),
            ("large_objects", self.large_objects),
        ],
    )

Comments