Skip to content

Use Compact Masks for Memory-Efficient Segmentation

CompactMask stores each instance mask as a run-length encoding of its bounding-box crop rather than a full (H, W) boolean frame. For high-resolution images with many sparse masks this can reduce memory from tens of gigabytes to tens of megabytes, and eliminates full-frame decode work in annotators that only need the cropped region.

Note

sv.mask_to_xyxy keeps supervision's inclusive max-coordinate convention for compatibility with CompactMask and current box-based adapters. Use sv.mask_to_roi when you need exclusive slice bounds for NumPy indexing or crop extraction.

This guide covers the main integration points:

  1. Ingesting COCO RLE payloads directly as CompactMask
  2. Parsing Roboflow Inference results without a dense stack
  3. Skipping mask materialisation for box/label annotators
  4. Merging mixed dense and compact detections
  5. Merging overlapping segmentation predictions

Ingest COCO RLE Payloads

If your model or API returns masks in the COCO RLE format ({"size": [H, W], "counts": "..."}) you can convert them directly to CompactMask without allocating an (N, H, W) boolean array:

import numpy as np
import supervision as sv
from supervision.detection.compact_mask import CompactMask

# Example: two COCO RLE masks for a 720×1280 frame.
# Replace the counts strings with actual compressed RLE payloads from your
# model or API — e.g., from pycocotools mask.encode() or an Inference response.
rles = [
    {"size": [720, 1280], "counts": "YOUR_RLE_COUNTS_STRING_HERE"},
    {"size": [720, 1280], "counts": "YOUR_RLE_COUNTS_STRING_HERE"},
]
xyxy = np.array(
    [
        [100.0, 50.0, 400.0, 300.0],
        [500.0, 200.0, 900.0, 600.0],
    ]
)

compact = CompactMask.from_coco_rle(rles, xyxy, image_shape=(720, 1280))

detections = sv.Detections(
    xyxy=xyxy,
    mask=compact,
    class_id=np.array([0, 1]),
)

from_coco_rle uses run-length arithmetic scoped to each bounding box so no dense pixel array is ever created. Uncompressed integer count lists are also accepted in place of compressed strings.

Parse Inference Results

Detections.from_inference accepts a compact_masks=True flag that routes the Roboflow RLE payload through CompactMask.from_coco_rle instead of decoding to a dense stack:

import supervision as sv

# result: a Roboflow Inference v2 response dict with instance masks.
detections = sv.Detections.from_inference(result, compact_masks=True)

from supervision.detection.compact_mask import CompactMask

assert isinstance(detections.mask, CompactMask)

Warning

compact_masks=True crops each mask to its detector bounding box. Pixels outside the box are silently dropped. For masks that extend meaningfully beyond the reported bounding box, use the default compact_masks=False (dense decode) to preserve all pixels.

To convert an existing dense-mask Detections to compact at any point:

detections_compact = detections.to_compact_masks()

Skip Unnecessary Materialisation

Annotators that do not draw masks (box, label, circle, ellipse, trace, keypoint) expose requires_mask = False. Integrations can branch on this flag to avoid decoding compact or RLE masks before annotation:

import supervision as sv

annotators = [
    sv.BoxAnnotator(),
    sv.LabelAnnotator(),
    sv.MaskAnnotator(),  # requires_mask = True
]

for ann in annotators:
    if ann.requires_mask:
        # Annotator reads mask pixels — CompactMask decodes lazily per crop.
        scene = ann.annotate(scene, detections)
    else:
        # Annotator ignores masks — strip mask field to eliminate any decode cost.
        det_no_mask = sv.Detections(
            xyxy=detections.xyxy,
            confidence=detections.confidence,
            class_id=detections.class_id,
        )
        scene = ann.annotate(scene, det_no_mask)

Annotators that set requires_mask = True: MaskAnnotator, PolygonAnnotator, HaloAnnotator.

All others default to requires_mask = False.

Note

PolygonAnnotator and MaskAnnotator both operate directly on CompactMask without materialising the full (N, H, W) frame — passing compact detections to them is already efficient.

Merge Mixed Detections

When merging Detections objects that mix dense ndarray masks and CompactMask instances, Detections.merge converts dense inputs to CompactMask automatically. No full (N, H, W) stack is allocated:

import numpy as np
import supervision as sv
from supervision.detection.compact_mask import CompactMask

H, W = 720, 1280

# Compact detections from an RLE-based source.
# Replace the counts string with a real compressed RLE payload from your model or API.
rles = [{"size": [H, W], "counts": "YOUR_RLE_COUNTS_STRING_HERE"}]
xyxy_a = np.array([[100.0, 50.0, 400.0, 300.0]])
cm = CompactMask.from_coco_rle(rles, xyxy_a, image_shape=(H, W))
det_a = sv.Detections(xyxy=xyxy_a, mask=cm, class_id=np.array([0]))

# Dense detections from a different source.
masks_b = np.zeros((1, H, W), dtype=bool)
masks_b[0, 200:400, 500:800] = True
xyxy_b = np.array([[500.0, 200.0, 799.0, 399.0]])
det_b = sv.Detections(xyxy=xyxy_b, mask=masks_b, class_id=np.array([1]))

# Output is CompactMask regardless of input order.
merged = sv.Detections.merge([det_a, det_b])
assert isinstance(merged.mask, CompactMask)
assert len(merged) == 2

Merge rules:

Inputs Output mask type
All CompactMask CompactMask
Mixed CompactMask + dense ndarray CompactMask
All dense ndarray ndarray (backward compatible)

All CompactMask inputs must share the same image_shape; mismatches raise ValueError.

Merge Overlapping Predictions

Detections.with_nmm and mask_non_max_merge keep compact masks compressed while forming mask unions. This applies to both the candidate updated during greedy matching and the final merged detection. They combine foreground run intervals, without allocating a full (N, H, W) mask stack or a dense union image.

Version requirement

This large-canvas example requires supervision>=0.31.0. Earlier versions materialize full-image dense union arrays while running CompactMask NMM, which can exhaust memory for a canvas this large.

import numpy as np
import supervision as sv
from supervision.detection.compact_mask import CompactMask

# Encode model output in tile coordinates, then relocate it to a large image.
tile_masks = np.ones((2, 32, 32), dtype=bool)
tile_boxes = np.array([[0, 0, 31, 31], [0, 0, 31, 31]])
masks = CompactMask.from_dense(tile_masks, tile_boxes, (32, 32))
masks = masks.with_offset(50000, 60000, new_image_shape=(100000, 100000))
detections = sv.Detections(
    xyxy=masks.bbox_xyxy.astype(float),
    mask=masks,
    confidence=np.array([0.9, 0.8]),
    class_id=np.array([0, 0]),
)
merged = detections.with_nmm(threshold=0.5)
assert len(merged) == 1
assert isinstance(merged.mask, CompactMask)
assert merged.mask.area.tolist() == [1024]

The same path is used by InferenceSlicer(compact_masks=True, overlap_filter=sv.OverlapFilter.NON_MAX_MERGE). The mask union preserves all stored foreground, including pixels outside the detection boxes. Matching thresholds, class grouping, confidence aggregation and winner metadata behave as with dense masks.

Union cost depends on the number of foreground column intervals (K), determined by mask fragmentation and the columns foreground runs cross, not by the logical canvas area. Highly fragmented masks can take more time and temporary memory than a dense union, including with large crops. Holding the encoded crops fixed keeps this interval cost independent of canvas size. Overlap evaluation still decodes overlapping crops and allocates pairwise overlap arrays, so a very large or loose crop can still be expensive. This change removes the additional full-image union allocations.

A reproducible benchmark varying canvas size and mask fragmentation is included in the repository. From a checkout, run python examples/compact_mask/benchmark_nmm.py --canvas 512 2048 4096. To expose fragmentation overhead, run python examples/compact_mask/benchmark_nmm.py --crop-size 200 --canvas 512 --pattern checkerboard: twelve masks in four groups of three duplicates increased peak traced NMM allocation from 4.34 MiB to 7.79 MiB (1.79×) when comparing the pre-change dense-union implementation with the interval implementation (maximum of three repeats; input construction excluded). This measures the complete NMM call, not just union storage; see the benchmark results for the matching solid-mask case.

Performance Notes

These estimates apply to the parsing and annotation stage, not end-to-end pipeline FPS. Model inference typically dominates total runtime.

Optimisation Realistic gain Applies when
from_coco_rle ingestion 25–60% faster parse Full-frame COCO RLE payload; current dense decode path
MaskAnnotator ROI blending 10–35% faster annotation Many small, sparse masks on high-res frames
PolygonAnnotator crop path 15–45% faster polygon draw Many compact masks; full-frame materialise was the bottleneck
Mixed-mask merge 5–20% faster merge Mix of compact and dense sources (e.g. multi-camera stitch)

Upper-end gains assume: ≥1080p frames, tens to hundreds of instances, masks covering less than ~20% of total pixels.

API Reference

Comments