YOLO | Instance Segmentation

Overview

The YOLO | Instance Segmentation plugin allows you to segment a variety of object types on image files in your project. The plugin will draw a segmentation around each object type you specify, using a segmentation tool of your choice.

Plugin Functionality

The YOLO Instance Segmentation Plugin allows you to automatically detect and segment instances in images using a pre-trained YOLO (You Only Look Once) model. It identifies and localizes objects by drawing polygon around detected instances and assigning them class labels with confidence scores.

Supported Data Types

• Image

Supported Annotation Tools

• Segmentation

• Polygon

• Brush

Plugin Configuration

The Overwrite setting in model plugins controls whether existing annotations are replaced or kept. When enabled, the plugin replaces all existing annotations with new model predictions; when disabled, it simply adds the new results without deleting what’s already there.

The Class Mapping setting defines how the model’s predicted classes are linked to your project’s label schema. Follow these steps to prepare your class mapping.

1. In the "Class Mapping" field, open the left dropdown, and pick from one of the classes the plugin can detect.

2. Open the right dropdown, and pick from one of the tools you have created in your project.

3. Click on the "plus" button to finalize the pairing. Now the object class and your bounding box tool are linked. The plugin will use the selected bounding box tool to label the selected category.

4. Starting again from Step 1, link as many tools to categories as needed.

YOLO | Instance Segmentation plugin integration to the workflow

You may vary a number of settings related to your export from the Config JSON field. Each option is detailed below:

{
  "only_mapped_classes": false,
  "threshold": 0.5,
  "polygon_simplification_tolerance": 0.8
}

• "only_mapped_classes": If this is set to true, the plugin will detect only the classes that are mapped in the class mapping. If it’s false, it will detect all supported classes, and any unmapped ones will be assigned to the "unknown" class.

  • Example:

    • "only_mapped_classes": true

    • "only_mapped_classes": false

• "threshold": The confidence score threshold for object detection. Detections with a confidence score below this value will be filtered out. A higher threshold reduces false positives but may miss low-confidence detections.

  • Example:

    • "threshold": 0.5

• "polygon_simplification_tolerance": Controls how much the output polygons are simplified to reduce complexity. Higher values produce simpler polygons with fewer vertices, while lower values preserve more detail.

  • Example:

    • "polygon_simplification_tolerance": 0.8

Detection Classes

The plugin can automatically detect the following object classes:

Traffic

person	bicycle	car
motorcycle	airplane	bus
truck	train	boat
traffic light	fire hydrant	stop sign
parking meter	bench

Animals

zebra	elephant	bird
cat	dog	horse
sheep	cow	giraffe

Clothes/Accessories

backpack	umbrella	handbag
tie	suitcase

Sports

frisbee	skis	snowboard
sports ball	kite	baseball bat
baseball glove	skateboard	surfboard
tennis racket

Kitchen

wine glass	cup	fork
knife	spoon	bowl

Food

banana	apple	sandwich
orange	broccoli	carrot
hot dog	pizza	donut
cake

Home

chair	couch	potted plant
bed	dining table	toilet
tv	microwave	oven
toaster	sink	refrigerator
book	clock	vase
scissors	teddy bear	hair drier
toothbrush

Electronics

laptop	mouse	remote
keyboard	cell phone