# YOLO | Instance Segmentation

## Overview

The YOLO | Instance Segmentation plugin allows you to segment a variety of object types on image files in your project. The plugin will draw a segmentation around each object type you specify, using a segmentation tool of your choice.

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXelwOD-gaQE6oYUFd18BXBWm8C6Yb7Afi0phHsT6RoR2bkbyhl2dlw53JY-ugMzYvybBqsPIN8wTHiMrvVtlN71Cz__uQqjDgMiLrp-s4seKwFJywJbyyWqjKcvMH0VanPtqaG6qA?key=cpQYlDRel4R0-nVf-0AtAQ" alt=""><figcaption></figcaption></figure>

## Plugin Functionality

The YOLO Instance Segmentation Plugin allows you to automatically detect and segment instances in images using a pre-trained YOLO (You Only Look Once) model. It identifies and localizes objects by drawing polygon around detected instances and assigning them class labels with confidence scores.

<figure><img src="https://3895963154-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FTcOUG6rfWxqGM0N4db2P%2Fuploads%2FnbRfdRWHYybp08xANcQ8%2Fimage.png?alt=media&#x26;token=203a86ad-45a6-425b-87c2-459df665418c" alt=""><figcaption></figcaption></figure>

### Supported Data Types <a href="#supported-data-types" id="supported-data-types"></a>

* Image

### Supported Annotation Tools <a href="#supported-annotation-tools" id="supported-annotation-tools"></a>

* Segmentation
* Polygon
* Brush

## Plugin Configuration <a href="#plugin-configuration" id="plugin-configuration"></a>

The **Overwrite** setting in model plugins controls whether existing annotations are replaced or kept. When enabled, the plugin replaces all existing annotations with new model predictions; when disabled, it simply adds the new results without deleting what’s already there.

The **Class Mapping** setting defines how the model’s predicted classes are linked to your project’s label schema. Follow these steps to prepare your class mapping.

1. In the "Class Mapping" field, open the left dropdown, and pick from one of the classes the plugin can detect.
2. Open the right dropdown, and pick from one of the tools you have created in your project.
3. Click on the "plus" button to finalize the pairing. Now the object class and your bounding box tool are linked. The plugin will use the selected bounding box tool to label the selected category.
4. Starting again from Step 1, link as many tools to categories as needed.

<figure><img src="https://3895963154-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FTcOUG6rfWxqGM0N4db2P%2Fuploads%2FfQ1gWPGiDSxSHRT0V6S4%2Fimage.png?alt=media&#x26;token=9f2c8453-9bf4-4175-aa0d-8aa815e4c35b" alt=""><figcaption><p>YOLO | Instance Segmentation plugin integration to the workflow</p></figcaption></figure>

You may vary a number of settings related to your export from the **Config JSON** field. Each option is detailed below:

```json
{
  "only_mapped_classes": false,
  "threshold": 0.5,
  "polygon_simplification_tolerance": 0.8
}
```

* **"only\_mapped\_classes":** If this is set to true, the plugin will detect only the classes that are mapped in the class mapping. If it’s false, it will detect all supported classes, and any unmapped ones will be assigned to the "**unknown"** class.
  * Example:
    * <kbd>"only\_mapped\_classes": true</kbd>
    * <kbd>"only\_mapped\_classes": false</kbd>&#x20;

* **"threshold":** The confidence score threshold for object detection. Detections with a confidence score below this value will be filtered out. A higher threshold reduces false positives but may miss low-confidence detections.
  * Example:
    * <kbd>"threshold": 0.5</kbd>

* **"polygon\_simplification\_tolerance":** Controls how much the output polygons are simplified to reduce complexity. Higher values produce simpler polygons with fewer vertices, while lower values preserve more detail.
  * Example:
    * <kbd>"polygon\_simplification\_tolerance": 0.8</kbd>&#x20;

## Detection Classes

The plugin can automatically detect the following object classes:

**Traffic**

| person        | bicycle      | car       |
| ------------- | ------------ | --------- |
| motorcycle    | airplane     | bus       |
| truck         | train        | boat      |
| traffic light | fire hydrant | stop sign |
| parking meter | bench        |           |

**Animals**

| zebra | elephant | bird    |
| ----- | -------- | ------- |
| cat   | dog      | horse   |
| sheep | cow      | giraffe |

**Clothes/Accessories**

| backpack | umbrella | handbag |
| -------- | -------- | ------- |
| tie      | suitcase |         |

**Sports**

| frisbee        | skis       | snowboard    |
| -------------- | ---------- | ------------ |
| sports ball    | kite       | baseball bat |
| baseball glove | skateboard | surfboard    |
| tennis racket  |            |              |

**Kitchen**

| wine glass | cup   | fork |
| ---------- | ----- | ---- |
| knife      | spoon | bowl |

**Food**

| banana  | apple    | sandwich |
| ------- | -------- | -------- |
| orange  | broccoli | carrot   |
| hot dog | pizza    | donut    |
| cake    |          |          |

**Home**

| chair      | couch        | potted plant |
| ---------- | ------------ | ------------ |
| bed        | dining table | toilet       |
| tv         | microwave    | oven         |
| toaster    | sink         | refrigerator |
| book       | clock        | vase         |
| scissors   | teddy bear   | hair drier   |
| toothbrush |              |              |

**Electronics**

| laptop   | mouse      | remote |
| -------- | ---------- | ------ |
| keyboard | cell phone |        |
