One-Click Segmentation

Overview

The One-Click Segmentation plugin allows you to create complex and detailed segmentations of an object starting with one point or one bounding box.

You may find a full slideshow tutorial on how to install and use the plugin in the following PDF:

3MB

One Click Segmentation.pdf

PDF

Open

Plugin Functionality

The One Click Segmentation plugin enables automatic segmentation generation using the SAM (Segment Anything Model). It produces high-quality segmentation annotations from a single point or a bounding box prompt, dramatically speeding up the annotation process for image assets.

When a user provides a prompt, either by clicking on an object (point prompt) or drawing a bounding box (box prompt), the plugin automatically predicts and creates the corresponding segmentation mask. The generated mask is then converted into a segmentation, polygon or brush annotation within Ango Hub.

The plugin will take as input one of the following:

A point
A bounding box

and will output either:

A segmentation object
A polygon object
A brush object

Supported Data Types

Image
Video
DICOM

Supported Annotation Tools

Segmentation
Polygon
Brush

Project Prerequisites

Your project's category schema should contain your segmentation or polygon classes paired with a point or bounding box class. The paired classes must have the exact same name.

For example, if you wish to draw a segmentation around dogs starting from points, you must create, in your project, a segmentation class named "Dogs" (or any other name of your choice), and a point class with the same exact name (in our case "Dogs").

This will tell the plugin that we want to use the points created with the point "Dogs" class to create segmentations with the segmentation class "Dogs".

Sample Category Schema

{
  "tools": [
    {
      "schemaId": "0953cd0bf1ef372c6b62521",
      "tool": "segmentation",
      "title": "Vehicle",
      "required": false,
      "classifications": [],
      "multiple": false,
      "color": "#f44336",
      "shortcutKey": "1"
    },
    {
      "schemaId": "949ffbda2006534261e9619",
      "tool": "polygon",
      "title": "Animal",
      "required": false,
      "classifications": [],
      "multiple": false,
      "color": "#03a9f4",
      "shortcutKey": "2"
    },
    {
      "schemaId": "40d993b02b7cc99c8c9c508",
      "tool": "point",
      "title": "Vehicle",
      "required": false,
      "classifications": [],
      "multiple": false,
      "color": "#f44336",
      "shortcutKey": "3"
    },
    {
      "schemaId": "eeb44aa688219cd6470d893",
      "tool": "point",
      "title": "Animal",
      "required": false,
      "classifications": [],
      "multiple": false,
      "color": "#03a9f4",
      "shortcutKey": "4"
    }
  ],
  "classifications": [],
  "relations": []
}

Plugin Usage - Through Workflow

A suggested workflow usage is the following:

where in the first labeling stage ("Point Labeling") annotators create the points (or bounding boxes) which will be used by the plugin, and in the second labeling stage annotators fix the results created by the plugin.

When using the plugin through the workflow's Plugin stage, please do not forget to configure the plugin through its Config JSON in its stage settings, following the config JSON as detailed in the section below.

Sample Workflow

[
  {
    "id": "Start",
    "type": "Start",
    "name": "Start",
    "position": {
      "x": -229.79769433425196,
      "y": 4.381554674015661
    },
    "assignedTo": [],
    "logic": {
      "conditions": []
    },
    "readOnly": false,
    "autoForward": true,
    "rememberAssignee": true,
    "next": [
      "ce8b34c8-12a6-4fb2-b4a6-406c0ddc1e1b"
    ]
  },
  {
    "id": "Complete",
    "type": "Complete",
    "name": "Complete",
    "position": {
      "x": 976.9769433425197,
      "y": 42.560803682283336
    },
    "assignedTo": [],
    "logic": {
      "conditions": []
    },
    "readOnly": false,
    "autoForward": true,
    "rememberAssignee": true,
    "next": []
  },
  {
    "id": "ce8b34c8-12a6-4fb2-b4a6-406c0ddc1e1b",
    "type": "Label",
    "name": "Point Labeling",
    "position": {
      "x": 62,
      "y": 4
    },
    "assignedTo": [],
    "logic": {
      "conditions": []
    },
    "readOnly": false,
    "autoForward": true,
    "rememberAssignee": true,
    "consensusId": null,
    "consensusConfig": {
      "version": 2
    },
    "next": [
      "e2780869-1455-4098-9356-d32cf7cc4f23"
    ]
  },
  {
    "id": "e2780869-1455-4098-9356-d32cf7cc4f23",
    "type": "Plugin",
    "name": "Plugin",
    "position": {
      "x": 362,
      "y": -128.50836629015785
    },
    "assignedTo": [],
    "logic": {
      "conditions": []
    },
    "readOnly": false,
    "autoForward": true,
    "rememberAssignee": true,
    "pluginId": "672b8f8be26866bd2469649c",
    "consensusId": null,
    "consensusConfig": {
      "version": 2
    },
    "next": [
      "3e9167d4-1271-41cd-bb61-39f29f851eb9"
    ]
  },
  {
    "id": "3e9167d4-1271-41cd-bb61-39f29f851eb9",
    "type": "Label",
    "name": "Label",
    "position": {
      "x": 673.9769433425196,
      "y": 14.676086969685343
    },
    "assignedTo": [],
    "logic": {
      "conditions": []
    },
    "readOnly": false,
    "autoForward": true,
    "rememberAssignee": true,
    "consensusId": null,
    "consensusConfig": {
      "version": 2
    },
    "next": [
      "Complete"
    ]
  }
]

Plugin Usage - Through the Labeling Editor

In the labeling editor, draw a rough bounding box or a point on the element:

Click on "Save" or press S on your keyboard to save.

Open the Plugin dropdown.

If you have saved a configuration preset for the plugin, click on the plugin name. The plugin will run. If you have not saved a configuration preset, click on the three dots to its right, configure the plugin as necessary (see section below), and click on Run. The plugin will create the polygon/segmentation, which you can now fix and edit as necessary.

Plugin Configuration

The Overwrite setting in model plugins controls whether existing annotations are replaced or kept. When enabled, the plugin replaces all existing annotations with new model predictions; when disabled, it simply adds the new results without deleting what’s already there.

You may vary a number of settings related to your export from the Config JSON field. Each option is detailed below:

{
  "model_name": "sam_base",
  "prompt_mode": "bounding-box",
  "threshold": 0.8,
  "polygon_simplification_tolerance": 0.9
}

"model_name": Specifies which Segment Anything Model (SAM) variant to use for generating segmentations.
- Options:
  - "sam_base"
  - "sam_large"

"prompt_mode": Defines the type of user input (prompt) used to guide the segmentation.
- Options:
  - "point"
  - "bounding-box"

"threshold": Sets the confidence threshold for accepting segmentation predictions. Predictions with a confidence score below this value are filtered out.
- Example:
  - "threshold": 0.8

"polygon_simplification_tolerance": Controls how much the output polygons are simplified to reduce complexity. Higher values produce simpler polygons with fewer vertices, while lower values preserve more detail.
- Example:
  - "polygon_simplification_tolerance": 0.9

Last updated 11 hours ago