Auto Suggestion

Auto Suggestion is an AI-powered labeling assistance feature letting you create complex segmentations, polygons, and polylines with a single click.

The AI model used to power this feature (a quantized version of SAM2) is downloaded to your browser and run locally, meaning that no data is ever sent to any server.

You can also upload your own custom model weights, and Auto Suggestion will use them in your project.

Supported

Labeling Tools

• Segmentation

• Polygon

• Polyline

Asset Types

• Images (single and image sequences)

How to use Auto Suggestion

In the labeling editor, click on the tool you'd like to use Auto Suggestion with. The Auto Suggestion button will appear:

Click on the Auto Suggestion button. A message will appear at the bottom of the screen informing you that the machine learning model used to power this feature is being downloaded. You may keep using Ango Hub normally as the model is downloaded in the background.

If Ango Hub detects that you have already downloaded the model previously by using this feature, and your browser has not deleted it, this step will be skipped.

Once the model has been downloaded, the image will be processed by the model. This is done locally in your browser.

Once processing is completed, you may hover over the image using your mouse cursor. As you hover, the model will automatically detect objects under your cursor. Clicking on the object will finalize the annotation.

Once you are done creating annotations, click on the Auto Suggestion button again to turn it off.

Use your Own Model

From your project settings, navigate to the "Auto Suggestion" section:

Overview

The preparation process involves four main steps:

1. Prepare model – Load SAM2 and its weights.

2. Export ONNX – Match model inputs and outputs.

3. Upload – Upload the exported ONNX files.

4. Use in editor – Initialize and decode points for use in the application.

Step 1: Prepare Model

Load the SAM2 model along with its pre-trained weights.

Encoder/Decoder ONNX Expectations

• Encoder input:

  • Image tensor of shape [1, 3, H, W] (default: H = W = 1024)

• Encoder outputs:

  • high_res_feats_0

  • high_res_feats_1

  • image_embed

    (Dimensions must align with decoder expectations.)

• Decoder inputs:

  Must match the app’s expected structure:

  • image_embed

  • high_res_feats_0, high_res_feats_1

  • point_coords [1, 1, 2]

  • point_labels [1, 1]

  • mask_input [1, 1, 256, 256]

  • has_mask_input [1]

  • orig_im_size [2]

• Decoder outputs:

  • masks

  • iou_predictions

  • low_res_masks

From the Auto Suggestion settings page, download:

• the Python export script

• requirements.txt

Step 2: Export ONNX

Once the model is oaded, export the encoder and decoder to the ONNX format. Ensure the inputs and outputs match the specifications listed above.

Step 3: Upload

Use the upload buttons in the Auto Suggestion section of the project settings to load your exported ONNX files.

Step 4: Activate

Enable the "Use Custom Segmentation Models" toggle to use your custom model in the project.

That's it - you can now use the Auto Suggestion feature in your project and it'll use your model.