Ango Hub Docs
Open Ango HubContact iMerit
  • Ango Hub Documentation
  • Video Guides
  • Changelog
  • FAQs & Troubleshooting
  • All Keyboard and Mouse Shortcuts
  • Core Concepts
    • Assets
    • Attachments
    • Batches
    • Benchmarks
    • Category Schema (Ontologies)
    • Frame Interpolation
    • Geofencing
    • Idle Time Detection & Time Tracking
    • Instructions
    • Issues
      • Issue Error Codes
    • Label Validation
    • Labeler Performance
    • Labeling
    • Labeling Queue
    • Multiple Classification
    • Notifications
    • Organizations
    • Projects
    • Requeuing
    • Reviewing
    • Review Queue
    • Skipping
    • Stage History
    • Tasks
    • Usage
    • User Roles
    • Workflow
      • Complete
      • Consensus
      • Hold
      • Label
      • Logic
      • Plugin
      • Review
      • Start
      • Webhook
  • Labeling
    • Managing Users in Projects
      • Profile Page
    • Managing the Project Ontology
    • Labeling Editor Interface
      • Audio Labeling Editor
      • Image Labeling Editor
      • Video Labeling Editor
      • DICOM Labeling Editor
      • Medical Labeling Editor
        • 3D Bounding Box
        • Fill Between Slices
        • Island Tools
        • Line (Tape Measure)
        • Smoothing
      • PDF Labeling Editor
      • Text (NER) Labeling Editor
      • LLM Chat Labeling Editor
      • Markdown Labeling Editor
      • 3D Multi-Sensor Fusion Labeling Editor
    • Labeling Classes
      • Tools
        • Bounding Box
        • Brush
        • Entity
        • Message
        • Nested Classifications
        • PCT
        • PDF Tool
        • Point
        • Polygon
        • Polyline
        • Rotated Bounding Box
        • Segmentation
        • Spline
        • Voxel Brush
      • Classification
        • Checkbox
        • Multiple Dropdown
        • Radio
        • Rank
        • Single Dropdown
        • Text
        • Tree Dropdown Tools (Single and Multiple Selection)
      • Relation
        • Single Relation
        • Group Relation
    • Magnetic Lasso
    • Performance & Compatibility Considerations
  • Data
    • Data in Ango Hub
      • Embedding Private Bucket Files in MD Assets
    • Importing Assets
      • Asset Builder
      • Bundled Assets
        • Importing Multiple Images as One Multi-Page Asset
        • Importing Multiple Single-Frame DICOM Files as One Multi-Page Asset
        • Importing multiple DICOM files to be annotated and displayed at once
        • Importing Multiple Single-Frame DICOM Files as a DICOM Series
        • Importing Multiple Markdown Files as One Multi-Page Asset
      • File Explorer
      • Supported Asset File Types & Codecs
      • Importing Cloud (Remote) Assets
      • Importing From Local Machine
      • Creating and Importing LLM Chat Assets
      • Importing Data in the 3D Multi-Sensor Fusion Labeling Tool
      • Bulk Importing Markdown/HTML Assets
      • Importing Attachments during Asset Import
      • Importing Annotations during Asset Import
      • contextData: Adding Extra Data to Assets
      • Importing Reference Images as Overlay
      • Importing Reference Medical Data During Asset Import
    • Importing and Exporting Annotations
      • Importing Annotations
        • Ango Import Format
        • Importing Brush Traces
        • Importing NRRD Annotations
      • Exporting Annotations
        • Ango Export Format
          • Asset
            • Task
              • Tools
              • Classifications
              • Relations
          • Stage History
    • Adding and Managing LLMs
    • Storages
      • Set up a storage integration with Azure
      • Set up a storage integration with AWS S3
      • Set up a storage integration with MinIO and S3-compatible custom storage services
      • Set up a storage integration with GCP (Google Cloud Platform)
      • Set up CORS
      • Validating Storage Integrations
    • Purging Data from Ango Hub
  • Plugins
    • Overview of Plugins in Ango Hub
      • Installing Plugins
      • Plugin Setting Presets
      • Monitoring Plugin Progress
    • First-Party Plugins
      • Ango Export Converter Plugins
      • Asset Converter Plugins
      • Ango to Mask Converter
      • Batch Assignment
      • ChatGPT
      • Column-Agnostic Markdown Generator
      • CSV Export for Classification
      • DALL-E
      • DALL-E (Model Plugin)
      • File Explorer Plugin
      • General Object Detector
      • General Object Segmenter
      • Markdown Generator
      • One-Click Segmentation
      • Open World Object Detection
      • Optical Character Recognition
      • TPT Export
      • YOLO | Instance Segmentation
      • YOLO | Pose Estimation
      • YOLO | Object Detection
      • YOLO | Image Classification
    • Plugin Developer Documentation
      • Export Plugins
      • Batch Model Plugins
      • Model Plugins
      • File Explorer Plugins
      • Markdown Generator Plugins
      • Plugin Logger
      • [WIP] Deploying your Plugin
      • Plugin 'Host' Information
  • SDK
    • SDK Documentation
      • Project Level SDK Functions
        • add_members_to_project
        • assign_batches
        • assign_task
        • create_attachment
        • create_batch
        • create_issue
        • create_label_set
        • create_project
        • delete_issue
        • export
        • exportV3
        • get_assets
        • get_batches
        • get_issues
        • get_metrics
        • get_project
        • get_project_performance
        • get_task
        • get_tasks
        • get_task_history
        • import_labels
        • list_projects
        • requeue_tasks
        • rerun_webhook
        • update_workflow_stages
        • upload_files
        • upload_files_cloud
        • upload_files_with_asset_builder
        • upload_chat_assets
      • Organization Level SDK Functions
        • create_storage
        • delete_organization_invites
        • delete_organization_members
        • delete_storage
        • get_organization_invites
        • get_organization_members
        • get_storages
        • invite_members_to_org
        • update_organization_members_role
    • SDK - Useful Snippets
    • SDK Changelog
  • API
    • API Documentation
  • How-To
    • Add Members
      • Add multiple users to a project
    • Annotate
      • Annotate 3D Point Cloud Files on Ango Hub
      • Perform targeted OCR on images
    • Export Data
      • Automatically send Ango Hub Webhook contents to Google Sheets, Email, Slack, and more with Zapier
      • Download a JSON of your project ontology
      • Download DICOM Segmentation Masks
      • Download your annotations in the COCO, KITTI, or YOLO format
      • Download your Segmentation Masks
      • Get your export as separate JSON files for each asset
    • Manage a Project
      • Get your API Key
      • Get your Organization ID
      • Mute your notifications
      • Open an asset provided the Asset ID
      • Pre-label assets
      • Share a filtered view of the Tasks table with others
      • Transfer project ontologies between projects
      • Transfer project workflows between projects
    • Perform Model Evaluation on Ango Hub
  • Troubleshooting
    • I get a "0 Tasks Labeled" alert when trying to pre-label tasks
    • I get a 'The data couldn't be loaded properly' error when opening certain assets
    • I get a "Unknown Classification" warning when opening a task
  • Feature Availability Status for projects of the 3D Multi-Sensor Fusion type
  • Comparison between QuickServe and Ango Hub
  • Changes from Ango Hub Legacy
  • Video V2 Breaking Changes and Transition
  • Data Access, Storage, and Security
  • Two-Factor Authentication
  • Single Sign-On (SSO) Support
  • Customer Support
  • Ango Hub Status Page
Powered by GitBook
On this page
  • Creating a Model Plugin
  • Running a Model using the Plugin
  • From Settings
  • From the Labeling Editor
  1. Plugins
  2. Plugin Developer Documentation

Model Plugins

PreviousBatch Model PluginsNextFile Explorer Plugins

Last updated 1 year ago

Currently, the logger function is not yet available in v3 for model plugins. Using it may cause issues. We will remove this notice once it is available again.

With the Model custom plugin, you can add a quick way for users to run your model on their project assets.

When users click to run your plugin, you will get one of their assets, feed them to your model, and return the model's results as pre-annotations to their assets.

Creating a Model Plugin

Following the steps outlined , create a new plugin from the UI, choosing "Model" as the plugin type.

In the "Class Names" section, enter the names of the classes used by your model.

For example, assume that your model has three classes it returns, "person," "car," and "light." At a later stage, the user will be able to "link" (map) these classes to their own label categories representing the ideas of "person," "car" etc.

Then, create and run a Python script using the ModelPlugin class you can find in our imerit-ango Python package under imerit_ango.plugins.

You will need to add the imerit-ango package to your Python environment by running

pip install imerit-ango

Here is the class's documentation, and an example:

ModelPlugin

Parameters:

  • id: string

    • The plugin's ID. You may obtain this ID from the plugin's information box in the Development section of the Plugin page.

  • secret: string

    • The plugin's secret. You can think of this as a private key you'll need to be able to connect your script to the plugin. You may obtain this secret from the plugin's information box in the Development section of the Plugin page.

  • callback: Callable[[str, dict], Tuple[str, BytesIO]]

    • The callback function. This function will be run whenever a user asks for your model to be run using this plugin. More on the callback function below.

Callback Function

Parameters:

  • **data: dict

    • projectId: string

      • The ID of the project for which the plugin was run.

    • categorySchema: dict

      • The label categories that have been passed to this plugin when the plugin is being run.

      • For example, when creating the plugin, if in the "Class Names" section you had input the classes "Vehicle" and "Person", and when running, if they were both mapped to existing labeling tools, this is what you would get as input here:

      [{'schemaId': '797ea755f5693c0dc902558', 'modelClass': 'Vehicle'}, {'schemaId': '797ea755f5693c0dc902558', 'modelClass': 'Person'}]
    • assetId: str

      • The external ID of the asset to be sent to the model.

      • Example: my_external_id_1

    • apiKey: str

      • The API Key of the user running the plugin.

    • orgId: str

      • The Organization ID of the organization where the plugin is run.

    • runBy: str

      • The user ID of the user running the plugin.

    • session: str

    • logger: PluginLogger

    • batches: List[str]

    • configJSON: str

      • The config JSON your users will pass to you through the Config JSON text field when running the plugin. Warning: the JSON will be passed as a string so you will have to destringify it. Example code to obtain the original JSON as a Python object:

    def sample_callback(**data):
        config_str = data.get('configJSON')
        config = json.loads(config_str)

Returns:

  • annotation_json: dict

    • A message to show users when the plugin finishes running.

    • Example:

{
  "data": "https://angohub-public-assets.s3.eu-central-1.amazonaws.com/bb26ccc1-3a9f-4e41-bb01-df285b0c5bd5.jpg",
  "answer": {
    "objects": [...],
    "classifications": [...],
    "relations": [...]
  }
}

Sample Python Script

Find the sample code here:

Running a Model using the Plugin

Your Python script needs to be running for users to be able to run your plugin.

From Settings

  1. Ensure the plugin is activated in your organization.

  2. From your project, enter Settings -> Plugins, then click on Open on the plugin you wish to run. This dialog will pop up:

  3. Perform your class mappings, and provide the external ID of the asset you wish to annotate.

  4. Click on Run. Your code will be run on the asset you provided.

From the Labeling Editor

  1. Perform steps 1 and 2 from the instructions above, until you reach the Run Plugin dialog.

  2. Perform the class mappings for your project.

  3. From your project's Asset or Task lists, open a labeling task.

If you wish to one-time run the plugin with different mappings, click on the three dots on the right of the plugin's name, set your temporary mappings, and click on Run.

In this script, for the sake of brevity, instead of running the asset through a model, we add a bounding box to it as pre-label, "simulating" the model's results. We then upload these annotations to the user's assets with a "To-Do" status as prelabels using the .

Click on the "Save" icon, indicated by a floppy disk (). This will save the current mappings for the project, and will allow you to run the plugin directly from the labeling editor.

Click on the "Plugin" icon () at the top of the editor, and click on the name of the Model plugin you'd like to run on the current asset. The plugin will pre-label your asset with the model's results.

SDK
plugin-samples/model_plugin.py at main · angoai/plugin-samplesGitHub
Logo
in this section