Ango Hub Docs
Open Ango HubContact iMerit
  • Ango Hub Documentation
  • Video Guides
  • Changelog
  • FAQs & Troubleshooting
  • All Keyboard and Mouse Shortcuts
  • Core Concepts
    • Assets
    • Attachments
    • Batches
    • Benchmarks
    • Category Schema (Ontologies)
    • Frame Interpolation
    • Geofencing
    • Idle Time Detection & Time Tracking
    • Instructions
    • Issues
      • Issue Error Codes
    • Label Validation
    • Labeler Performance
    • Labeling
    • Labeling Queue
    • Multiple Classification
    • Notifications
    • Organizations
    • Projects
    • Requeuing
    • Reviewing
    • Review Queue
    • Skipping
    • Stage History
    • Tasks
    • Usage
    • User Roles
    • Workflow
      • Complete
      • Consensus
      • Hold
      • Label
      • Logic
      • Plugin
      • Review
      • Start
      • Webhook
  • Labeling
    • Managing Users in Projects
      • Profile Page
    • Managing the Project Ontology
    • Labeling Editor Interface
      • Audio Labeling Editor
      • Image Labeling Editor
      • Video Labeling Editor
      • DICOM Labeling Editor
      • Medical Labeling Editor
        • 3D Bounding Box
        • Fill Between Slices
        • Island Tools
        • Line (Tape Measure)
        • Smoothing
      • PDF Labeling Editor
      • Text (NER) Labeling Editor
      • LLM Chat Labeling Editor
      • Markdown Labeling Editor
      • 3D Multi-Sensor Fusion Labeling Editor
    • Labeling Classes
      • Tools
        • Bounding Box
        • Brush
        • Entity
        • Message
        • Nested Classifications
        • PCT
        • PDF Tool
        • Point
        • Polygon
        • Polyline
        • Rotated Bounding Box
        • Segmentation
        • Spline
        • Voxel Brush
      • Classification
        • Checkbox
        • Multiple Dropdown
        • Radio
        • Rank
        • Single Dropdown
        • Text
        • Tree Dropdown Tools (Single and Multiple Selection)
      • Relation
        • Single Relation
        • Group Relation
    • Magnetic Lasso
    • Performance & Compatibility Considerations
  • Data
    • Data in Ango Hub
      • Embedding Private Bucket Files in MD Assets
    • Importing Assets
      • Asset Builder
      • Bundled Assets
        • Importing Multiple Images as One Multi-Page Asset
        • Importing Multiple Single-Frame DICOM Files as One Multi-Page Asset
        • Importing multiple DICOM files to be annotated and displayed at once
        • Importing Multiple Single-Frame DICOM Files as a DICOM Series
        • Importing Multiple Markdown Files as One Multi-Page Asset
      • File Explorer
      • Supported Asset File Types & Codecs
      • Importing Cloud (Remote) Assets
      • Importing From Local Machine
      • Creating and Importing LLM Chat Assets
      • Importing Data in the 3D Multi-Sensor Fusion Labeling Tool
      • Bulk Importing Markdown/HTML Assets
      • Importing Attachments during Asset Import
      • Importing Annotations during Asset Import
      • contextData: Adding Extra Data to Assets
      • Importing Reference Images as Overlay
      • Importing Reference Medical Data During Asset Import
    • Importing and Exporting Annotations
      • Importing Annotations
        • Ango Import Format
        • Importing Brush Traces
        • Importing NRRD Annotations
      • Exporting Annotations
        • Ango Export Format
          • Asset
            • Task
              • Tools
              • Classifications
              • Relations
          • Stage History
    • Adding and Managing LLMs
    • Storages
      • Set up a storage integration with Azure
      • Set up a storage integration with AWS S3
      • Set up a storage integration with MinIO and S3-compatible custom storage services
      • Set up a storage integration with GCP (Google Cloud Platform)
      • Set up CORS
      • Validating Storage Integrations
    • Purging Data from Ango Hub
  • Plugins
    • Overview of Plugins in Ango Hub
      • Installing Plugins
      • Plugin Setting Presets
      • Monitoring Plugin Progress
    • First-Party Plugins
      • Ango Export Converter Plugins
      • Asset Converter Plugins
      • Ango to Mask Converter
      • Batch Assignment
      • ChatGPT
      • Column-Agnostic Markdown Generator
      • CSV Export for Classification
      • DALL-E
      • DALL-E (Model Plugin)
      • File Explorer Plugin
      • Markdown Generator
      • One-Click Segmentation
      • Open World Object Detection
      • Optical Character Recognition
      • TPT Export
      • YOLO | Instance Segmentation
      • YOLO | Pose Estimation
      • YOLO | Object Detection
      • YOLO | Image Classification
    • Plugin Developer Documentation
      • Export Plugins
      • Batch Model Plugins
      • Model Plugins
      • File Explorer Plugins
      • Markdown Generator Plugins
      • Plugin Logger
      • [WIP] Deploying your Plugin
      • Plugin 'Host' Information
  • SDK
    • SDK Documentation
      • Project Level SDK Functions
        • add_members_to_project
        • assign_batches
        • assign_task
        • create_attachment
        • create_batch
        • create_issue
        • create_label_set
        • create_project
        • delete_issue
        • export
        • exportV3
        • get_assets
        • get_batches
        • get_issues
        • get_metrics
        • get_project
        • get_project_performance
        • get_task
        • get_tasks
        • get_task_history
        • import_labels
        • list_projects
        • requeue_tasks
        • rerun_webhook
        • update_workflow_stages
        • upload_files
        • upload_files_cloud
        • upload_files_with_asset_builder
        • upload_chat_assets
        • upload_instructions
      • Organization Level SDK Functions
        • create_storage
        • delete_organization_invites
        • delete_organization_members
        • delete_storage
        • get_organization_invites
        • get_organization_members
        • get_storages
        • invite_members_to_org
        • update_organization_members_role
    • SDK - Useful Snippets
    • SDK Changelog
  • API
    • API Documentation
  • How-To
    • Add Members
      • Add multiple users to a project
    • Annotate
      • Annotate 3D Point Cloud Files on Ango Hub
      • Perform targeted OCR on images
    • Export Data
      • Automatically send Ango Hub Webhook contents to Google Sheets, Email, Slack, and more with Zapier
      • Download a JSON of your project ontology
      • Download DICOM Segmentation Masks
      • Download your annotations in the COCO, KITTI, or YOLO format
      • Download your Segmentation Masks
      • Get your export as separate JSON files for each asset
    • Manage a Project
      • Get your API Key
      • Get your Organization ID
      • Mute your notifications
      • Open an asset provided the Asset ID
      • Pre-label assets
      • Share a filtered view of the Tasks table with others
      • Transfer project ontologies between projects
      • Transfer project workflows between projects
    • Perform Model Evaluation on Ango Hub
  • Troubleshooting
    • I get a "0 Tasks Labeled" alert when trying to pre-label tasks
    • I get a 'The data couldn't be loaded properly' error when opening certain assets
    • I get a "Unknown Classification" warning when opening a task
  • Other
    • Feature Availability Status for projects of the 3D Multi-Sensor Fusion type
    • Comparison between QuickServe and Ango Hub
    • Changes from Ango Hub Legacy
    • Video V2 Breaking Changes and Transition
    • Data Access, Storage, and Security
    • Two-Factor Authentication
    • Single Sign-On (SSO) Support
    • Customer Support
    • Ango Hub Status Page
    • Features Unavailable in Private Cloud and On-Premise Deployments of Ango Hub
Powered by GitBook
On this page
  • General Structure
  • Coordinates in the Ango Annotation Format
  • Changes in Video Assets
  • Polygon
  • Sample
  • Properties
  • Segmentation
  • Sample
  • Properties
  • Bounding Box
  • Sample
  • Properties
  • Rotated Bounding Box
  • Rotated Bounding Box Angles
  • Sample
  • Properties
  • Polygon
  • Sample
  • Properties
  • Polyline
  • Sample
  • Properties
  • Medical Line Tool
  • Sample
  • Properties
  • Spline
  • Sample
  • Properties
  • Point
  • Sample
  • Properties
  • PDF
  • Sample
  • Properties
  • PCT
  • Sample
  • Entity (NER)
  • Sample
  • Properties
  • Special Considerations for Video Assets
  • interpolationStopped
  1. Data
  2. Importing and Exporting Annotations
  3. Exporting Annotations
  4. Ango Export Format
  5. Asset
  6. Task

Tools

Tools fields in Ango Annotation Format exports.

PreviousTaskNextClassifications

Last updated 2 months ago

The tools object is part of the object in the Ango Annotation Format. It contains all annotation data pertaining to labels created with tools, e.g. not classifications, relations, or brushes.

Annotations created with the following annotation tools will appear in the tools object:

Annotations created with the and tools are not saved here. They are instead saved, respectively, in the brushDataUrl and the medicalBrushDataUrl objects within the object. See for more.

That said, if the voxel brush in your project has nested classifications, the nested classification contents will appear here.

The page property, in the export, is 0-indexed, while in the Ango Hub user interface it is 1-indexed.

What this means in practice is that pages start at 0 in the export, but 1 in the UI.

So for example, if, in the UI, you create a bounding box at page "12", it will appear under page "11" in the export.

General Structure

"tools" : [{
    bounding-box OR rotated-bounding-box OR polygon OR polyline OR segmentation OR ner OR point OR pdf OR spline// object, tool coordinate info
    objectId // string, ID of the object
    schemaId, // string, ID of the labeling tool in the project
    classifications, // list, nested classifications
    title // string, title of the tool
    page // int, page where the object is located (only multi-page, non-video assets)
    medical-brush, // bool, true if the current tool is a medical brush
    },
    { ... }
]

Coordinates in the Ango Annotation Format

This is the coordinate system used when determining the position of points in the Ango export.

The origin point lies at the top left corner, and each point has a (X, Y) coordinate pair determining its position, in pixels, on the image.

Changes in Video Assets

Video assets have a slightly different format for the "tools" property:

"tools" : [{
    schemaId, // string, ID of the labeling tool in the project
    objectId, // string, ID of the object
    segments, // list, start and end frames of segments for this class
    keyFrames, // object, list of keyframes for this annotation for this class
    bounding-box OR rotated-bounding-box OR polygon OR polyline OR segmentation OR ner OR point OR pdf OR spline// object, tool coordinate info
    lock, // bool, whether this object has been locked from being edited or not
    ocr, // bool, whether this object had OCR enabled or not (bounding-box only)
    metadata, // object, metadata about the object,
    outOfViewSegments, // list, start and end frames for out-of-view portions for this class
    classifications, // list, nested classifications, following the same format as parent
    },
    { ... }
]

We recommend examining a sample export from Ango Hub to better understand how it fits together, such as the following:

Sample Video Export
[
  {
    "asset": "https://angohub-test-assets.s3.eu-central-1.amazonaws.com/67b2efe918816f1f08d26758/assets/video.mp4",
    "assetId": "67b2f05618816f1f08d26835",
    "externalId": "Atlas _ Partners in Parkour.mp4",
    "metadata": {
      "width": 1280,
      "height": 720,
      "frameTotal": 1972,
      "frameRate": 29.97
    },
    "batches": [],
    "task": {
      "taskId": "67b2f07418816f1f08d268c0",
      "type": "default",
      "stage": "Start",
      "stageId": "Start",
      "updatedAt": "2025-02-18T09:59:05.682Z",
      "updatedBy": "example@imerit.net",
      "totalDuration": 0,
      "tools": [
        {
          "schemaId": "6898f9b5d827f3583355586",
          "segments": [
            0,
            100
          ],
          "keyFrames": {
            "0": {
              "bounding-box": {
                "x": 0.6799999999999997,
                "y": 512.1166666666667,
                "width": 20,
                "height": 40
              }
            },
            "1": {
              "bounding-box": {
                "x": 0.6676905999999997,
                "y": 509.2601909333333,
                "width": 21.1217784,
                "height": 46.2854962
              }
            }
          },
          "objectId": "1",
          "lock": false,
          "ocr": null,
          "metadata": {
            "createdAt": 1739797009308,
            "createdBy": "import"
          },
          "outOfViewSegments": [],
          "classifications": [
            {
              "objectId": "6",
              "schemaId": "cb64ec98cca7c5bdb8f8330",
              "tool": "radio",
              "title": "bb-rad",
              "keyFrames": {
                "60": {
                  "answer": "a"
                },
                "61": {
                  "answer": "a"
                }
              },
              "segments": null,
              "outOfViewSegments": [],
              "classifications": []
            }
          ]
        },
        {
          "schemaId": "6898f9b5d827f3583355586",
          "segments": [
            150,
            175
          ],
          "keyFrames": {
            "155": {
              "bounding-box": {
                "x": 0.6799999999999997,
                "y": 512.1166666666667,
                "width": 20,
                "height": 40
              }
            },
            "156": {
              "bounding-box": {
                "x": 0.6184529999999997,
                "y": 497.834288,
                "width": 25.608892,
                "height": 71.427481
              }
            }
          },
          "objectId": "2",
          "lock": false,
          "ocr": null,
          "metadata": {
            "createdAt": 1739797009308,
            "createdBy": "import"
          },
          "outOfViewSegments": [],
          "classifications": []
        }
      ],
      "classifications": [
        {
          "objectId": "e6af03900d3d1727d083528",
          "schemaId": "8ae3d748a66f22f4e954374",
          "tool": "radio",
          "title": "top-radio",
          "answer": "topr3",
          "classifications": []
        },
        {
          "objectId": "d4047efd2336ecab924c770",
          "schemaId": "328e7973dbe087d4849b299",
          "tool": "checkbox",
          "title": "top-chkbox",
          "answer": [
            "topc1"
          ],
          "classifications": []
        },
        {
          "objectId": "3",
          "schemaId": "a5f51ee4a6ff0f827fb0442",
          "tool": "radio",
          "title": "radio",
          "answer": [
            "r1",
            "r2",
            "r3"
          ],
          "keyFrames": {
            "50": {
              "answer": "r1"
            },
            "51": {
              "answer": "r1"
            },
            "52": {
              "answer": "r1"
            }
          },
          "segments": [
            50,
            250
          ],
          "outOfViewSegments": [],
          "classifications": [
            {
              "objectId": "4",
              "schemaId": "44b504df31195596ca46557",
              "tool": "single-dropdown",
              "title": "radio-dropdown",
              "keyFrames": {
                "60": {
                  "answer": "rr1"
                },
                "61": {
                  "answer": "rr1"
                },
                "62": {
                  "answer": "rr1"
                }
              },
              "segments": null,
              "outOfViewSegments": [],
              "classifications": []
            }
          ]
        }
      ],
      "relations": [],
      "brushDataUrl": null,
      "medicalBrushDataUrl": null
    }
  }
]

Polygon

Sample

{
  "polygon": [
    [
      600.4611650485438,
      537.2912621359224
    ],
    [
      600.4611650485438,
      472.03883495145635
    ],
    [
      543.5388349514564,
      399.84466019417476
    ]
  ],
  "objectId": "6b8a4338d299c63b5ee7825",
  "classifications": [
    {
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net",
      },
      "classifications": []
    }
  ],
  "schemaId": "a966c7799383465850cd803",
  "title": "gon",
}

Properties

Property
Type
Description

polygon

List

List of lists containing X, Y coordinates of each polygon point. List may contain any number of x, y coordinate pairs.

objectId

String

Object ID of the polygon. Unique in the asset.

classifications

List

metadata

Dict

Only appears if the Annotation Metadata toggle has been turned on before export. createdAt: int, Time of creation in Unix epoch time in milliseconds. GMT. createdBy: string, email of user creating the object updatedAt: int, optional. Time of last object update in Unix millisecond epoch time. GMT. Only appears if object has been updated. updatedBy: string, optional. email of user last updating the object. Only appears if object has been updated.

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

Segmentation

Segmentation point coordinates will not be visible in your export unless you activate the "Segmentation Points" toggle in the Export tab under the Fields dropdown.

Metadata for segmentations will not appear even when enabling the Annotation Metadata toggle in the export settings.

Segmentations are composed of instances, zones, regions, and holes.

An instance may contain more than one zone. One zone contains exactly one region. A region can contain more than one hole.

Each of the points composing the segmentation is represented as an X/Y coordinate pair.

Sample

{
  "schemaId": "0ccae6f3a41e4f8cd1de721",
  "objectId": "eace3fa4ad53eedfa033266",
  "segmentation": {
    "zones": [
      {
        "region": [
          [
            313.162,
            414.94
          ],
          [
            313.717,
            411.981
          ],
          [
            316.517,
            397.045
          ]
        ],
        "holes": []
      },
      {
        "region": [
          [
            369.814,
            331.194
          ],
          [
            376.151,
            328.957
          ],
          [
            389.216,
            324.346
          ]
        ],
        "holes": [
          [
            [
              406.712,
              372.575
            ],
            [
              407.344,
              380.782
            ],
            [
              409.238,
              388.358
            ]
          ]
        ]
      }
    ]
  },
  "classifications": [],
  "title": "Vehicle"
}

Properties

Property
Type
Description

segmentation

Object

objectId

String

Object ID of the segmentation instance. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

Bounding Box

Sample

{
  "bounding-box": {
    "x": 399.840782122905,
    "y": 400.4,
    "height": 196.84469273743017,
    "width": 199.08156424581006
  },
  "objectId": "9e9550c056d05bea9e01135",
  "classifications": [],
  "schemaId": "b1c6805d054f0e44ae11500",
  "title": "bb"
}

Properties

Property
Type
Description

bounding-box

Object

Object containing details about the bounding box.

x - The X coordinate of the top-left point y - The Y coordinate of the top-left point height - The height, in pixels, of the bounding box width - The width, in pixels, of the bounding box

objectId

String

Object ID of the bounding box. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

Rotated Bounding Box

Rotated Bounding Box Angles

Sample

{
  "rotated-bounding-box": {
    "x": 210.33495145631068,
    "y": 179.0970873786408,
    "height": 197.14563106796118,
    "width": 13.883495145631068,
    "corners": [
      [
        210.33495145631068,
        179.0970873786408
      ],
      [
        325.56796116504853,
        338.75728155339806
      ],
      [
        336.825603074518,
        330.6322008709114
      ],
      [
        221.59259336578015,
        170.97200669615412
      ]
    ],
    "rotation": -35.81945804237385
  },
  "objectId": "c23f41f1bddead556d82085",
  "classifications": [],
  "schemaId": "c8c1f3d2388109519849399",
  "title": "rbb"
}

Properties

Property
Type
Description

rotated-bounding-box

Object

Object containing coordinate and dimension details about the rotated bounding box. x - The X coordinate of the origin point (i.e. where the user first clicked) of the rotated bounding box

y - The Y coordinate of the origin point (i.e. where the user first clicked) of the rotated bounding box

width - The width of the rotated bounding box, in pixels

height - the height of the rotated bounding box, in pixels

rotation - the angle of rotation of the rotated bounding box, in degrees. Range: -180|+180

corners - A list of X, Y coordinate pairs pertaining to the location of the four corners of the rotated bounding box.

objectId

String

Object ID of the rotated bounding box. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

Polygon

Sample

{
  "polygon": [
    [
      514.3834951456311,
      338.75728155339806
    ],
    [
      338.06310679611653,
      488.6990291262136
    ],
    [
      408.86893203883494,
      590.0485436893204
    ],
    [
      549.0922330097087,
      590.0485436893204
    ],
    [
      578.247572815534,
      445.6601941747573
    ]
  ],
  "objectId": "21f11e94e6d438d15979682",
  "classifications": [
    {
      "objectId": "a390f9434a330d14cf16759",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "2",
      "classifications": []
    }
  ],
  "schemaId": "a966c7799383465850cd803",
  "title": "gon"
}

Properties

Property
Type
Description

polygon

Object

Object containing coordinate details about the points making up the polygon. A list of X, Y coordinates of its points. Points are represented consecutively. The first point in the list is the first point that was clicked when creating the polygon.

objectId

String

Object ID of the polygon. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

Polyline

Sample

{
  "polyline": [
    [
      359.734375,
      155.53732638888889
    ],
    [
      357.99652777777777,
      244.16753472222223
    ],
    [
      512.6649305555555,
      242.4296875
    ]
  ],
  "objectId": "7f9483f8f5506c6eeb72654",
  "classifications": [],
  "schemaId": "7a6e6f643d4adb817a35776",
  "title": "ine"
}

Properties

Property
Type
Description

polyline

Object

Object containing coordinate details about the points making up the polyline. A list of X, Y coordinates of its points. Points are represented consecutively. The first point in the list is the first point that was clicked when creating the polyline.

objectId

String

Object ID of the polyline. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

Medical Line Tool

Sample

"objects": [
  {
    "objectId": "6ed9d9b1dd6f2f0d3b8f168",
    "classifications": [],
    "polyline": {
      "x1": 18,
      "y1": 38,
      "z1": 17,
      "x2": 47,
      "y2": 10,
      "z2": 17,
      "measurement": {
        "distance": 130.64254312955103,
        "unit": "mm"
      }
    },
    "metadata": {
      "createdAt": 1724930935199,
      "createdBy": "example@example.net",
    },
    "schemaId": "unknown",
    "title": "unknown",
    "tool": "polyline"
  }
]

Properties

Property
Type
Description

polyline

Object

Object containing coordinate details about the points making up the line. A dict containing 3D coordinates, in pixels, of its points, as well as its length. (Length is only available for lines created or modified after the Ango Hub 3.21 update.)

objectId

String

Object ID of the line. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

Spline

Sample

{
  "spline": [
    [
      2842.6660649516775,
      1795.0065094066958,
      74.93719405360343,
      0
    ],
    [
      2737.752990914815,
      1905.0068647589223,
      84.92891539044695,
      0
    ],
    [
      2537.918564177935,
      2015.0072201111489,
      79.93305472202519,
      1
    ]
  ],
  "objectId": "6e6b6246bf8e569ce121052",
  "classifications": [
    {
      "objectId": "f38eff68ebf84d3d4d56678",
      "schemaId": "0e3ec72c773eaf41d739275",
      "tool": "radio",
      "title": "Color",
      "answer": "White",
      "classifications": []
    }
  ],
  "schemaId": "00cfee8cef67487be0b3363",
  "title": "External Road Marking"
}

Properties

Property
Type
Description

spline

List

List containing coordinate, width, and marking details about the points making up the spline. A list of X, Y coordinates of its points, the width information of each point in pixels, as well as whether the point was marked (1) or not (0). Points are represented consecutively. The first point in the list is the first point that was clicked when creating the spline.

A spline may contain any number of points starting from 2.

objectId

String

Object ID of the spline. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

Point

Sample

{
  "point": [
    721.2065972222223,
    146.8480902777778
  ],
  "objectId": "d0d9f24d6e927bd72dfe987",
  "classifications": [],
  "schemaId": "376a18175f35d976d204251",
  "title": "poi"
}

Properties

Property
Type
Description

point

Object

Object containing coordinate details about the point. The X, Y coordinates of the point.

objectId

String

Object ID of the point. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

PDF

Sample

{
  "pdf": {
    "position": {
      "boundingRect": {
        "x1": 12,
        "y1": 207.234375,
        "x2": 346,
        "y2": 292.234375,
        "width": 733.44,
        "height": 1073.9200133333334
      },
      "rects": [],
      "pageNumber": 1
    },
    "content": {
      "text": "FACHZEITSCHRIFT\n\nFUR DAS MODELLEISENBAHNWESEN\nUND ALLE FREUNDE\n\nDER EISENBAHN JAHRGANG 29\n",
      "confidence": 91
    }
  },
  "objectId": "49f4415f1fab3bbbc96a275",
  "classifications": [],
  "schemaId": "aca6765850e6fa107053050",
  "title": "pdf"
}

Properties

Property
Type
Description

pdf

Object

Object containing coordinate, size, and content details about the PDF area.

  • position: dict, positional information regarding the object

    • boundingRect: dict, positional information regarding the object

      • x1 and y1 X, Y coordinates of the top left point of the PDF area

      • x2 and y2 X, Y coordinates of the bottom right point of the PDF area

    • rects: list, deprecated.

    • pageNumber: int, the number of the PDF page where the object is found. 1-indexed.

  • content: dict, information regarding the content of the object

    • text: string, if OCR was run for the object, the OCR result.

    • confidence: int, range: 0-100, if OCR was run for the object, the confidence in the OCR result as returned by Hub.

objectId

String

Object ID of the PDF area. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

PCT

Sample

{
  "objectId": "afabc5ad-cbac-4714-8b52-c70b54928e14",
  "classifications": [],
  "schemaId": "1bc59958525e1c8992a2256",
  "page": "1",
  "markingTool": "Polygon",
  "pct": {
    "label": "1bc59958525e1c8992a2256",
    "_id": "672c608c12a2f1820a433449",
    "id": "afabc5ad-cbac-4714-8b52-c70b54928e14",
    "object_type": "polygongroup",
    "identity": 0,
    "position": [
      0,
      0,
      0
    ],
    "orientation": [
      0,
      0,
      0,
      1
    ],
    "points": [
      {
        "position": {
          "x": 0.04556074766355134,
          "y": -0.06644275700934575,
          "z": 1.4210144207991604e-18
        }
      },
      {
        "position": {
          "x": 0.022196261682243094,
          "y": -0.030373831775700934,
          "z": 1.4210144207991604e-18
        }
      },
      {
        "position": {
          "x": 0.0887850467289719,
          "y": 0.09681658878504669,
          "z": 1.4210144207991604e-18
        }
      },
      {
        "position": {
          "x": 0.2756254016834791,
          "y": 0.29481451192726466,
          "z": -0.021115406983940727
        }
      },
      {
        "position": {
          "x": 0.24406891483779214,
          "y": 0.14731302359852402,
          "z": -0.02109511367217651
        }
      },
      {
        "position": {
          "x": 0.24406891483779214,
          "y": 0.14731302359852402,
          "z": -0.02109511367217651
        }
      }
    ],
    "taxonomy_attribute": {}
  },
  "title": "Polygons"
}

Entity (NER)

The Entity tool is used in both audio and text to select a portion of the asset.

Sample

Audio

{
  "ner": {
    "start": 5.125670601851851,
    "end": 6.434645621141975
  },
  "objectId": "f3a454f8d23d212c73a4479",
  "classifications": [],
  "schemaId": "b1919d09c9770392ebdb948",
  "title": "ent"
}

Text

{
  "ner": {
    "start": 319,
    "end": 429,
    "selection": "desk surface/wrist rest and still reach the buttons. I liked this enough to try to grip it a bit differently (h"
  },
  "objectId": "b6846451de619fa72bf0466",
  "classifications": [],
  "schemaId": "b1919d09c9770392ebdb948",
  "title": "ent"
}

Properties

Property
Type
Description

pdf

Object

Object containing start/end details about the entity selected.

  • start

    • In text, int, 0-indexed index of the first character in the entity.

    • In audio, float, value in seconds where the entity starts.

  • end

    • In text, int, 0-indexed index of the last character in the entity.

    • In audio, float, value in seconds where the entity stops.

  • selection (text only): Text content of the entity.

objectId

String

Object ID of the entity. Unique in the asset.

classifications

List

schemaId

String

ID of the class to which the object belongs.

title

String

Name of the class to which the object belongs.

Special Considerations for Video Assets

interpolationStopped

An annotator may choose to make an object disappear from a video at a certain frame. This can happen, for example, if the annotator was tracking a car in a video and the car left the frame.

To do this, Hub creates an object with the following format in the tools list:

{
    "objectId": "785a0dc546fe191926ed526",
    "schemaId": "b1c6805d054f0e44ae11500",
    "classifications": [],
    "interpolationStopped": true,
    "page": 4,
    "title": "bb"
}

Where the objectId noted matches with the objectId of the object disappearing.

The page property indicates the page at which the object disappeared, 0-indexed. e.g. in the example above, the object was visible in pages 0, 1, 2, and 3.

If the object, later, reappears again, Hub creates another object in the tools list, this time with the interpolationStopped property set to false:

{
      "objectId": "785a0dc546fe191926ed526",
      "schemaId": "b1c6805d054f0e44ae11500",
      "classifications": [],
      "interpolationStopped": false,
      "page": 8,
      "title": "bb",
}

In this case, it means that starting from frame 8 onwards, the object is visible again, until another interpolationStopped=true object appears or, if it doesn't appear, until the end of the video.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

Object containing details about a segmentation instance. A segmentation instance can be composed of multiple zones. Each zone contains at least one region and may optionally contain a hole, which is always included, coordinate wise, within the region. More details in the section above.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the format. See its docs page for more information.

"polygon": [
    [
      600.4611650485438, // x
      537.2912621359224  // y
    ],
    [
      600.4611650485438, // x
      472.03883495145635 // y
    ],
    [
      543.5388349514564, // x
      399.84466019417476 // y
    ]
  ]
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net",
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"segmentation": {
    "zones": [
      {
        "region": [
          [
            313.162, // x
            414.94.  // y
          ],
          [
            313.717,
            411.981
          ],
          [
            316.517,
            397.045
          ]
        ],
        "holes": []
      },
      {
        "region": [
          [
            369.814,
            331.194
          ],
          [
            376.151,
            328.957
          ],
          [
            389.216,
            324.346
          ]
        ],
        "holes": [
          [
            [
              406.712,
              372.575
            ],
            [
              407.344,
              380.782
            ],
            [
              409.238,
              388.358
            ]
          ]
        ]
      }
    ]
}
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net",
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"bounding-box": {
    "x": 399.840782122905,
    "y": 400.4,
    "height": 196.84469273743017,
    "width": 199.08156424581006
  }
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net",
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"rotated-bounding-box": {
          "x": 210.33495145631068,
          "y": 179.0970873786408,
          "height": 197.14563106796118,
          "width": 13.883495145631068,
          "corners": [
            [
              210.33495145631068,
              179.0970873786408
            ],
            [
              325.56796116504853,
              338.75728155339806
            ],
            [
              336.825603074518,
              330.6322008709114
            ],
            [
              221.59259336578015,
              170.97200669615412
            ]
          ],
          "rotation": -35.81945804237385
        }
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net",
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"polygon": [
              [
                514.3834951456311,
                338.75728155339806
              ],
              [
                338.06310679611653,
                488.6990291262136
              ],
              [
                408.86893203883494,
                590.0485436893204
              ],
              [
                549.0922330097087,
                590.0485436893204
              ],
              [
                578.247572815534,
                445.6601941747573
              ]
            ]
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net",
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"polyline": [
              [
                514.3834951456311,
                338.75728155339806
              ],
              [
                338.06310679611653,
                488.6990291262136
              ],
              [
                408.86893203883494,
                590.0485436893204
              ],
              [
                549.0922330097087,
                590.0485436893204
              ],
              [
                578.247572815534,
                445.6601941747573
              ]
            ]
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net",
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"polyline": {
      "x1": 18,
      "y1": 38,
      "z1": 17,
      "x2": 47,
      "y2": 10,
      "z2": 17,
      "measurement": {
        "distance": 130.64254312955103,
        "unit": "mm"
      }
    }
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net",
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"spline": [
    [
      2842.6660649516775,
      1795.0065094066958,
      74.93719405360343,
      0
    ],
    [
      2737.752990914815,
      1905.0068647589223,
      84.92891539044695,
      0
    ],
    [
      2537.918564177935,
      2015.0072201111489,
      79.93305472202519,
      1
    ]
  ]
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net"
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"point": [
          721.2065972222223,
          146.8480902777778
        ]
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net"
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"pdf": {
          "position": {
            "boundingRect": {
              "x1": 12,
              "y1": 207.234375,
              "x2": 346,
              "y2": 292.234375,
              "width": 733.44,
              "height": 1073.9200133333334
            },
            "rects": [],
            "pageNumber": 1
          },
          "content": {
            "text": "FACHZEITSCHRIFT\n\nFUR DAS MODELLEISENBAHNWESEN\nUND ALLE FREUNDE\n\nDER EISENBAHN JAHRGANG 29\n",
            "confidence": 91
          }
        }
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net"
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
"ner": {
          "start": 319,
          "end": 429,
          "selection": "desk surface/wrist rest and still reach the buttons. I liked this enough to try to grip it a bit differently (h"
        }
"objectId": "6b8a4338d299c63b5ee7825"
"classifications": [    
{
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "lorenzo@example.net"
     },
    "classifications": []
  }
]
"schemaId": "4460fa700ef030c4e575604"
"title": "Vehicle"
task
Bounding Box
Rotated Bounding Box
Polygon
Polyline
Spline
Segmentation
Entity
Point
PDF
PCT
Brush
Voxel Brush
task
the page on the Task object
classification
classification
classification
classification
classification
classification
classification
classification
classification
classification
classification
segmentation
All measurements in pixels.
Rotated Bounding Box Angles