Tools

Tools fields in Ango Annotation Format exports.

The tools object is part of the task object in the Ango Annotation Format. It contains all annotation data pertaining to labels created with tools, e.g. not classifications, relations, or brushes.

Annotations created with the following annotation tools will appear in the tools object:

• Bounding Box

• Rotated Bounding Box

• Polygon

• Polyline

• Spline

• Segmentation

• Entity

• Point

• PDF

• PCT

Annotations created with the Brush and Voxel Brush tools are not saved here. They are instead saved, respectively, in the brushDataUrl and the medicalBrushDataUrl objects within the task object. See the page on the Task object for more.

That said, if the voxel brush in your project has nested classifications, the nested classification contents will appear here.

The page property, in the export, is 0-indexed, while in the Ango Hub user interface it is 1-indexed.

What this means in practice is that pages start at 0 in the export, but 1 in the UI.

So for example, if, in the UI, you create a bounding box at page "12", it will appear under page "11" in the export.

General Structure

"tools" : [{
    bounding-box OR rotated-bounding-box OR polygon OR polyline OR segmentation OR ner OR point OR pdf OR spline// object, tool coordinate info
    objectId // string, ID of the object
    schemaId, // string, ID of the labeling tool in the project
    classifications, // list, nested classifications
    title // string, title of the tool
    page // int, page where the object is located (only multi-page, non-video assets)
    medical-brush, // bool, true if the current tool is a medical brush
    },
    { ... }
]

Coordinates in the Ango Annotation Format

This is the coordinate system used when determining the position of points in the Ango export.

The origin point lies at the top left corner, and each point has a (X, Y) coordinate pair determining its position, in pixels, on the image.

All measurements in pixels.

Changes in Video Assets

Video assets have a slightly different format for the "tools" property:

"tools" : [{
    schemaId, // string, ID of the labeling tool in the project
    objectId, // string, ID of the object
    segments, // list, start and end frames of segments for this class
    keyFrames, // object, list of keyframes for this annotation for this class
    bounding-box OR rotated-bounding-box OR polygon OR polyline OR segmentation OR ner OR point OR pdf OR spline// object, tool coordinate info
    lock, // bool, whether this object has been locked from being edited or not
    ocr, // bool, whether this object had OCR enabled or not (bounding-box only)
    metadata, // object, metadata about the object,
    outOfViewSegments, // list, start and end frames for out-of-view portions for this class
    classifications, // list, nested classifications, following the same format as parent
    },
    { ... }
]

We recommend examining a sample export from Ango Hub to better understand how it fits together, such as the following:

Sample Video Export

[
  {
    "asset": "https://angohub-test-assets.s3.eu-central-1.amazonaws.com/67b2efe918816f1f08d26758/assets/video.mp4",
    "assetId": "67b2f05618816f1f08d26835",
    "externalId": "Atlas _ Partners in Parkour.mp4",
    "metadata": {
      "width": 1280,
      "height": 720,
      "frameTotal": 1972,
      "frameRate": 29.97
    },
    "batches": [],
    "task": {
      "taskId": "67b2f07418816f1f08d268c0",
      "type": "default",
      "stage": "Start",
      "stageId": "Start",
      "updatedAt": "2025-02-18T09:59:05.682Z",
      "updatedBy": "[e-mail]",
      "totalDuration": 0,
      "tools": [
        {
          "schemaId": "6898f9b5d827f3583355586",
          "segments": [
            0,
            100
          ],
          "keyFrames": {
            "0": {
              "bounding-box": {
                "x": 0.6799999999999997,
                "y": 512.1166666666667,
                "width": 20,
                "height": 40
              }
            },
            "1": {
              "bounding-box": {
                "x": 0.6676905999999997,
                "y": 509.2601909333333,
                "width": 21.1217784,
                "height": 46.2854962
              }
            }
          },
          "objectId": "1",
          "lock": false,
          "ocr": null,
          "metadata": {
            "createdAt": 1739797009308,
            "createdBy": "import"
          },
          "outOfViewSegments": [],
          "classifications": [
            {
              "objectId": "6",
              "schemaId": "cb64ec98cca7c5bdb8f8330",
              "tool": "radio",
              "title": "bb-rad",
              "keyFrames": {
                "60": {
                  "answer": "a"
                },
                "61": {
                  "answer": "a"
                }
              },
              "segments": null,
              "outOfViewSegments": [],
              "classifications": []
            }
          ]
        },
        {
          "schemaId": "6898f9b5d827f3583355586",
          "segments": [
            150,
            175
          ],
          "keyFrames": {
            "155": {
              "bounding-box": {
                "x": 0.6799999999999997,
                "y": 512.1166666666667,
                "width": 20,
                "height": 40
              }
            },
            "156": {
              "bounding-box": {
                "x": 0.6184529999999997,
                "y": 497.834288,
                "width": 25.608892,
                "height": 71.427481
              }
            }
          },
          "objectId": "2",
          "lock": false,
          "ocr": null,
          "metadata": {
            "createdAt": 1739797009308,
            "createdBy": "import"
          },
          "outOfViewSegments": [],
          "classifications": []
        }
      ],
      "classifications": [
        {
          "objectId": "e6af03900d3d1727d083528",
          "schemaId": "8ae3d748a66f22f4e954374",
          "tool": "radio",
          "title": "top-radio",
          "answer": "topr3",
          "classifications": []
        },
        {
          "objectId": "d4047efd2336ecab924c770",
          "schemaId": "328e7973dbe087d4849b299",
          "tool": "checkbox",
          "title": "top-chkbox",
          "answer": [
            "topc1"
          ],
          "classifications": []
        },
        {
          "objectId": "3",
          "schemaId": "a5f51ee4a6ff0f827fb0442",
          "tool": "radio",
          "title": "radio",
          "answer": [
            "r1",
            "r2",
            "r3"
          ],
          "keyFrames": {
            "50": {
              "answer": "r1"
            },
            "51": {
              "answer": "r1"
            },
            "52": {
              "answer": "r1"
            }
          },
          "segments": [
            50,
            250
          ],
          "outOfViewSegments": [],
          "classifications": [
            {
              "objectId": "4",
              "schemaId": "44b504df31195596ca46557",
              "tool": "single-dropdown",
              "title": "radio-dropdown",
              "keyFrames": {
                "60": {
                  "answer": "rr1"
                },
                "61": {
                  "answer": "rr1"
                },
                "62": {
                  "answer": "rr1"
                }
              },
              "segments": null,
              "outOfViewSegments": [],
              "classifications": []
            }
          ]
        }
      ],
      "relations": [],
      "brushDataUrl": null,
      "medicalBrushDataUrl": null
    }
  }
]

Polygon

Sample

{
  "polygon": [
    [
      600.4611650485438,
      537.2912621359224
    ],
    [
      600.4611650485438,
      472.03883495145635
    ],
    [
      543.5388349514564,
      399.84466019417476
    ]
  ],
  "objectId": "6b8a4338d299c63b5ee7825",
  "classifications": [
    {
      "objectId": "78d263ed59db8dd1dcb4666",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "1",
      "metadata": {
        "createdAt": "2023-09-05T10:28:18.667Z",
        "createdBy": "[e-mail]",
      },
      "classifications": []
    }
  ],
  "schemaId": "a966c7799383465850cd803",
  "title": "gon",
}

Properties

Property	Type	Description
polygon	List	List of lists containing X, Y coordinates of each polygon point. List may contain any number of x, y coordinate pairs. "polygon": [ [ 600.4611650485438, // x 537.2912621359224 // y ], [ 600.4611650485438, // x 472.03883495145635 // y ], [ 543.5388349514564, // x 399.84466019417476 // y ] ]
objectId	String	Object ID of the polygon. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]", }, "classifications": [] } ]
metadata	Dict	Only appears if the Annotation Metadata toggle has been turned on before export. createdAt: int, Time of creation in Unix epoch time in milliseconds. GMT. createdBy: string, email of user creating the object updatedAt: int, optional. Time of last object update in Unix millisecond epoch time. GMT. Only appears if object has been updated. updatedBy: string, optional. email of user last updating the object. Only appears if object has been updated.
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

Segmentation

Segmentation point coordinates will not be visible in your export unless you activate the "Segmentation Points" toggle in the Export tab under the Fields dropdown.

Metadata for segmentations will not appear even when enabling the Annotation Metadata toggle in the export settings.

Segmentations are composed of instances, zones, regions, and holes.

An instance may contain more than one zone. One zone contains exactly one region. A region can contain more than one hole.

Each of the points composing the segmentation is represented as an X/Y coordinate pair.

Sample

{
  "schemaId": "0ccae6f3a41e4f8cd1de721",
  "objectId": "eace3fa4ad53eedfa033266",
  "segmentation": {
    "zones": [
      {
        "region": [
          [
            313.162,
            414.94
          ],
          [
            313.717,
            411.981
          ],
          [
            316.517,
            397.045
          ]
        ],
        "holes": []
      },
      {
        "region": [
          [
            369.814,
            331.194
          ],
          [
            376.151,
            328.957
          ],
          [
            389.216,
            324.346
          ]
        ],
        "holes": [
          [
            [
              406.712,
              372.575
            ],
            [
              407.344,
              380.782
            ],
            [
              409.238,
              388.358
            ]
          ]
        ]
      }
    ]
  },
  "classifications": [],
  "title": "Vehicle"
}

Properties

Property	Type	Description
segmentation	Object	Object containing details about a segmentation instance. A segmentation instance can be composed of multiple zones. Each zone contains at least one region and may optionally contain a hole, which is always included, coordinate wise, within the region. More details in the segmentation section above. "segmentation": { "zones": [ { "region": [ [ 313.162, // x 414.94. // y ], [ 313.717, 411.981 ], [ 316.517, 397.045 ] ], "holes": [] }, { "region": [ [ 369.814, 331.194 ], [ 376.151, 328.957 ], [ 389.216, 324.346 ] ], "holes": [ [ [ 406.712, 372.575 ], [ 407.344, 380.782 ], [ 409.238, 388.358 ] ] ] } ] }
objectId	String	Object ID of the segmentation instance. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]", }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

Bounding Box

Sample

{
  "bounding-box": {
    "x": 399.840782122905,
    "y": 400.4,
    "height": 196.84469273743017,
    "width": 199.08156424581006
  },
  "objectId": "9e9550c056d05bea9e01135",
  "classifications": [],
  "schemaId": "b1c6805d054f0e44ae11500",
  "title": "bb"
}

Properties

Property	Type	Description
bounding-box	Object	Object containing details about the bounding box. x - The X coordinate of the top-left point y - The Y coordinate of the top-left point height - The height, in pixels, of the bounding box width - The width, in pixels, of the bounding box "bounding-box": { "x": 399.840782122905, "y": 400.4, "height": 196.84469273743017, "width": 199.08156424581006 }
objectId	String	Object ID of the bounding box. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]", }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

Rotated Bounding Box

Rotated Bounding Box Angles

Rotated Bounding Box Angles

Sample

{
  "rotated-bounding-box": {
    "x": 210.33495145631068,
    "y": 179.0970873786408,
    "height": 197.14563106796118,
    "width": 13.883495145631068,
    "corners": [
      [
        210.33495145631068,
        179.0970873786408
      ],
      [
        325.56796116504853,
        338.75728155339806
      ],
      [
        336.825603074518,
        330.6322008709114
      ],
      [
        221.59259336578015,
        170.97200669615412
      ]
    ],
    "rotation": -35.81945804237385
  },
  "objectId": "c23f41f1bddead556d82085",
  "classifications": [],
  "schemaId": "c8c1f3d2388109519849399",
  "title": "rbb"
}

Properties

Property	Type	Description
rotated-bounding-box	Object	Object containing coordinate and dimension details about the rotated bounding box. x - The X coordinate of the origin point (i.e. where the user first clicked) of the rotated bounding box y - The Y coordinate of the origin point (i.e. where the user first clicked) of the rotated bounding box width - The width of the rotated bounding box, in pixels height - the height of the rotated bounding box, in pixels rotation - the angle of rotation of the rotated bounding box, in degrees. Range: -180|+180 corners - A list of X, Y coordinate pairs pertaining to the location of the four corners of the rotated bounding box. "rotated-bounding-box": { "x": 210.33495145631068, "y": 179.0970873786408, "height": 197.14563106796118, "width": 13.883495145631068, "corners": [ [ 210.33495145631068, 179.0970873786408 ], [ 325.56796116504853, 338.75728155339806 ], [ 336.825603074518, 330.6322008709114 ], [ 221.59259336578015, 170.97200669615412 ] ], "rotation": -35.81945804237385 }
objectId	String	Object ID of the rotated bounding box. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]", }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

Polygon

Sample

{
  "polygon": [
    [
      514.3834951456311,
      338.75728155339806
    ],
    [
      338.06310679611653,
      488.6990291262136
    ],
    [
      408.86893203883494,
      590.0485436893204
    ],
    [
      549.0922330097087,
      590.0485436893204
    ],
    [
      578.247572815534,
      445.6601941747573
    ]
  ],
  "objectId": "21f11e94e6d438d15979682",
  "classifications": [
    {
      "objectId": "a390f9434a330d14cf16759",
      "schemaId": "4460fa700ef030c4e575604",
      "tool": "radio",
      "title": "Nested Radio",
      "answer": "2",
      "classifications": []
    }
  ],
  "schemaId": "a966c7799383465850cd803",
  "title": "gon"
}

Properties

Property	Type	Description
polygon	Object	Object containing coordinate details about the points making up the polygon. A list of X, Y coordinates of its points. Points are represented consecutively. The first point in the list is the first point that was clicked when creating the polygon. "polygon": [ [ 514.3834951456311, 338.75728155339806 ], [ 338.06310679611653, 488.6990291262136 ], [ 408.86893203883494, 590.0485436893204 ], [ 549.0922330097087, 590.0485436893204 ], [ 578.247572815534, 445.6601941747573 ] ]
objectId	String	Object ID of the polygon. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]", }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

Polyline

Sample

{
  "polyline": [
    [
      359.734375,
      155.53732638888889
    ],
    [
      357.99652777777777,
      244.16753472222223
    ],
    [
      512.6649305555555,
      242.4296875
    ]
  ],
  "objectId": "7f9483f8f5506c6eeb72654",
  "classifications": [],
  "schemaId": "7a6e6f643d4adb817a35776",
  "title": "ine"
}

Properties

Property	Type	Description
polyline	Object	Object containing coordinate details about the points making up the polyline. A list of X, Y coordinates of its points. Points are represented consecutively. The first point in the list is the first point that was clicked when creating the polyline. "polyline": [ [ 514.3834951456311, 338.75728155339806 ], [ 338.06310679611653, 488.6990291262136 ], [ 408.86893203883494, 590.0485436893204 ], [ 549.0922330097087, 590.0485436893204 ], [ 578.247572815534, 445.6601941747573 ] ]
objectId	String	Object ID of the polyline. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]", }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

Medical Line Tool

Sample

"objects": [
  {
    "objectId": "6ed9d9b1dd6f2f0d3b8f168",
    "classifications": [],
    "polyline": {
      "x1": 18,
      "y1": 38,
      "z1": 17,
      "x2": 47,
      "y2": 10,
      "z2": 17,
      "measurement": {
        "distance": 130.64254312955103,
        "unit": "mm"
      }
    },
    "metadata": {
      "createdAt": 1724930935199,
      "createdBy": "[e-mail]",
    },
    "schemaId": "unknown",
    "title": "unknown",
    "tool": "polyline"
  }
]

Properties

Property	Type	Description
polyline	Object	Object containing coordinate details about the points making up the line. A dict containing 3D coordinates, in pixels, of its points, as well as its length. (Length is only available for lines created or modified after the Ango Hub 3.21 update.) "polyline": { "x1": 18, "y1": 38, "z1": 17, "x2": 47, "y2": 10, "z2": 17, "measurement": { "distance": 130.64254312955103, "unit": "mm" } }
objectId	String	Object ID of the line. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]", }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

Spline

Sample

{
  "spline": [
    [
      2842.6660649516775,
      1795.0065094066958,
      74.93719405360343,
      0
    ],
    [
      2737.752990914815,
      1905.0068647589223,
      84.92891539044695,
      0
    ],
    [
      2537.918564177935,
      2015.0072201111489,
      79.93305472202519,
      1
    ]
  ],
  "objectId": "6e6b6246bf8e569ce121052",
  "classifications": [
    {
      "objectId": "f38eff68ebf84d3d4d56678",
      "schemaId": "0e3ec72c773eaf41d739275",
      "tool": "radio",
      "title": "Color",
      "answer": "White",
      "classifications": []
    }
  ],
  "schemaId": "00cfee8cef67487be0b3363",
  "title": "External Road Marking"
}

Properties

Property	Type	Description
spline	List	List containing coordinate, width, and marking details about the points making up the spline. A list of X, Y coordinates of its points, the width information of each point in pixels, as well as whether the point was marked (1) or not (0). Points are represented consecutively. The first point in the list is the first point that was clicked when creating the spline. A spline may contain any number of points starting from 2. "spline": [ [ 2842.6660649516775, 1795.0065094066958, 74.93719405360343, 0 ], [ 2737.752990914815, 1905.0068647589223, 84.92891539044695, 0 ], [ 2537.918564177935, 2015.0072201111489, 79.93305472202519, 1 ] ]
objectId	String	Object ID of the spline. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]" }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

Point

Sample

{
  "point": [
    721.2065972222223,
    146.8480902777778
  ],
  "objectId": "d0d9f24d6e927bd72dfe987",
  "classifications": [],
  "schemaId": "376a18175f35d976d204251",
  "title": "poi"
}

Properties

Property	Type	Description
point	Object	Object containing coordinate details about the point. The X, Y coordinates of the point. "point": [ 721.2065972222223, 146.8480902777778 ]
objectId	String	Object ID of the point. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]" }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

PDF

Sample

{
  "pdf": {
    "position": {
      "boundingRect": {
        "x1": 12,
        "y1": 207.234375,
        "x2": 346,
        "y2": 292.234375,
        "width": 733.44,
        "height": 1073.9200133333334
      },
      "rects": [],
      "pageNumber": 1
    },
    "content": {
      "text": "FACHZEITSCHRIFT\n\nFUR DAS MODELLEISENBAHNWESEN\nUND ALLE FREUNDE\n\nDER EISENBAHN JAHRGANG 29\n",
      "confidence": 91
    }
  },
  "objectId": "49f4415f1fab3bbbc96a275",
  "classifications": [],
  "schemaId": "aca6765850e6fa107053050",
  "title": "pdf"
}

Properties

Property	Type	Description
pdf	Object	Object containing coordinate, size, and content details about the PDF area. "pdf": { "position": { "boundingRect": { "x1": 12, "y1": 207.234375, "x2": 346, "y2": 292.234375, "width": 733.44, "height": 1073.9200133333334 }, "rects": [], "pageNumber": 1 }, "content": { "text": "FACHZEITSCHRIFT\n\nFUR DAS MODELLEISENBAHNWESEN\nUND ALLE FREUNDE\n\nDER EISENBAHN JAHRGANG 29\n", "confidence": 91 } } position: dict, positional information regarding the object boundingRect: dict, positional information regarding the object x1 and y1 X, Y coordinates of the top left point of the PDF area x2 and y2 X, Y coordinates of the bottom right point of the PDF area rects: list, deprecated. pageNumber: int, the number of the PDF page where the object is found. 1-indexed. content: dict, information regarding the content of the object text: string, if OCR was run for the object, the OCR result. confidence: int, range: 0-100, if OCR was run for the object, the confidence in the OCR result as returned by Hub.
objectId	String	Object ID of the PDF area. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]" }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

PCT

Sample

{
  "objectId": "afabc5ad-cbac-4714-8b52-c70b54928e14",
  "classifications": [],
  "schemaId": "1bc59958525e1c8992a2256",
  "page": "1",
  "markingTool": "Polygon",
  "pct": {
    "label": "1bc59958525e1c8992a2256",
    "_id": "672c608c12a2f1820a433449",
    "id": "afabc5ad-cbac-4714-8b52-c70b54928e14",
    "object_type": "polygongroup",
    "identity": 0,
    "position": [
      0,
      0,
      0
    ],
    "orientation": [
      0,
      0,
      0,
      1
    ],
    "points": [
      {
        "position": {
          "x": 0.04556074766355134,
          "y": -0.06644275700934575,
          "z": 1.4210144207991604e-18
        }
      },
      {
        "position": {
          "x": 0.022196261682243094,
          "y": -0.030373831775700934,
          "z": 1.4210144207991604e-18
        }
      },
      {
        "position": {
          "x": 0.0887850467289719,
          "y": 0.09681658878504669,
          "z": 1.4210144207991604e-18
        }
      },
      {
        "position": {
          "x": 0.2756254016834791,
          "y": 0.29481451192726466,
          "z": -0.021115406983940727
        }
      },
      {
        "position": {
          "x": 0.24406891483779214,
          "y": 0.14731302359852402,
          "z": -0.02109511367217651
        }
      },
      {
        "position": {
          "x": 0.24406891483779214,
          "y": 0.14731302359852402,
          "z": -0.02109511367217651
        }
      }
    ],
    "taxonomy_attribute": {}
  },
  "title": "Polygons"
}

Entity (NER)

The Entity tool is used in both audio and text to select a portion of the asset.

Sample

Audio

{
  "ner": {
    "start": 5.125670601851851,
    "end": 6.434645621141975
  },
  "objectId": "f3a454f8d23d212c73a4479",
  "classifications": [],
  "schemaId": "b1919d09c9770392ebdb948",
  "title": "ent"
}

Text

{
  "ner": {
    "start": 319,
    "end": 429,
    "selection": "desk surface/wrist rest and still reach the buttons. I liked this enough to try to grip it a bit differently (h"
  },
  "objectId": "b6846451de619fa72bf0466",
  "classifications": [],
  "schemaId": "b1919d09c9770392ebdb948",
  "title": "ent"
}

Properties

Property	Type	Description
pdf	Object	Object containing start/end details about the entity selected. "ner": { "start": 319, "end": 429, "selection": "desk surface/wrist rest and still reach the buttons. I liked this enough to try to grip it a bit differently (h" } start In text, int, 0-indexed index of the first character in the entity. In audio, float, value in seconds where the entity starts. end In text, int, 0-indexed index of the last character in the entity. In audio, float, value in seconds where the entity stops. selection (text only): Text content of the entity.
objectId	String	Object ID of the entity. Unique in the asset. "objectId": "6b8a4338d299c63b5ee7825"
classifications	List	List of nested classifications (if any). May contain further nested classifications in its own classifications property. Each object in the list is of the classification format. See its docs page for more information. "classifications": [ { "objectId": "78d263ed59db8dd1dcb4666", "schemaId": "4460fa700ef030c4e575604", "tool": "radio", "title": "Nested Radio", "answer": "1", "metadata": { "createdAt": "2023-09-05T10:28:18.667Z", "createdBy": "[e-mail]" }, "classifications": [] } ]
schemaId	String	ID of the class to which the object belongs. "schemaId": "4460fa700ef030c4e575604"
title	String	Name of the class to which the object belongs. "title": "Vehicle"

Special Considerations for Video Assets

interpolationStopped

An annotator may choose to make an object disappear from a video at a certain frame. This can happen, for example, if the annotator was tracking a car in a video and the car left the frame.

To do this, Hub creates an object with the following format in the tools list:

{
    "objectId": "785a0dc546fe191926ed526",
    "schemaId": "b1c6805d054f0e44ae11500",
    "classifications": [],
    "interpolationStopped": true,
    "page": 4,
    "title": "bb"
}

Where the objectId noted matches with the objectId of the object disappearing.

The page property indicates the page at which the object disappeared, 0-indexed. e.g. in the example above, the object was visible in pages 0, 1, 2, and 3.

If the object, later, reappears again, Hub creates another object in the tools list, this time with the interpolationStopped property set to false:

{
      "objectId": "785a0dc546fe191926ed526",
      "schemaId": "b1c6805d054f0e44ae11500",
      "classifications": [],
      "interpolationStopped": false,
      "page": 8,
      "title": "bb",
}

In this case, it means that starting from frame 8 onwards, the object is visible again, until another interpolationStopped=true object appears or, if it doesn't appear, until the end of the video.