Labeling Editor Interface
Overview of the labeling interface on Ango Hub.
Last updated
Overview of the labeling interface on Ango Hub.
Last updated
Ango Hub has labeling editors for each of the supported formats: audio, image, DICOM video, PDF, and text (NER).
Even though the data types supported by Ango Hub are all different, we designed our labeling editors to be as similar to one another as possible. This way, once an annotator has learned how to use one, they will be able to quickly switch to the other, without having to spend time retraining.
In this page, we will go through the features in common between all editors. You can find further information on each editor (e.g. image, audio, etc.) in the pages found within this page in the left sidebar of the docs.
The following is an overview of the editor’s interface.
At the top of the left sidebar is the list of all labeling tools available in the project. Bounding boxes, polygons, points, and entities, for example, are all labeling tools.
Tools of different types can be present in the same project. To annotate, the labeler clicks on the tool from this section then clicks on the asset, where the label needs to be placed.
In the middle of the sidebar are the top-level classification tools for this project.
If created from the Label Set section of the Settings tab, classifications will be shown here. From this section, annotators can categorize the asset shown, with radio buttons, dropdowns, free text, and more.
Below the classification section is a list of all annotations (objects) placed on the current asset.
If the label has a nested question, by clicking on it in the list, you can expand it.
Create an issue bound to the selected object.
If in your project you have more than one of the same tool, you can quickly change the label’s category from here. If we had another bounding box tool called “tree”, for example, we would be able to change this label’s category from “vehicle” to “tree.”
It is not possible to change an annotation to a label of a different type. (for example, a bounding box into a polygon.)
Another way to quickly change a label's category is to hover over it with the mouse cursor and pressing Alt + the new category's keyboard shortcut.
Add text which will be shown in the object list. This text will be bound to the object and will appear in the final export.
Each object (annotation) has a unique ID. Clicking this button will copy the annotation's ID to the clipboard.
A locked label cannot be edited. The lock function is particularly useful when importing labels, if there are labels you'd like the annotators not to change.
If your project ontology contains group relation classes, by picking the group relation from the "Grouping" menu, you may create a new group with the selected class and add the selected object to such new group.
You may bring the selected annotation to the front or the back. When you do, the annotation will visually appear in front or behind other annotations of its class. (e.g. if an annotation of class A is behind an annotation of class B, clicking on "Bring to Front" will not bring this annotation in front of that of class B.)
In terms of the final export, annotations that have been brought to the back will appear last (after all annotations, even those not in the same class), and annotations that have been brought to the front will appear first (before all annotations, even those not in the same class).
You can quickly move between objects in the editor by hovering over an object such that it is selected, and then by pressing on Shift + Up or Shift + Down to navigate between objects. The selected object will automatically be shown in the middle of the screen and the zoom level will be accommodated to better view the object.
On the left-hand side are links to navigate between projects and within the current project.
The arrows allow users to move back to the previous asset, and forward to the next asset.
The navigation arrows are only shown if the current task was opened by following an internal link from within the platform. Directly opening a task from an URL will prevent the arrows from showing.
Whenever a task moves from one stage to another, the circle highlighted in this section changes.
The circle at the foreground represents the current stage. The ones on its left, the previous stages, and the ones to the right the expected future stages for the task.
By hovering over each circle, with the mouse, you are able to access more information about the task in each stage.
By clicking on the speech bubble, you activate the Targeted Issue cursor. When this cursor is active, you may, on images, videos, and medical data, click on the asset to leave issues which are specifically tied to a particular X,Y coordinate on the asset.
By clicking on the three dots, you may access the overflow menu, housing options used less frequently.
Reset: Return the annotations to their original state.
Copy: Copy the current asset's annotations to the system clipboard.
Paste: Paste annotations from the clipboard to the current asset. Only works for annotations copied using the Copy button in the overflow menu.
Clear: Deletes all annotations and resets all classifications in the current asset.
Set as Sample: Sets the current task as sample.
Re-queue: Opens the re-queue dialog for the current task.
Copy Answers: Copies to clipboard a JSON of the annotations and classifications currently on the asset, previewing what the asset annotations will look like in the final export. Especially useful in label validation workflows.
At the top right of the editor, we find the Save and Submit buttons.
The Save button is indicated by a floppy disc icon. Clicking on it will save your current annotations without submitting them.
When you click on Submit, you will be shown a confirmation dialog asking you if you are sure you wish to submit. When you click on OK on the dialog, changes will be saved and you will be brought to the next unlabeled asset.
If you wish to submit but not be brought to the next task in the queue, please click on the arrow next to the "Submit" button and click on "Submit & Exit":
You can also submit by pressing the Enter button on your keyboard, then pressing Enter when the confirmation dialog appears.
There is no autosave.
If you try to leave the page without saving, you will be asked to save before exiting.
From the right sidebar, open the Issues panel. This will show all issues present in the current task.
Users can type text in the + Create Issue field and press Enter to create issues, which will be sent as in-app notifications to task assignees, project owners, and project reviewers.
The user who created the issue, as well as reviewers and project owners, will be able to reply to the issue, resolve it, or delete it from the buttons on the right side of the panel.
The Instructions button opens a drawer showing labeling instructions for the current project, if present. Project owners can upload instructions as a PDF file from the project’s Settings tab.
The buttons above the PDF allow the user to reset the zoom, zoom in and out, and to navigate between pages.
The Samples button opens a drawer showing all samples set in the project. More on samples here.
Clicking on a sample in the list will open its related asset and description. Clicking on the Arrow in box button in the sample will open an enlarged view of the sample:
Clicking on Attachment will open a drawer showing the attachment linked to the current asset, if available. More on attachments here.
Clicking on Task info will show information related to the current task.
Clicking on the right-facing arrow on the left side of the screen will open the Assets drawer.
The assets drawer contains a list of all assets in the current project, together with their related information such as their status, consensus score, assignees, and more. Clicking on an item in the list will open the related asset.
Clicking on the red Settings button on the bottom right of the editor will open a panel with a number of quick labeling settings.
Each data type has its own set of settings. This page will only go through those that are common to all data types. For more details on each data type's settings, visit its page, for example, Image Labeling Editor.
If active, when hovering over an annotation, you'll see a summary of the information related to the annotation immediately next to it.
If Show Summary on Hover is active, the annotation's unique ID is shown among the annotation information summary.
In the text labeling editor, toggles displaying class names next to NER entities.
Activate to hide all segmentation points in the asset. Will improve performance in larger projects.
By default, after placing an annotation, the current labeling tool is deselected, and the annotator needs to click on a labeling tool again to place another annotation.
When this toggle is selected, placing annotations will not deselect the currently selected labeling tool, and the annotator can immediately place another one. The annotator can deselect the currently selected labeling tool by pressing Esc.
When this setting is activated, right-clicking on an asset will automatically open the expanded version of the context menu, instead of opening the collapsed one.
On occasion, for example as a result of running a plugin, annotations may not be associated with a category. Such annotations are known as "unknown". Activate to hide all unknown annotations.
Disable to disable image smoothing. Useful for pixel-wise annotation tasks.
When this toggle is enabled, dashed lines will be visible around the cursor's location.
Opacity
Alter the opacity of annotations on screen.
The brightness, contrast, and saturation sliders apply to the asset on screen.
Changes the thickness of bounding boxes.
Change the size of point annotations.
Invert the colors of the asset.
As the annotator adds labels to the asset, they will be shown in this list. The “eye” icon next to Objects allows you to hide all labels on the screen.
The eye icon allows you to hide the label from the asset. You can also use the shortcut H while hovering over an annotation to hide it. To unhide all annotations, press Shift + H.
The trash can deletes it. If present, nested labels will be shown below. In the example above, we ask the labeler what kind of vehicle they have just labeled with the bounding box tool.
Clicking on the three dots will open a submenu:
The Ango Logo opens your project list. The Home opens the current project’s dashboard. The Back button brings you back one page.
From the top, clicking on will refresh the asset list. Clicking on X will close the drawer. At the bottom, arrows and page numbers allow you to navigate the asset list.
