Frame Interpolation

When annotating videos, Hub can use keyframes to carry annotation information across multiple frames. This lets annotators label only the frames where something changes, instead of manually editing the same label on every frame.

Frame interpolation is available for all video file types.

Keyframes

Whenever you create or edit an object or frame-specific classification on a video frame, you are creating a keyframe for that label.

For example, if I create a new bounding box on frame 1, I will have created a keyframe for it on frame 1. Keyframes are marked with white diamonds on the video timeline:

If I navigated to frame 20 and edited the position of the bounding box, I will have created a new keyframe for it:

The position of the bounding box will be interpolated between frames 1 and 20. You can add, remove, and edit keyframes using the video labeling editor interface.

Object Interpolation

For object tools, interpolation means that Hub calculates the object's shape or position on frames between keyframes.

For example:

1. You draw a bounding box around a car on frame 1.

2. You move to frame 20 and move the same bounding box to the car's new position.

3. Hub calculates the bounding box positions for frames 2 through 19.

Interpolation is currently only available for the Bounding Box, Polygon, Segmentation, and Point labeling tools.

Frame-Specific Classifications

Classifications can also be frame-specific. In the project ontology, enable the Frame-specific setting on a classification when answers should apply to individual video frames instead of the whole asset.

Frame-specific classifications do not interpolate geometry. Instead, the answer from the latest previous keyframe applies until another keyframe changes the answer, or until the classification's segment ends.

For example, if a frame-specific radio classification is set to Green on frame 5, Red on frame 15, and Blue on frame 40:

• frames 5 through 14 use Green

• frames 15 through 39 use Red

• frame 40 and later frames in the classification segment use Blue

You can mark frames within the classification's segment as out of view. On those frames, the editor shows the existing answer as read-only. Exports with Only Key Frames disabled omit answers on those frames; exports with Only Key Frames enabled retain the stored keyframes.

Import Format Counterparts

When importing video annotations, the UI concepts above map to these Ango Import Format fields:

UI concept	Import field	What it controls
Timeline row for one tracked object or frame-specific classification	objectId	Groups keyframes that belong to the same object or classification instance.
Start and end of the timeline row	segments	The frame range where the object or frame-specific classification exists.
Frames where the object or answer changes	keyFrames	Frame-number keys containing the object geometry or classification answer for that frame.
Out-of-view spans	outOfViewSegments	Frame ranges where the object is hidden or the frame-specific classification is read-only. Full-frame exports omit these frames.
A single page in multi-image or DICOM assets	page	The page or frame where a non-interpolated annotation belongs.

Video keyframe numbers are frame indexes. They start from 0.

For complete JSON examples, see the video object and video classification sections in the Ango Import Format page.