Keyframe Interpolation

Interpolation of an annotation is a process of labelling a moving object by the tool estimating its position across frames between two defined frames. These frames in which the user sets the position of the annotation are called Keyframes. A keyframe can also be created with the Shortcut K.

On creating a keyframe, the tool automatically calculates and repositions the annotations in the interim frames between defined keyframes.

The keyframes are visible on the timeline (blue line under the frames) for a given selected annotation indicating that the position was set by the annotators themselves in those frames.

For instance, in the example below, the object was in motion from Frame 1 to Frame 25 and moved out of the scene. Frame 1, 5, 10, 15 & 25 are keyframes where the annotator manually set the annotation positions, while the intermediate frames have positions calculated by the tool.

To remove a keyframe or bring the annotation back to the tool-calculated position, visit the frame and press K provided the annotation was selected in the prior frame.

Benefits

Efficiency: Reduces the time and effort required to annotate frames individually by automatically propagating annotations across intermediate frames.
Consistency: Ensures uniformity in annotations across sequential frames, minimizing discrepancies and improving accuracy.

Steps to use Keyframe Interpolation

Keyframe Interpolation is used for labelling object(s) in motion. Once a moving object has been identified, the process for Keyframe Interpolation can begin.

Identify the best frame

Identify the frame where the object is most clearly visible — often the frame where the object is closest to the ego vehicle.

Create the annotation

Create the annotation for the object in that frame. Modify its dimensions to annotate accurately and provide the attribute values.

The first time the annotation is created, it inherently becomes a Keyframe.

Propagation

The annotation is propagated across all frames (forward and backward) on the same world coordinates (provided the ego pose data is available in the dataset).

Track and reposition

Track the object by navigating in one direction. After approximately 5–7 frames from the initial frame, reposition the annotation on the object accurately.

As soon as the cuboid is repositioned, the current frame becomes a Keyframe.
The tool automatically adjusts the positions of all annotations between the two Keyframes.

Repeat until object leaves scene

Repeat this process until the object is no longer visible in the sequence.

Deleting annotation and resulting keyframes

Deleting the annotation in any frame will make the immediate next/previous frame a keyframe where the annotation is present.

Examples:

The object is available between Frame 1–9 and not visible on Frame 10. The annotation is created on Frame 1. If the annotation is deleted from Frame 10, then Frame 9 becomes a Keyframe.

The object is available between Frame 11–14 and leaves the scene. The annotation is created on Frame 11. If the annotation is deleted from Frame 15 onwards, then Frame 14 becomes a Keyframe.

Adding Keyframes

A frame turns into a Keyframe when the user does any of the following:

Creates an annotation on any frame.
Makes any position modification to an annotation manually.
Deletes the annotation on a frame which makes the immediate next and/or previous frames the annotation occurs in a Keyframe.
Uses the Shortcut K while an annotation is selected. This makes the frame a Keyframe without making any changes to the positioning of the annotation.

This is useful when the object is both stationary and moving across frames.

Example workflow:

An object is stationary from Frame 1–10 and starts moving on Frame 11.
- Create the annotation on Frame 1 - it propagates across frames in the same position.
- On Frame 10 press K to mark it as a Keyframe.
- On Frame 11 reposition the annotation where the object has started to move - Frame 11 becomes a Keyframe.

By adding the Keyframe on Frame 10, the tool is prevented from interpolating the annotation beyond Frame 1–11.

Removing a Keyframe

Keyframes can be removed by selecting an annotation on a keyframe and pressing the Shortcut K. This shortcut toggles adding and removing a Keyframe.

Label Occluded Objects

When an object is occluded in interim frames, follow these steps:

Create the annotation on the object

Create the annotation on the object (e.g., Frame 1).

Mark a keyframe before occlusion

Go to the frame where the object is visible just before occlusion (e.g., Frame 8) and press K to make it a Keyframe.

Delete annotation for occluded range

Go to the frame where the object reappears (e.g., Frame 12).

Right-click on the selected annotation and choose the option to Delete annotation between a frame range.
Select the frame range (e.g., Frame 9 - 11) and apply the action.

The annotation will be deleted from this range without re-positioning it in frames that were not Keyframes.

Last updated 21 hours ago