Main Content

Get Started with the Video Labeler


You can use labeled data to validate or train algorithms such as image classifiers, object detectors, and semantic and instance segmentation networks. Consider your application when choosing a labeling drawing tool to create ROI labels. For more details on how to select the right label type and drawing tool for your application, see ROI Labels, Sublabels, and Attributes.


  • Manually label an image frame from a video.

  • Automatically label across image frames using an automation algorithm.

  • Export the labeled ground truth data.

ROI and Scene Label Definitions

  • An ROI label corresponds to either a rectangular, polyline, pixel, or polygon region of interest. These labels contain two components: the label name, such as "cars," and the region you create.

  • A Scene label describes the nature of a scene, such as "sunny." You can associate this label with a frame.

Load Unlabeled Data

Programmatically open the app and load a video . Videos must be in a file format readable by VideoReader.

You can also load a video from the app. Open the app from the Apps tab, under Image Processing and Computer Vision. Click Import to load a video or image sequence, or click Open Session to load a saved session.


To load an image sequence with corresponding timestamps, select Import > Image Sequence. The app supports all image file formats supported by imread. To read additional file formats, you can create an imageDatastore and use the ReadFcn property.

To load a custom data source that cannot be read by VideoReader or imread, see Use Custom Image Source Reader for Labeling.

Create Label Definitions


Set Time Interval

You can label the entire video or start with a portion of the video. In this example, you label a five-second time range within the loaded video. In the text boxes below the video, enter these times in seconds:



  3. In the End Time box, type 10.

Optionally, to make adjustments to the time range, click and drag the red interval flags.

The entire app is now set up to focus on this specific time range. The video plays only within this interval, and labeling and automation algorithms apply only to this interval. You can change the interval at any time by moving the flags.

To expand the time range to fill the entire playback section, click Zoom in Time Range.

Create ROI Label

An ROI label is a label that corresponds to a region of interest (ROI) in an image. You can define these ROI label types.

  • Rectangle — Draw 2-D rectangular bounding box labels around objects in an image, such as vehicles, boats, buildings.

  • Projected cuboid — Draw 3-D bounding box labels around objects in an image, such as vehicles, boats, buildings.

  • Line — Draw linear ROIs to label lines, such as lane boundaries.

  • Pixel label — Draw pixels to label various classes, such as road or sky, for semantic segmentation. For more information about pixel labeling, see Label Pixels for Semantic Segmentation.

  • Polygon — Draw polygon labels around objects. You can label distinct instances of the same class. For more information on drawing polygon ROI labels for instance and semantic segmentation networks, see Label Objects Using Polygons

For more details about these ROI label definitions, see ROI Labels, Sublabels, and Attributes.

In this example, you define a vehicle group for labeling types of vehicles, and then create two Rectangle ROI labels, Car and Truck. After creating labels, you can use the Show ROI Labels drop-down menu to select On Hover, Always, or Never to control how the ROI label names appear during labeling. By default, the names will appear when you hover on an ROI.

  1. In the ROI Labels pane on the left, click Label.

  2. Create a Rectangle label named Car.

  3. Optionally, change the label color by clicking the preview color.


  5. Click OK.

    The Vehicle group name appears in the ROI Labels pane with the label Car created. You can move a label in the list to a different position or group in the list by left-clicking and dragging the label up or down.

  6. To add the second label, click Label. Name the label Truck and make sure the Vehicle group is selected. Click OK.

  7. Use the mouse to draw rectangular Car ROIs around the two vehicles.

Create Sublabels

A sublabel is a type of ROI label that corresponds to a parent ROI label. Each sublabel must belong to, or be a child of, a specific label defined in the ROI Labels pane. For example, in a driving scene, a vehicle label might have sublabels for headlights, license plates, or wheels. For more details about sublabels, see ROI Labels, Sublabels, and Attributes.

Define a sublabel for headlights.

  1. In the ROI Labels pane on the left, click the Car label.

  2. Click Sublabel.


    The headlight sublabel appears in the ROI Labels pane. The sublabel is nested under the selected ROI label, Car, and has the same color as its parent label.


  4. In the ROI Labels pane, select the headlight sublabel.


    Draw headlight sublabels for each of the cars.

  6. Repeat the previous steps to label the headlights of the other car. To draw the labels more precisely, use the pan and zoom options located in the upper-right corner of the labeling window.


Show or Hide Labels and Sublabels

You can show or hide the labels or sublabels in a labeled video by using the Eye icon icon on the ROI Labels pane. The Eye icon appears only after you define a label or sublabel. By default, the app displays all the labels and the sublabels.

To hide a label or sublabel, click on the Eye icon icon along side the label or sublabel name. The app hides the corresponding label or sublabel and displays the Hide icon icon.

The show or hide option is available only for the Rectangle, Line, Polygon, and Projected cuboid ROI labels.

Show labels

Hide labels


Create Attributes

An ROI attribute specifies additional information about an ROI label or sublabel. For example, in a driving scene, attributes might include the type or color of a vehicle. In the ocean scene, attributes might include the type of sailboat or number of sails. You can define ROI attributes of these types.

  • Numeric Value — Specify a numeric scalar attribute, such as the number of doors on a labeled vehicle or the number of sails on a sailboat.

  • String — Specify a string scalar attribute, such as the color of a vehicle or boat.

  • Logical — Specify a logical true or false attribute, such as whether a vehicle is in motion or a boat is making way.

  • List — Specify a drop-down list attribute of predefined strings, such as make or model of a vehicle or boat.

For more details about these attribute types, see ROI Labels, Sublabels, and Attributes.

Add an attribute for the vehicle type.

  1. In the ROI Labels pane on the left, select the Car label and click Attribute.


  3. In the List Items section, type different types of cars, such as Sedan, Hatchback, SUV, and Wagon, each on its own line. Optionally give the attribute a description, and click OK. You can hover the information icon that appears next to the attribute field to display the added description.

  4. Select a Car ROI label in the image frame. In the Attributes and Sublabels pane, select the appropriate carType attribute value for that vehicle.



  1. In the ROI Labels pane on the left, select the headlight sublabel and click Attribute.





Create Scene Labels


Create a scene label to apply to an image.

  1. In the left pane of the app, select the Scene Labels tab next to the ROI Labels tab.

  2. Click Define new scene label, and in the Label Name box, enter a scene label named daytime.

  3. Change the color of the label definition to light blue to reflect the nature of the scene label. Under the Color parameter, click the color preview and select the standard light blue colors. Then, click OK to close the color selection window.

  4. Leave the Group parameter set to the default of None and click OK. The Scene Labels pane shows the scene label definition.

  5. Click Apply to Image to apply the daytime label to the scene. A checkmark appears for the scene label.

  6. To edit or delete a scene label, right-click on the label and select either Edit Label or Delete Label.

Label Ground Truth


Label Ground Truth Manually



Label Ground Truth Using Automation Algorithm





  3. (optional) Configure the automation settings. For example, make sure that Import selected ROIs is selected so that the Car labels you selected are imported into the automation session.



  6. Examine the results of running the algorithm.






Export Labeled Ground Truth






gTruth = 

  groundTruth with properties:

          DataSource: [1×1 groundTruthDataSource]
    LabelDefinitions: [2x6 table]
           LabelData: [531×3 timetable]

Data Source

DataSource is a groundTruthDataSource object containing the path to the images or video and timestamps. Display the properties of this object.

ans = 

groundTruthDataSource for a video file with properties

    TimeStamps: [531×1 duration]

Label Definitions



ans =
  3×6 table

      Name         Type        LabelColor        Group       Description     Hierarchy  
    _________    _________    ____________    ___________    ___________    ____________

    {'Car'  }    Rectangle    {1×3 double}    {'Vehicle'}    {0×0 char}     {1×1 struct}
    {'Truck'}    Rectangle    {1×3 double}    {'Vehicle'}    {0×0 char}     {0×0 double}
    {'Sunny'}    Scene        {1×3 double}    {'Weather'}    {0×0 char}     {0×0 double} 


Display the sublabel and attribute information for the Car label.

ans = 

  struct with fields:
       numDoors: [1×1 struct]
          color: [1×1 struct]
       inMotion: [1×1 struct]
        carType: [1×1 struct]
      headlight: [1×1 struct]
           Type: Rectangle
    Description: ''

Display information about the headlight sublabel.

ans = 

  struct with fields:
           Type: Rectangle
    Description: ''
          Color: [0.5862 0.8276 0.3103]
           isOn: [1×1 struct]

Display information about the carType attribute.

ans = 

  struct with fields:

      ListItems: {3×1 cell}
    Description: ''

Label Data


Display the first few rows of the timetable. The first few timestamps indicate that no vehicles were detected and that the sunny scene label is false. These results are because this portion of the video was not labeled. Only the time range of 5–10 seconds was labeled.

labelData = gTruth.LabelData;
ans =

  8×3 timetable

       Time           Car            Truck        sunny
    __________    ____________    ____________    _____



ans =

  8×3 timetable

       Time           Car            Truck        sunny
    __________    ____________    ____________    _____



Display the bounding box positions for the vehicles at the start of the time range. Your bounding box positions might differ from the ones shown here.

ans =

  1×4 single row vector

  415.8962   82.4737  130.8474  129.3805

ans =

  1×4 single row vector

  235.2182    1.0000  117.0611   55.3500

Save App Session


At any time during a session, you can select New Session to start a new session. You have the option of saving the current session or cancelling.


See Also



Related Topics