Define the Perspective

To understand the effect of perspective on the scene, Media Server requires a projection matrix. This is an array of twelve numbers that it uses to convert locations in the scene image into real-world 3D coordinates.

You can specify a projection matrix in your session configuration, by setting the Projection parameter.

[Session]
Projection=12.0269 0.041904 -12020.4 -0.652671 -18.9201 15430.2 0 0 0 0 1 426.708

Alternatively, you can configure Media Server to calculate the projection matrix during analysis, by observing objects in the scene. In this case, instead of setting the Projection parameter, you set the horizontal field of view of the camera, and define real-world dimensions for objects in the scene.

Set the Projection using Real-World Measurements

When you have physical measurements of the scene, you can configure the projection matrix using the Media Server graphical user interface. This method requires you to choose at least four points in the scene. The points must be on the same plane, but cannot be on a straight line. For example, you could choose four points at ground level as long as the points are not in a line. You must provide the coordinates of the points in the scene image (in two dimensions) and the coordinates of the points in the real world (in three dimensions).

To set the projection parameter using the Media Server GUI

  1. Open the Media Server GUI:

    http://localhost:14000/action=GraphicalUserInterface
  2. Click Configuration Builder.
  3. Open (Import) your session configuration.
  4. In the configuration, click the [Session] section.
  5. Click the Parameters tab, find the Projection parameter, and click .

    The Draw For Perspective dialog box opens.

  6. Ingest video of the scene, either from a file or from the camera, so that you can see the scene image.
  7. Click on the scene image four times, to draw four points. The points must be on the same plane, but cannot be on a straight line. If you make a mistake, you can right-click on the scene image and click Remove all regions and points.

  8. In the World Coordinates area, use your real-world measurements to specify the coordinates of the points that you have drawn. All of the points must be on a plane (for example the road), so you can set the Z-coordinates to zero. You could use the coordinates 0,0,0 for the first point and set the other coordinates appropriately. You can use any units, but the units must match those you use to define the size of objects in the scene.
  9. Click Submit.

    The GUI generates the projection matrix and sets the Projection parameter.

  10. Save (Export) your configuration.

Calculate the Projection During Analysis

Media Server can calculate the projection matrix, by observing objects as they move through the scene. The objects must move on a plane, such as a road with a constant gradient. This approach might be more convenient, because you do not need to provide coordinates of points in the scene.

In this case, instead of setting the Projection parameter, you set the horizontal field of view of the camera, and define dimensions for objects in the scene. Media Server takes a short time to calculate the projection, perhaps ten seconds when there is heavy traffic. Perspective features are not available immediately after processing starts, but are enabled automatically as soon as the projection has been calculated.

To calculate the projection matrix automatically during analysis

  1. Open your session configuration.

  2. Specify the horizontal field of view of the camera (in degrees), by setting the FieldOfView parameter in your object class recognition or vehicle recognition task. Micro Focus recommends that you set this parameter from the camera specifications. For example:

    [ObjectClassRecognition]
    Type=ObjectClassRecognition
    Recognizer=MyRoadUsersRecognizer
    FieldOfView=75

    If you do not set this parameter, Media Server attempts to calculate the correct value.

  3. Specify the size of object classes or vehicle models that will appear in the scene. For information about how to do this, see Define Object Dimensions.

    [ObjectClassRecognition]
    Type=ObjectClassRecognition
    Recognizer=MyRoadUsersRecognizer
    FieldOfView=75
    WorldDimensionsMetadata=dimensions(m)
  4. Save your session configuration.

Calculate the Projection Before Starting Analysis

Media Server can calculate a projection matrix by observing objects, as described in Calculate the Projection During Analysis.

You can use the perspective engine (Type=Perspective) to calculate a projection matrix before you start analysis. This engine should not be used in a production environment. Instead, you take the output from the engine and set the Projection parameter in your session configuration. This requires additional steps, but calculating the projection and setting the Projection parameter in advance has some advantages:

  • You can calculate the projection matrix using a video file, and process that file with a shorter sample interval than is possible when processing a stream. This ensures the projection matrix is calculated with the best possible accuracy.
  • The projection matrix is available as soon as processing starts, so perspective features are enabled immediately.
  • The projection matrix does not need to be calculated again if you stop and then restart processing.

To calculate the projection matrix automatically before analysis

  1. Construct a session configuration that uses the perspective engine to determine the projection matrix. This configuration can prioritize accuracy at the expense of processing speed, because you only need to process a short clip. For example, the following configuration sets SampleInterval=10ms and FullAnalysisRate=1 so that objects are tracked with the best possible accuracy. This can result in a more accurate projection.

    [Session]
    Engine0=Source
    Engine1=ObjectClassRecognition
    Engine2=Perspective
    Engine3=OutputProjectionMatrix
    IngestRate=0
    
    [Source]
    Type=Video
    
    [ObjectClassRecognition]
    Type=ObjectClassRecognition
    Recognizer=MyRoadUsersRecognizer
    NumParallel=8
    FullAnalysisRate=1
    SampleInterval=10ms
    
    [Perspective]
    Type=Perspective
    Input=ObjectClassRecognition.Data
    WorldDimensionsMetadata=dimensions(m)
    
    [OutputProjectionMatrix]
    Type=XML
    Mode=AtEnd
    Input=Perspective.Result
    XMLOutputPath=output/%session.token%/projection.xml
  2. Process a video clip from the camera. For example:

    curl http://localhost:14000/action=Process -F source=@video.mp4 -F config=@MyConfig.cfg
  3. Obtain the projection matrix. The example configuration, above, outputs the projection matrix to an XML file in the output directory.
  4. Set the Projection parameter in the [Session] section of your session configuration file. The session configuration that you use in production should not include the perspective engine.