Image Embedding

An embedding is a numerical representation of an image - a vector that you can index into your IDOL Content component. IDOL Content can compare embeddings to other embeddings, to see whether two images are conceptually similar.

An image embedding encoder is a model that is used to generate an embedding. Before running this task, import an encoder into your training database by using the action ImportImageEmbeddingEncoder.

Sometimes pairs of encoders are trained simultaneously, one to generate vectors from images and another to generate vectors from text. In this case, you can use IDOL Content to search for images that are conceptually similar to keywords.

Configuration Parameter Description
Encoder The encoder to use to generate image embeddings.
EmbeddingPrecision The number of decimal places to use when writing embeddings into the output.
Input The image track to process.
Region A region of the image or video to restrict analysis to.
SampleInterval The interval at which frames are selected to be analyzed.
SegmentDuration The maximum duration of a record in the SegmentedResult track.
SyncDatabase Specifies whether to synchronize with the training database before beginning the analysis task.
Type The analysis engine to use. Set this parameter to ImageEmbedding.

Output Tracks

Output track Description
Data Contains a record, with an embedding, for each processed image or video frame.
Result

Records in the result track contain vectors that are the numerical average of the embeddings that exist, for the same timeframe, in the Data track. When the input track does not identify specific objects, a single result record is generated for the entire video.

When the input track identifies specific events (for example object class recognition results), the result record timestamps match the times of those events. In this case you should usually set Region=Input so that embeddings are generated for identified regions, and not from entire video frames.

SegmentedResult

(Available only when processing video). The same as the Result track, except the duration of each record is limited by the parameter SegmentDuration. If a result exceeds the specified duration, Media Server begins a new record with the same ID.