04. PRIME VSAR OVERVIEW

PRIME VSAR is a plugin for Unreal Engine 5 and is developed as such. That being said, it installs its own modified version of Unreal Editor.

 

PRIME VSAR brings the following elements to UE5:

• Video I/O.
• Interface with tracking (Cesium).
• Primitives (a set of procedural geometries, for instance to address some business graphic purposes).
• Interface with CAMIO Universe.
• A/B Switch (a tool to manage transitions between media, comparable to a small video mixer/switcher).
• Exposes UE5 and PRIME VSAR core elements to LUA scripting.
• A subset of related functions/materials/components.

 

Most of these elements can be managed via the Configuration Panel.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Creating a PRIME VSAR Project

VSAR Full Build

1. Start PRIME VSAR (Warning: this may take time to load, especially during the first start, (due to the shader compilation) see remark below.
2. Search for “PRIME VSAR”
3. Create a new or open an existing project.

 

‎ 

VSAR Plugin

1. Start Epic’s Unreal Engine (Warning: this may take time to load, especially during the first start, (due to the shader compilation) see remark below.
2. Search for “PRIME VSAR”
3. Create a new or open an existing project.

 

Select PRIME VSAR Example Projects by clicking on it to have it highlighted and enable the Next button.

PRIME VSAR comes with a couple of Virtual Studio samples originating from Unreal marketplace.

As for any Unreal projects, settings may be set up before creating it, but may be later changed.

It is recommended to have Maximum Quality and Desktop/Console by default.

‎PRIME VSAR Configuration Panel

Open the PRIME VSAR Config Panel by clicking on the top ToolBar close to Settings.

PRIME VSAR related info is located here.

The panel is divided into the following sections/tabs:

• General
• Cameras
• Video Output
• Remote control
• Tools

 

‎ 

General

Options listed in this tab can be used while running in Play mode, mainly to decrease performance needs.

 

• On Play Viewport Settings:
  • Nothing: this settings leaves the quality of rendering on default presets.
  • Unlit: play with “Unlit” lightning mode - this decreases performance demands on the rendering
  • Disable Viewport: this completely disables viewport, which saves the most performance for the output rendering, Warning: Screen space reflections do not update and GPU Particles don’t work when the viewport is disabled. There is large memory allocation over time that will require VSAR restart.
• On Play Screen Percentage:
  ‎Reduces rendering of the viewport resolution by percentage
• Hide Player Pawn Mesh:
  ‎(Checkbox checked by default) In Unreal, a default Player Pawn is created at play time as a sphere. Irrelevant for PRIME VSAR usage

  

   

• Show Mouse Cursor:

(Checkbox checked by default) Shows (or not) the cursor in the viewport while in play mode.

 

The About PRIME VSAR link at the bottom of the window will display the about dialog, showing current PRIME VSAR release info. Pressing any tab will switch back to its related content.

 

Cameras

For managing PRIME VSAR cameras.
‎

A PRIME VSAR camera, by default named CH_CesiumCamera (at first glance and for almost all applications) have 2 purposes:

• Generate an output onto Matrox playout ports through HAL or through NDI (Not supported in 2.0.0 release)
• Manage the communication with Cesium and thus the tracking data

In the above image, there is 1 available camera.

To create a new one, click the Add Camera button.

 

⚠️ Maximum number of Cesium Cameras is 12

 

To set/get (and reveal) the video parameters of the camera, just select (click on) it.

 

Note: after upgrading the PRIME VSAR version, in some cases, opening an older PRIME VSAR .uproject with a previously created PRIME VSAR camera may show inconsistencies. It may be useful to delete and recreate it.

 

The right side of the panel shows (a subset of) the Details panel of the selected camera, also available in Unreal’s standard Details one.

 

Transform

This shows the coordinates of the PRIME VSAR camera in space as for any UE Actor.

If Cesium is Activated (see below) these values come straight from Cesium data, and thus cannot be edited.

 

Cesium

 

Synchronization

The Synchronize button allows for forcing immediate re-synchronisation with incoming Cesium data. Useful if you notice some introduced delay between real and virtual.

⚠️Unexpected delay may happen after dropping frames, and/or if you do some editing manipulations inside the Editor.

 

Cesium Activated

If unchecked this camera won’t follow Cesium information. Should be checked for normal operation (camera tracking scenario).

 

Camera Index

Cesium software can handle multiple cameras. This figure selects which camera we want to use. Index starts at 0. This number relates to the Rig in Cesium (for example if we have two rigs in Cesium and we want to use the second one we would input 1 here)

 

 

Advanced Display (rev arrow ▼)

These values display the current value of the cesium Camera. This feature can be used to check that you actually receive data from Cesium. If you move the camera these values should change (this can be verified eg. by using simulation drivers in Cesium).

 

 

 

 

Lens File

Cesium Camera allows loading Unreal Engine lens files (.ulens) as lens distortion files. When Cesium Camera works with a .ulens file within VSAR, the lens file value in the Cesium software must be set to "Unreal Lens File."

This way, Cesium delegates the lens distortion information to the local VSAR file. For more information check Cesium Reference Guide.

 

Video Output

To select which available output will be used for this camera.

The dropdown list is contextual to what outputs are available in the session.

See below regarding the Video Output tab.

Defaults to Hal:0:0 (for a standard workstation with one Matrox board installed).

Video Output - Rendering

Post Process Settings

A list of Unreal camera’s rendering parameters.

 

Optical Center

When it comes to calibration/registration/targeting of the real camera, you need to check the optical center and target positions. Optical center display parameters are set here:

• Optical Center Show

  Displays or not the optical center. During normal operations it should be unchecked.

• Optical Center Color

  Color of the center lines

• Optical Center Thickness

Thickness of the center lines

Note: this displays a cross on the whole screen, both in RGB and Alpha channels, so that it’s always in front of everything when using any keyer with external matte feature.

 

Advanced settings (revealed when clicking on the down arrow ⤈).

Use TAA

Checked by default. For using Temporal Anti-Aliasing on the output. This setting smooths out jagged edges of objects in the scene.

‎⚠️For this to work TAA needs to be also selected in the project setting as the Anti-Aliasing mode.

 

Disable Alpha Capture

To disable management of alpha channels.

Could help increase performances if no alpha is needed. (for most use-cases by very little)

 

Enable Distortion

To be checked when Cesium data is providing lens distortion info.

 

Then a list of Unreal Render Targets is used for generating the output.

Each of them may be visualized in Unreal’s texture editor by double clicking on it.

 

Fill Render Target

RGB components for the rendered video output.

 

Key Render Target

Alpha components for the rendered video output.

 

Final Render Target

Combination of both above.

 

Distortion Render Target

The texture used for rendering distortions.

 

Shadow Render Target

The texture used to render shadows and reflections in Augmented Reality mode.

 

Outputs Scene Capture Advanced Edit Mode

Enabling this will display the Fill_SceneCaptureComponent and Key_SceneCaptureComponent allowing you to edit more advanced parameters.

 

Video Output - Key

Objects in Foreground

Set the list of Actors that are going to be displayed in the Fill and Key signals. Usually used to have objects in front of the talent.

Should usually be used for the Targets (see below).

 

Objects in Matte

Set the list of Actors that are going to be displayed in Key signal only. Usually used to hide parts of the real sets (ceilings, walls,...)

 

Advanced settings (revealed when clicking on the down arrow ⤈).

Enable Additional Key Capture

This option enables additional key filtering for AR Mode when environmental effects are present in the Level (Atmosphere, Fog, Sky, Volumetric Clouds), but is computationally expensive.

 

AR mode

Hides objects in fill that are not in the foreground.

 

AR mode Background

Select a texture as a background signal for AR, usually the video signal from the tracked camera. If the AR texture is cleaned up, AR objects are rendered without a background.

 

Force Alpha

Forces a full white alpha output.

 

Invert Alpha

Inverts the alpha.

 

 

 

Include Attached Actors

Applies to the actors in the Foreground/Matte lists and also all of their children.

 

Matte Plane

The Matte Plane is a surface always facing the camera, and exactly fulfilling the PRIME VSAR camera field of view. If the camera moves or rotates it will follow it (it is attached to the camera).

Its purpose is to immerse the Video In (talent) into the 3D world, in front or behind other compositions

Matte Plane Texture

The texture applied on the object. Usually using the Video Input as a media source.

Matte Plane Enabled

To use it (or not).

Enable Clean Mode

Clean Mode allows to capture the Talent's video feed cleanly, without applying any texture filters or tonemapping. Clean Mode only works with Masked materials, as it uses Custom Depth Stencil.

Original

Without Clean Mode Enabled With Clean Mode Enabled

By default, all VSAR templates are compatible with the Clean Mode. However, if you want to use this feature in your own project, here's how to activate it:

1. Go to Edit
2. Open “Project Settings”
3. Type “custom depth” on the top search bar
4. Under Postprocessing > Custom Depth Stencil Pass select “Enabled with Stencil”

Additional Clean Mode options

Clean Mode consists of two modes that can be combined or used separately. Masked mode works on a material with the Blend Mode set to Masked, while Translucent mode works on a material with the Blend Mode set to Translucent. By default, both are combined to achieve accuracy in RGB and Alpha pixels.

• Show Masked Plane - Enable/Disable Mask mode
• Show Translucent Plane - Enable/Disable Translucent mode

Matte Plane Opacity Bias

Additional control over the opacity of the Alpha channel of the Matte Plane texture (0 -> 1).

Matte Plane Mask Clipping Level

Additional control over the clipping of the Alpha channel of the Matte Plane texture (1 -> 0).

Matte Plane Mask Invert

Inverts Alpha channel of the Matte Plane texture.

‎ 

Translucent Plane depth bias

This control allows to override the intersection between the Matte Plane and the floor, so that the plane can be positioned in front of the floor, to align virtual shadows and reflections with the talent's video signal.

Without Depth Bias With Depth Bias

 

.Matte Plane Distance

Distance of the object to the camera in UE units (centimeters).

📝 Size is aligned to the camera projection making it look like nothing changed when adjusting Plane Distance.

 

Matte Plane Advanced Component Edit Mode

When checked, UE’s Details panel shows up all the parameters of the PRIME VSAR camera, otherwise, only a subset is displayed.

 

Matte Plane Scale Offset

Adds a scale bias to the Plane. The default value is 1 and makes the plane exactly fill the camera viewport.

 

Matte Plane Location Offset

Adds a translation bias to the Plane. The default value is 0. and makes the plane exactly fill the camera viewport.

‎Matte Plane Rotation Offset 1

Adds a rotation bias to the Plane, prior to the translation. The default value is 0. and makes the plane exactly fill the camera viewport.

‎Matte Plane Rotation Offset 2

Adds a rotation bias to the Plane, after the translation. The default value is 0. and makes the plane exactly fill the camera viewport.

‎Compensate Optical Center

Apply or do not apply the Cesium optical center to the plane.

 

 

 

‎ 

Video Output

For managing all possible Outputs:

This Tab displays the following sub-tabs:

• Outputs
• Video Output Configuration
• HAL

 

Outputs

This tab details the parameters related to the video outputs available for any CH_CesiumCamera (see above).

 

Each extra output (NDI) may be easily deleted using the “X” button on the top right corner.

⚠️(NDI is Not supported in 2.0.0 release)

 

The right side panel displays the different parameters related to the selected output, as Details does

.

 

Custom Render Target

when inserted render target, this render target will be used as Output instead of the hardware.

 

Resolution Scaler

increases or decreases the rendered resolution then it gets coveted to output resolution. For example, when we have 1080p resolution and we set this value to 2 we can use this as a custom super sampling anti aliasing, as it will render in 4k and then be scaled to 1080p.

 

⚠️for interlaced mode we recommend using only scale larger than 1.5 or TAA but not both at the same time.

 

Render Targets

Render Targets used for the selected video Output. Also called ‘Render Texture’ in the Game industry, it is used to make an intermediate calculation as a Texture or surface, e.g. to handle the Fill, the Key or any extra composition.

 

Output

Number of Texture Buffers (from Unreal ToolTip)

Number of textures used to transfer the texture from the GPU to the system memory.

A smaller number is most likely to block the GPU (wait for the transfer to complete).

A bigger number is most likely to increase latency.

 

Video Output Configuration

Mostly used for eventually generating new NDI outputs (via the “+ NDI Out” button).

⚠️(NDI is Not supported in 2.0.0 release)

 

Config

Advanced: Entry for managing the video configuration file. Configuration files may be User or Project -wise. This file should not be confused with Hal.xml, which is the HAL configuration file.

Default is User-mode.

The user file is %LOCALAPPDATA%/ChyronHego/Fresh/VideoOutputConfig.json\

The project file is displayed, as a tooltip, when you hover the mouse on top of the drop down box.

 

Logs

Entry for managing On Screen Verbosity. Changes the level of detail of HUD messages in the editor window.

Helpful for monitoring and analyzing real-time efficiency.

When set to Condensed or Verbose, the viewer will display subsequent lines informing the state of the calculation, as long as Viewer’s parameters have Show Stats checked.

The time rate between each line is driven by the On Screen Verbosity Rate (in seconds).

 

For example in Condensed mode, you may read:

Meaning:

E : externally genlock required

L: genlock is locked.

output0_0: logs for the first output

1069: number of drop frames since PRIME VSAR started

0.0: idle time (milliseconds)

3/3: output fifo fill. In normal operation this should be N/N.

output0_1: logs for the second output

1069: number of drop frames since PRIME VSAR started

18.3: idle time (milliseconds). Means that there are 18.3 milliseconds left when rendering a frame. A frame usually last 40ms, or 33.6 ms. So 18.3 is a good score. Note that only the last output will contain the relevant idle time.

3/3: output fifo fill. In normal operation this should be N/N.

input0_0: logs for the first input

0: number of dropped frames since PRIME VSAR started

3/3: input fifo fill. In normal operation this should be 0|1/N. Having 3/3 means that the video is not currently used in the scene

OK: video is present on the incoming BNC.

input0_1: logs for the second input. Same fields as upper.

 

 

HAL

A read only entry displaying how HAL is configured. This configuration can be changed by editing the HAL.xml file and restarting PRIME VSAR.
‎

 

‎ 

Remote Control

For managing communication with external utilities, mainly Cesium (tracking), DataEngine (CAMIO and remote Lua triggering) and Lua scripting.

 

 

Cesium Receiver

Status

Connected is displayed in Green when the communication occurs with Cesium (this does not necessarily mean that PRIME VSAR is receiving proper data, see below Testing). A Red label is displayed if Cesium is not connected (running).

 

The Reconnect button helps forcing reconnection if need be (network interruption, …)

Note:If Auto Reconnect is checked (see below), then reconnection will happen automatically.

 

Socket Addr
‎The name/ip of the machine where Cesium is running

 

Socket Port

The port onto which Cesium delivers data.

Defaults to 7100 for Legacy protocol. (in VSAR “Use Legacy Protocol” under VSAR Config -> Remote Control -> Cesium Receiver -> Advanced, we recommend to not have this checked and use the new protocol instead )

Defaults to 7101 for the new protocol.

Extra advanced Parameters (revealed when clicking on the down arrow ⤈).

Default values should cover most of the usual cases:

 

Activated

Should be checked in normal operation. If unchecked, cesium camera data won’t be read anymore. This can be useful for outputting video for non-cesium cameras.

 

Auto Reconnect

If checked, Cesium will automatically be reconnected when a disconnection happens. In normal operation, it should be checked.

 

Camera Delay

Queue is used to smooth out network data transmission. Incoming data from cesium is appended to the Queue(s) and then used by an individual camera on tick. If the rate of camera data is not genlocked to PRIME VSAR video frame rate, then at some moment a desynchronisation of the queue may happen. VSAR automatically tries to sync the Queue to be as close to the aimed Camera Delay while the camera is not moving.

• CameraDelay - how many frames of delay should be between Cesium and VSAR. This also determines the Queue size, which is CameraDelay times two plus one, In example of delay 3 the Queue size will be 7.
• CameraQueueAutoSyncEnabled - enables auto Synchronization functionality where the frame counter jumps in time to current Queue size - CameraQueueSyncOffset, after camera synchronization has happened camera may jump in space.
• CameraQueueAutoSyncNumPass - number of frames that are desynced before Synchronization is triggered (CameraQueueAutoSyncEnabled has to be enabled)

 

 

Data Engine

Remote triggering is handled via CH Data Engine.

Data Engine remote control is mainly used by Chyron Panels (LAP) and CAMIO (among others).

 

Status

Connected is displayed in green when the connection with DE is alive. If it’s not the case, click Reconnect.

 

Reconnect

Force the reconnection to the DataEngine.

 

IP

IP address of the machine where the DataEngine is running. 127.0.0.1 stands for localhost.

 

Port

Port used by the DataEngine (defaults is 4300)

Extra advanced Parameters (revealed when clicking on the down arrow ⤈):

 

Auto Reconnect

When checked, as soon as the DataEngine disconnects, it will be automatically reconnected.

Reconnect Delay

Timeout in seconds, before the DataEngine is declared disconnected.

 

Lua

Lua Preload File

When PRIME VSAR is started, a Lua initialization is loaded. The default file is: ../../../MtLuaPreload/Main.lua.

It will load several Lua libraries useful for VSAR. If you need to add your own Lua initialization scripts, most of the time, instead of changing the path of this file, it’s better to edit this file and add your own file entries.

Clicking Reload Lua will execute the file content again.

 

Extra advanced Parameters (revealed when clicking on the down arrow ⤈):

 

Bucket

The DataEngine bucket used to communicate with PRIME VSAR (Default: ue4).

 

Lua in Key

Bucket’s key where PRIME VSAR should read the Lua commands to execute (Default: lua_in)

 

gRPC Port

sets port on what the gRPC Server is run on, this connection is important for VSAR Controller service.

 

 

 

 

Intelligent Interface

Intelligent Interface is remote control protocol. Works similar to Prime Intelligent Interface. VSAR receives messages on either UDP or TCP (specified by Protocol) on the specified Port. Received message is decoded by selected Encoding format and then matched against command patterns.

 

MOS Message

Directory is path where should be MOS files located and in this directory will VSAR try to find MOS files in case, that message does not contain a full path to MOS but just its ID or basename. This property can be changed using blueprint node SetMOSMessageDirectory. When changed via blueprint, property editor in VSAR config will not be updated, but value will be changed. To see this change, close VSAR Config panel and open again. Default Error Pattern is used in case no rule will match incoming messages or selected commands in all matched rules are missing some arguments. All of those settings are project specific. During execution of some commands, the opened project can be switched. TCP Buffering Enabled checkbox is visible just if Protocol is selected as TCP. This option is overriding default behavior (handle each packet as a CII message), and is buffering (concatenating) the packets to one big message. Messages need to be delimited by CRLF.

By clicking on the Receiver Rules button, DataTable asset editor containing rules table is opened. Here, there are predefined rules with their patterns, and user can alter patterns, responses or behaviors.

Patterns:

Patterns like the Prime Intelligent Interface consist of literals and arguments. Backslash is reserved character for argument delimiting. Arguments are enclosed in <>. Argument names are limited to alphanumeric characters and underscore. Using _ instead of <ArgumentName> will discard argument value. ( P\PLAY\_\<FunctionName>\\ will discard the first argument and use the second argument as FunctionName) Everything else (outside arguments and discard character _) is considered a literal and will be matched case insensitive.

This means that for Pattern: “TEST::\abracadabra\<ArgOne>\_” following messages would match:

TEST::\abracadabra\1\l
‎teSt::\AbracadabrA\with space\5
‎test::\Abracadabra\with:comma\5
‎test::\Abracadabra\with_underscore\5

For pattern: “SHOW\<key>:<val>\”

show\aaa:bbb\ (key will be “aaa” and val will be “bbb”)

Special arguments:

• <OrderedReplaceablesValues> = backslash delimited arguments
• <NameValuePairs> = \name1\value1\name2\value2\ ….
• <Values> = \name1:value1\name2:value2\ ….

Special Arguments for reply pattern:

• <Command> = whole received message
• <ErrorCode> = 4 digit hexadecimal error code (just in Error Reply)
• <ErrorMsg> = Verbose error message (just in Error Reply)

 

 

 

 

 

 

Error Codes:

 

Tools

A tab for managing specific dedicated tools like the Targets and ABSwitch features.

 

Targets

Used in Tracking mode, Targets are geometrical references (“dummies”) that should reflect physical (XYZ) real positions in your studio. They’re parented to a specific actor usually named CH_TargetDisplayer.

Their main purpose is to have in the 3D (virtual) environment references to the real world, either for calibrating/calibration checking or virtual object positioning.

 

 

If there are no Target Display objects created yet, click Add Target Display to create a new one.

Then, load the .tgt file by either entering the file path or clicking the “...” button, and press Reload Targets.

 

As for Cameras and Video Output tabs, the right side of the panel shows the Details of the selected element.

 

Transform: allows to globally translate, rotate, scale the targets. Note that in normal operation (ie to have a perfect match between Cesium and PRIME VSAR ), the Transform should be set to Identity; that is: “Location=0,0,0”, “Rotation=0,0,0”, “Scale=”1,1,1”.

 

Reload Targets: deletes and reload the targets based on the specified File Path.

 

Destroy Targets deletes all the targets contained in this target displayer object.

 

File Path: specifies the path to the target file (.tgt) to display. Usually, this file is the same as the one used in Cesium.

 

Show Targets: if unchecked the targets won’t be displayed in the scene. In production, it should be unchecked.

 

Adding targets to Highlighted Targets will enhance display of the selected targets. You can control their color with Target Highlight Color, and control the color of other targets with Target Regular Color.

 

It is a good practice to have the targets displayed in foreground (see Video Output - Key above).

 

AB Switch

AB Switch is a tool to feature texture transitions mapped on a 3D object (for instance a virtual screen in your scenes where we expect transitions to happen).

It should behave close to standard A/B (or Program/Preview) transition effects in a mixer.