12. Internal Chroma Keyer

Composure Compositing

Internal Chroma Key Overview

Unreal provides two methods for internal chroma keying:

• A single pass chroma key material.
• A multi pass chroma key with the plugin Composure.

The recommended method for Prime VSAR is the chroma key based on Composure.

For more information about Composure take a look into Unreal Engine documentation.

 

For more information about the Chroma Key algorithm developed by Unreal.

Setup

There is a dedicated UI in order to create all the necessary assets needed.

In the Content Browser, right-click to show the contextual menu, go to “PRIME VSAR” category and select “Chroma Keyer Comp”.

You can optionally select assets to create along and link them to the comp.

If it is the first Chroma Keyer Comp created, It is recommended to check all options in order to also create and link all necessary assets. The MediaPlayer will also automatically play the MediaSource in this case.

Assets created when all options are checked, the main asset named “ChromaKeyer”.

Drag and drop the asset “ChromaKeyer” in the level.

Parameters

Input

Select a Media Texture as the input (it is already set if the option to create a Media Texture was checked previously).

Then go under Composure > Transform/Compositing Passes > Transform Passes.

Multi Pass Chroma Keyer

Select the key colors with the color picker .

Multiple key colors can be added but be aware that each color picked adds an additional rendering pass.

Despill

Select the key colors with the color picker .

Multiple key colors can be added but be aware that each color picked adds an additional rendering pass and thus increasing demands on performance.

‎Despill off

Despill on

Erode

Erodes outline of the keyed subject, useful for removing spilled color on hair outline.

Crop

Normalized crop from each side in range [0-1]

The alpha channel can be cropped as a garbage matte.

Output

The output resolution is adjustable, it should match the input’s resolution (FHD by default).
‎

Billboard for Chroma Key

The result of the chroma key is rendered to a Target Texture. This texture can be used in Material to apply on geometries. For the Trackless solution, we are displaying the texture on a billboard (i.e. a plane always facing the camera).

There are multiple ways to create a billboard in PRIME VSAR.

• Unreal Engine’s native Component “Material Billboard”
• PRIME VSAR’s Billboard Actor
• Matte plane

 

‎ 

PRIME VSAR’s Billboard Actor

From the menu Place Actors > PRIME VSAR Tools, drag and drop the Actor “BillboardPlane” in the level.

Select the Actor and choose a “Target”, i.e. the Object the plane should “look at” typically the Cesium Camera and choose the Texture from the Chroma Key as the “Input Texture”

📝 Billboard Plane does only rotate towards camera on the Z axis (Yaw), It does not Pitch towards the camera.

Input Texture

The texture that feeds the Billboard, usually a signal with alpha either from the internal VSAR Chroma Keyer, or an external Chroma Keyer.

Enable Clean Mode

Clean Mode allows to capture the Talent's video feed cleanly, without applying any texture filters or tonemapping. Clean Mode only works with Masked materials, as it uses Custom Depth Stencil.

Original

Without Clean Mode Enabled With Clean Mode Enabled

By default, all VSAR templates are compatible with the Clean Mode. However, if you want to use this feature in your own project, here's how to activate it:

1. Go to Edit
2. Open “Project Settings”
3. Type “custom depth” on the top search bar
4. Under Postprocessing > Custom Depth Stencil Pass select “Enabled with Stencil”

Additional Clean Mode options

Clean Mode consists of two modes that can be combined or used separately. Masked mode works on a material with the Blend Mode set to Masked, while Translucent mode works on a material with the Blend Mode set to Translucent. By default, both are combined to achieve accuracy in RGB and Alpha pixels.

• Show Masked Plane - Enable/Disable Mask mode
• Show Translucent Plane - Enable/Disable Translucent mode

Translucent depth bias

This control allows to override the intersection between the Matte Plane and the floor, so that the plane can be positioned in front of the floor, to align virtual shadows and reflections with the talent's video signal.

Without Depth Bias With Depth Bias

Opacity Bias

Additional control over the opacity of the Alpha channel of the Matte Plane texture (0 -> 1).

Mask Clipping Level

Additional control over the clipping of the Alpha channel of the Matte Plane texture (1 -> 0).

Invert Mask

Inverts Alpha channel of the Matte Plane texture.

 

Matte plane for Chroma Key

Plane directly attached to the Cesium camera, VSAR 2.3.1 Setup Guide has information about matte plane parameters.

To use the output of the VSAR Chroma Keyer, or the output of an external Chroma Keyer, within the Matte Plane, you must assign that texture as the Matte Plane texture within the Matte Plane parameters in the Cesium Camera. An RGBA texture is expected.

‎