12: Internal Chroma Keyer

Composure Compositing

Internal Chroma Key Overview

Unreal provides two methods for internal chroma keying:

• A single pass chroma key material.
• A multi pass chroma key with the plugin Composure.

The recommended method for Prime VSAR is the chroma key based on Composure.

For more information about Composure take a look into Unreal Engine documentation.

For more information about the Chroma Key algorithm developed by Unreal.

Setup

There is a dedicated UI in order to create all the necessary assets needed.

In the Content Browser, right-click to show the contextual menu, go to “PRIME VSAR” category and select “Chroma Keyer Comp”.

You can optionally select assets to create along and link them to the comp.

If it is the first Chroma Keyer Comp created, It is recommended to check all options in order to also create and link all necessary assets. The MediaPlayer will also automatically play the MediaSource in this case.

Assets created when all options are checked, the main asset named “ChromaKeyer”.

Drag and drop the asset “ChromaKeyer” in the level.

Parameters

Input

Select a Media Texture as the input (it is already set if the option to create a Media Texture was checked previously).

Then go under Composure > Transform/Compositing Passes > Transform Passes.

Multi Pass Chroma Keyer

Select the key colors with the color picker .

Multiple key colors can be added but be aware that each color picked adds an additional rendering pass.

Despill

Select the key colors with the color picker .

Multiple key colors can be added but be aware that each color picked adds an additional rendering pass and thus increasing demands on performance.

‎Despill off

Despill on

Erode

Erodes outline of the keyed subject, useful for removing spilled color on hair outline.

Crop

Normalized crop from each side in range [0-1]

The alpha channel can be cropped as a garbage matte.

Output

The output resolution is adjustable, it should match the input’s resolution (FHD by default).

Billboard for Chroma Key

The result of the chroma key is rendered to a Target Texture. This texture can be used in Material to apply on geometries. For the Trackless solution, we are displaying the texture on a billboard (i.e. a plane always facing the camera).

There are multiple ways to create a billboard in PRIME VSAR.

• Unreal Engine’s native Component “Material Billboard”
• PRIME VSAR’s Billboard Actor
• Matte plane

 

“Material Billboard” Component

Unreal Engine’s “Material Billboard” Component added to an Actor

PRIME VSAR’s Billboard Actor

From the menu “PRIME VSAR Tools”, drag and drop the Actor “BillboardPlane” in the level.

PRIME VSAR Tools menu

Select the Actor and choose a “Target”, i.e. the Object the plane should “look at” typically the Cesium Camera and choose the Texture from the Chroma Key as the “Input Texture”

📝 Billboard Plane does only rotate towards camera on the Z axis (Yaw), It does not Pitch towards the camera.

Clean Mode

Billboard’s Clean Mode allows to capture the Talent's video feed cleanly, without applying any texture filters or tonemapping.

Original

Without Clean Mode Enabled

With Clean Mode Enabled

Matte plane

Plane directly attached to the Cesium camera, VSAR 2.2.0 Setup Guide has information about matte plane parameters.

Setting for Matte plane can be changed in when clicking on cesium camera in the Matte plane section or in the VSAR Config → Cameras section

• Enable with the checker box “Matter Plane Enabled”
• Create material from the RenderTarget
  ‎right click on the RenderTarget for the chroma key and click “Create Material”
  ‎
• Open the material and set the blend mode to Translucent | Masked
  ‎
• Plug the A (alpha value) from the Texture sample to the Opacity | Opacity Mask
  ‎
• (Optional step) set the “Shading Mode” to Unlit and plug the RGB from the Texture sample to Emissive color - this might be something that you want to make the Matter Plane not get affected by the lighting in level.
  ‎
• Assign this new material to “Matte Plane Material”
  ‎

To set up shadow in the Matte Plane material when using translucent blend mode:

• In the advanced section of the material section, enable “Cast Dynamic shadow as Masked”
  ‎

To set up reflection for matte plane with AR Plane:
‎Add Cesium camera actor itself into “Objects in Foreground” in Cesium camera details.
‎