08: Pitch Calibration and Spatial Workflows

3D Pitch

3D Calibration

Calibration Editor

	Default state. The calibration editor is not active. No perspective has been set for the current clip. Click the button to enter the calibration editor.
	The calibration editor is active. You can setup the perspective or see tracking results. Click the button to save the current perspective and leave the calibration editor.
	Perspective has been set and tracking is good. The calibration editor is not active. Click the button to enter the calibration editor.
	Calibration is disabled. This usually means that no clip is selected.
	Rotate Calibration Wireframe

Perspective definition

	Default state. The point is ready to be set.
	The point cannot be set. This happens when there are 2 other points already set that form a line with this point.
	The point has been set.

Now jog through the video sequence in which you would like to use the 3D features. If the calibration starts to drift away, fix it by defining another keyframe.

When done, press the Calibration editor button which saves the calibration to the clip.

When a point is placed on the wrong position in the video, you can use the Undo button to undo the last edit or Clear button the clear keyframe.

Occasionally, it could be the case that the computed perspective is wrong. In this case just define another point in the current keyframe. When the computed perspective doesn't allow you to define another point (e. g. all points are projected beyond the screen boundary), press and hold the CTRL key (which causes the pitch to be rendered in default projection while the key is down) and define the point. If this doesn't help, press the Clear button and start the keyframe again.

For calibration of behind-goal camera angles you can rotate the pitch wireframe by calling the RotateWireframe action.

Using Paint tools with the 3D pitch

Some tools, like the 3D arrow and the Column tool, are inherently dependent on an existing pitch calibration. Many tools support both on-screen and on-pitch use.

For control over placement of tool graphics, tools have a 3D Scale property. With 3D Scale disabled, on-screen graphics are created.With 3D Scale enabled and a valid pitch calibration, the tool creates on-pitch graphics. The 3D Scale value controls the relationship between on-screen and on-pitch size of the graphics.

3D and 2D tools distinction

Based on their ability to be used on-pitch or on-screen, tools are visually differentiated by a small cube icon in the top left corner (when using default skin) once placed into the toolbar.

	A white cube represents tools that strictly require a calibrated pitch. Such tools can only be used with a valid pitch calibration.
	A gray cube indicates tools suitable for both on-screen and on-pitch use. These tools have 3D Scale enabled and act like 3D tools on clips with a calibration and like 2D tools on clips without it.
	Tools that are not marked by any cube icon either cannot be used on-pitch or they have 3D Scale disabled.

3D tools without 3D pitch

In different scenarios, you can be presented with a floating message at the top of the preview. The message can be of an informational, warning or error character and it may also contain a clickable action.

If you attempt to place a 3D tool into a clip without pitch calibration, an error message appears providing a possibility to quickly activate the calibration editor.

Pitch Auto Calibration

The Pitch Auto Calibration plugin is an additional licensed feature. The feature allows for automatic calibration of pitch within a clip. It creates several calibration frames throughout the clip in case the camera moves and the pitch position changes.

Workflow

When a clip is imported to the Paint the pitch calibration will automatically start running in the background once the import is finished. Calibration frames will start to appear from the beginning of the clip.

Progress of the background calibration is visualized by a progress bar over the clip. The clip can also be used normally during the calibration process.

Re-calibrate pitch

The calibration process can also be triggered manually by an action in the clip context menu. If you want to re-track players simply select the Calibrate Pitch option.

Note: Current calibration frames are overwritten!

The calibration process can also be canceled by selecting the Cancel Calibration option in the clip context menu.

Limitations

macOS machines need to have version Big Sur or newer. The speed of the calibration process is dependent on system hardware. M1 Macs and PCs with strong GPUs will perform best. Intel GPU is not recommended for this feature because the calculations may be very slow. Preferred GPU is NVIDIA GeForce GTX 1650 or higher.

Currently not available for recorded clips with multiple inputs, only single input recordings or imported clips.

Sweep Calibration

In Paint 10 you can calibrate up to three game cameras (Live inputs) for Soccer, NFL, NCAA Football and NBA Basketball - Create a calibration before the game by using the new function inside the UI called sweep calibration (blue highlighted tool)
‎
‎

Setup

Feature needs to be turned on inside Paint config under the plugins tab. The Plugin also allows you to load an existing Virtual Placement calibration and use it inside of Paint and select to load previous calibration on startup
‎ 
‎Calibration process

• Select what type of field you are looking to calibrate
• Select the camera you want to calibrate in the UI, in this case left, main or right. (No other angles supported at the moment)
• Perform a sweep from left to right or vice versa on one camera at a time together with the camera operator while inside the sweep calibration. The camera operator needs to be as zoomed in as possible but needs to see all the lines of the pitch during the sweep for the best result.
• Sweep is saved inside the recorded footage and the operator can go back and forward within the video using a shuttle or UI to mark the different lines during the sweep process.
• Stop the video four to five times during the sweep provided by the camera operator and give the system reference of 4 point in each specific stop two times on the left side of the pitch and two on the right side (make sure the stops cover left to right of the field in general)
• Once done press the calibration button inside the UI and move to the next camera.
• When camera calibration exist an overlay of the lines are seen on the pitch when stepping back to live camera

Multi Angle Telestration

Paint has the ability to synchronize clips with footage of the same game from multiple angles and then share 3D graphics between them. All clips need to have pitch calibration defined.

Clip Group Editor

To synchronize multiple clips you first need to create a clip group. It contains information about the synchronization frames and the shared drawings.

To create new or edit existing clip group choose the "Sync Clip" option inside the clip context menu.

An editor will appear over the tools panel showing the content of the clip group with the chosen clip.

There will be a row for each clip with its name and the sync frame visualized as a timer and a screenshot.

Synchronization Frame

Clicking on the clips will select it for the purpose of editing the sync frame. To change it first seek to the desired frame and confirm by clicking the button in the preview. Both timer and screenshot will update.

Add Angle

Click the "Add Angle" button. All usable clips in the bank will start blinking. Clip on one of them to add it or press ESC to cancel operation.
‎Clips that are already in a different group cannot be added.

Remove Angle

Each clip row has a bin button for removal from the group. Removing the last clip will result in group deletion and loss of its synchronized drawings.

Existing Drawings

Clips added to the group can already have some drawings in them. Whether or not they will be converted to synchronized is set by the checkbox in the bottom right.

Exit Editor

"Ok" will save the current group state and close the editor. "Cancel" will revert the group to previous state before editing and close the editor.
‎Selecting another clip in the bank will

• load its group into the editor - if the selected clip is/can be synchronized
  ‎or
• just close the editor - if the selected clip cannot be synchronized.

If some changes were made in the current group a confirmation dialog will ask whether you want to save them.

Visualization

For clarity both synchronized clips and drawings are marked with this camera icon.

Synchronized Drawings

Synchronized drawings are inside the clip group and will be visible in its clips.

There are 2 ways to create synchronized drawings.

1. Create a regular 3D drawing inside one of the synchronized clips and click save.
2. Add a clip which contains some drawings as a new angle.

Switching Angles

When inside a synchronized clip use "PreviousAngle" and "NextAngle" actions to cycle through the angles in one way or the other. Current time and playback speed is maintained during the switch.

In default these actions are bound to:

Win: CTRL + SHIFT + LEFT/RIGHT

Mac: CMD + SHIFT + LEFT/RIGHT

Limitations

The accuracy of MultiAngle telestration depends completely on 3D pitch calibration. Therefore, if one clip is recorded from a difficult angle and the automatic calibration doesn't produce accurate enough result, the drawings of other clips in such a clip will appear shifted and vice versa.

For such clips user can edit or define the calibration manually though the Calibration Editor.