05: Pitch Calibration

Camera Calibration

Camera calibration feature for sports is optional. It is available only if you have a license for this feature.

For precise tracking and product placement in correct perspective you have to calibrate cameras. You can get to the calibration dialog using calibration button in the toolbar. Calibration dialog appears in the Product bar.
‎

Open existing calibration

If the cameras have been calibrated already for the particular stadium and the calibration has been saved, you can just load it using Open command from the Calibration bar or Calibration menu. It is strongly recommended to verify the match of the existing calibration. Go through the calibration process straight to the verification part and follow the instruction in the verification section.

Create new calibration

‎Select New calibration from the calibration editor or Calibration menu. Select the appropriate pitch from the available types.

Type	Description	License
	Soccer - Suitable for main and offside cameras. Tracks all pitch lines.	Football license
	Soccer center - Center camera can be tracked with only central part of the pitch. The calibration is much easier then in the full soccer pitch.	Football license
	Tennis - Suitable for camera behind base line.	Tennis license
	Badminton pitch - Suitable for camera behind base line.	Tennis license
	Rugby League - Wide and middle zoom cameras are supported. Tracks all pitch lines. Beware this pitch is not compatible with Rugby Union.	Rugby license
	International Rugby (Rugby Union).	Rugby license
	NFL – American Football. The field is NFL approved.	Rugby license
	NCAA – American Football. The field is NCAA approved.	Rugby license

Calibration

For all cameras do following (1):

• Capture Frames
• Define Line correspondences
• Set Pitch Color
• Configure Active Image Area

And follow with steps common for all cameras (2):

• Calibrate all cameras at once
• Define pitch borders (not available for all pitch types)

When the first step is done, press the Calibrate button. The calibration engine calibrates all cameras at once. When the calibration succeeds, and you are satisfied with the result, press Done button. You return here and continue with borders definition (Borders button).

Capture frames for calibration

‎There are several screen previews in the list (1). Go through the list, move the camera and find shots as much close to the ones in the list as possible. Every time you find corresponding angle of view from the camera, press the Capture button (2). Small camera in the corner indicates the frame has been captured successfully. After capture verify quality of capture frame. In case the frame is blurred, drop the frame using the Remove button (4), which appears after the frame was captured. You don't have to keep order of frames given by the preview images.

After all frames has been captured, press the Done button to proceed back to camera selection and following calibration phases.

‎Generally not all presented views must be captured. Higher number of corresponding views will give you higher probability of good match and better tracking quality.
‎
‎

Line calibration

‎Select a line (point) in the model (1) and draw corresponding line on the pitch in the video. The most important lines for each frame are drawn in white color in the model (1). Make sure, all important lines are calibrated. In case of the unimportant lines, higher number of definitions implies lower probability you will have to return and correct them. But it is not necessary.
‎It can happen, only part of a line is visible in the video frame. In this case just draw line in the video for the visible part of the pitch line. Don't skip it.
‎Actually selected line is red colored in the model. Repeat calibration step for all previously captured frames.

To go to next frame, press the Next Frame button. You can delete already defined lines/points from the pop-up menu above the model pitch resp. To remove all lines at once, there's 'Remove all' function accessible from popup menu triggered outside any pitch line/point. You can also redefine any line. Just select the line in the model, it appears in the video and draw the line again on the desired position.When you're done, press Done button and return back to the main stage.

How to draw calibration lines over pitch lines

‎The calibration line should be as close as possible to the real pitch line. The length of the calibration line does not necessarily be the same as the real line. Because of camera lens distortion and pitch curvature the lines in video are not straight. It is therefore sometimes impossible to draw the calibration line to exactly match the pitch line. In this case try to draw the line shorter in the middle, most straight segment of the real line. See following examples:

This is picture demonstrates wrong calibration line. The line on the sides does not match.

This is picture demonstrates wrong calibration line. The line in the middle does not match.

This is picture demonstrates proper calibration line. It is shorted but matches the pitch line well.

American Football Point correspondence guidelines

‎Calibration of field for American Football (NFL, NCAA) needs certain points correspondences to be defined. These points are located at intersection of the solid 5 yard lines with 1 yard hash marks.
‎

Sidelines

‎Sidelines for American Football and similar fields that use very thick Sidelines need the calibration lines to be drawn over the inner side of the pitch line. Example:

Image correspondence in soccer center calibration

The soccer center pitch has very simplified calibration. Because the amount of information that can be extracted from the image is much lower than in normal soccer pitch tracking, extra attention must be paid during the calibration.

To calibrate it properly, please follow these rules:

• The center line must always be calibrated. If the line is curved in the image (due to the lens distortion or pitch curvature) then it is better to draw the line closer to the center then to line ends in the image.
• At least one horizontal pitch edge line must be calibrated. Either the top or bottom.
  ‎There are four points in the UI for the center circle. You must calibrate all of them. It is important to place the points on the circle in the image. It is not so important where on the circle the point is. It just needs to have 4 points on the circle roughly equally spaced.

Calibration of pitch grass color

Correct setting of the pitch mask is very important for correct functionality of the tracking. It is fully independent on the chroma keyer used for graphics insertions.

1. Set the pitch mask color to fit the grass color. You can do it from live input or from images captured in previous step (Capture images)
2. Check the mask periodically during the game. When the light conditions change, alter the mask to fit actual pitch color.
3. Check the mask when the tracking becomes inaccurate.

You are allowed to adjust only Hue/Saturation. Change those values until you find good result for all examined images. Suitable pitch mask is shown in the picture below.
‎

Calibration of active picture area

‎In case, the input video contain black border or any mess, don't forget this step. Otherwise the black borders can negatively influence tracking quality.

‎Get to the active area calibration from cameras panel. You can see a rectangle enclosing the whole video area. Drag the four draggers (1) in the middle of each side so the rectangle covers only active area. Black borders (2) are drawn with hatched-pattern.
‎

Calibration

When all the camera-wise steps are done, for all cameras you wish to be tracked, continue to the calibration process.

Screen with calibration progress appears.

The duration of calibration process depends on number of calibrated cameras and pitch type. It could be from 5 seconds to 3 minutes.

Verification of the result

Go through all frames of all cameras using the Navigation panel (1) in the list and check the quality of the match between computed and real pitch lines. Computed lines are shown in the video yellow colored. If this happens or quality of match is poor (computed lines don't fit the real ones), go back to the previous section “Calibration” using the Repair button (2) and redefine the correspondences. In case the calibration cannot be computed for the next set of correspondences, try to recapture the problematic frame.

When the calibration results match, press the Done button to go to the main stage and continue with border calibration.

Calibration of pitch borders

This stage gets enabled after successful calibration.
‎The border calibration is made on pitch top-view, which is generated during the calibration from the captured images. You can also see the result of the calibration, yellow pitch. This pitch should fit the real pitch lines.

‎The pitch area, beyond the lines, can contain some additional advertisement or other objects like roof of the substitute bench etc.

‎How to calibrate pitch borders

‎This step is for the camera tracker to let it know where to look for pitch lines. Define the border on the edge of pitch grass. Don't follow the pitch lines. If any advertisement lies between the pitch line and the grass edge, draw the line between the pitch line and the advertisement.
‎Properly defined grass borders are very critical when the lines are badly visible. Please have a look at the picture below to learn how the borders should be well defined.

In the pitch-side selector (1) select area containing such objects. Draw a line in the video between that object and pitch line (2). Switch area (1) to define borders for other sides. Repeat for all cameras.

Basics of the line painting mechanism:

Action	Steps
Define new line	Simply draw a pointed line.
Extend existing line	Click on one of the end points and continue with the line.
Change shape of existing line	Select point on the line and move the point. The line is divided
Remove existing line	Drag the end point above the start point and release the mouse. Or use clear function from the pop-up menu on the area buttons (1).
Finish existing line	Click on the end point. Or change border area (1).

The result should like shown on following picture (the red lines).

Good and bad practices in calibration border

 

 

Semi-automatic Pitch Tracking

Sometimes can happen, the tracking gets lost. It might be caused by badly visible lines, the camera must not have any line the current view etc. Nevertheless Virtual Placement 7 allows you to work with products localized on the pitch in such situation as well. If you reach any scene, where the tracking is lost or incorrect, simply switch to the semiautomatic tracking. This correction is very fast.

‎Starting Semi-automatic Tracking

This type of tracking is started by definition of the first point. You can use points shown in the following dialog. The points in the video are selected by the right mouse button. You can do that anytime you want, when a valid camera calibration is loaded

1. Select with right mouse button a point in the video, corresponding with one of the points shown above. The Point Correspondence dialog is displayed. Select the same point in the top dialog.
2. You should see yellow pitch on the top of video, showing the computed position of the pitch. If it approximately fits the pitch, you can set the product and begin with production. If not, repeat step 1 for one more point.
3. The tracking is set, adjust the product and put it on-air.
4. When the production is done, you must manually switch back to the automatic tracking. You can do that by pressing the semi-automatic button in the toolbar, which is flashing during the time, when semiautomatic tracking is on (1).
5. Anytime you want to use the semi-automatic tracking again, just begin from 1.

As you can see in the following picture, two points should be enough to define the pitch. Semi-automatic tracking is designed for short productions (like free kick) and might get inaccurate after longer time. 

Don't forget every time after productions switch back to the automatic mode.

Control of the semiautomatic tracking is designed for very fast use. The graphic operators will master the setting very quickly and will be able to prepare the production within 3-4 seconds, even when the tracking gets lost.

Tracking status

During production you can oversee the current tracking status. The tracking results depends on video quality, lines visibility and calibration quality. The status is just an indicator for operators. Most of the time the tracking should be good. Actual tracking status is shown for each camera. Order of the cameras is same as in the Camera button popup menu. If a Live camera is selected, the info is shown for all live cameras. If a Recorder camera is selected, info for Recorder cameras is shown.

‎The tracking status has following states:

State	Description
	No tracking, the system cannot recognize the pitch. You cannot place any objects on the pitch.
	Pitch was recognized short while ago but is not recognized now. The system tries to recover in the background. The tracking quality may vary is this state. It is acceptable to have this tracking state for short periods of time.
	Pitch tracking works reliably. You should see this status most of the time during production.
	Semi-automatic tracking. To disable it, select “rescue circle“ button on the toolbar.
	Tracking information is not available.

Display tracking result

‎All the time you can see the actual state of the tracking. Press the Show pitch button. The system draws pitch lines to the video (to preview only, video output not affected). Those lines should fit the real pitch lines. Displaying of the tracking result can be disabled by clicking the Show pitch button again.

Pitch color

‎When you choose “Display tracking result”, it may happen the lines might be poorly visible due to color conflict of the lines and the real environment (real pitch color). You may change the color of the drawn pitch outline in main application Menu → View → Pitch Color.