Calibration

In order to track an external camera, the Kinect must be mounted or constrained to that camera, so that the distance and rotation between them does not change.

We also need a way to measure the offset between these two cameras. NextStage Pro comes with a built in tool for calibrating an external camera. Using this tool NextStage can calculate how far away the external camera is from the Kinect, the difference in rotation between them, as well as the field of view of the external camera.

Once this calibration data has been created and loaded, NextStage can automatically export tracking data with the calibrated information.

Keep in mind that changing the camera configuration in any way, by changing the external camera, camera lenses, or even taking the camera rig apart and putting it back together again will result in different calibration data, and the camera will have to be recalibrated.

This does not mean that calibration data cannot be reused. But careful planning and consideration needs to be taken into account.

Camera Setup

The easiest way to mount the Kinect to an external camera is by building a camera rig using 15mm rails. There are countless rigs, mounts and setups available for purchase, and it will depend on what camera you are using and what the needs are of your shoot.

Fortunately the Kinect has a standard 1/4” tripod mounting thread at it’s base, and should be compatible with most rigs.

The Kinect has an issue in that it freely rotates up and down from it’s mounting plate. You will need to find a way to secure the Kinect so that it does not move at all.

It is recommended that you mount the Kinect on top of the external camera, and place it as close as possible to the lens.

While bulky the Kinect is a very light weight device, placing a larger camera on top of the Kinect could make the rig top heavy and unbalanced.

It also allows the Kinect to see over a subject in the external camera’s field of view. This will give the Kinect a less obstructed view of the markers.

Placing the Kinect close to the lens reduces the distance between the Kinect and the external camera’s nodal point, which should minimize any error caused by this offset.

It is strongly recommended that you only use lenses with fixed focal lengths. It is very difficult to exactly repeat focal lengths on most zoom lenses, and if the field of view changes by even a degree it will have an impact on the final result.

If you have to use a zoom lens lock it in place for the calibration and shoot, or use the minimum or maximum zoom length, as these are easy to repeat.

Finally do not use any lens or internal body stabilization while recording, as NextStage has no way to account for the difference.

Load Calibration

Next Stage will let you choose from a list of previously created calibrations stored in the main Calibration folder. Once loaded any tracking data exported through Sync mode will include the calibration data. The calibration can also be edited, finished, or recalibrated if necessary.

Edit Calibration

This will open up the loaded calibration file in the calibration window. Calibrations are automatically saved and can be finished or continued at any time.

New Calibration

This will create a new calibration and open up the calibration window.

Calibration Chessboard

Calibration works by having the Kinect and the external camera take a picture of the same black and white chessboard at the exact same time.

By calculating the difference between the two images, NextStage can estimate the difference in position and rotation between the two cameras.

You will need to take several of these pictures, with the chessboard at different locations. The more images you take, the more accurate the calibration will be. The minimum recommendation is 10.

You could also keep the chessboard still and move the camera rig around to get these positions. The important this is that the distance between the Kinect and the External camera does not change in any of the pairs of images.

A chessboard image is supplied with NextStage Pro or can be downloaded here to be printed.

Chessboard Inputs

Before you begin capturing the positions you must tell NextStage how big the chessboard is. There are 3 inputs to do so.

Width

This is how many corners wide the chessboard is. A corner is defined as the point where 4 squares meet.

Remember that this number is not how many squares wide it is, it is how many corners wide it is on the “inside”.

This defaults to the width of the supplied chessboard.

Height

This is how many corners high the chessboard is. This defaults to the height of the supplied chessboard.

Size

This is the distance between corners in millimeters. It is critical that this is as accurate as possible, otherwise the calibration may not be to scale.

Because the Kinect’s infrared camera has a wide field of view and a low resolution, it’s recommended that you print the chessboard large. 50mm is the recommended distance between chessboard corners.

Capturing Positions

Once everything has been set up, make sure that your computer’s speakers are turned up high.

Begin recording on your external camera and in NextStage press the Capture Position button.

If NextStage can detect a chessboard in the Kinect’s field of view, it will capture the current infrared image and emit a short beep.

This sound is intended to make it easy to find the same moment in the recording from the external camera.

If NextStage cannot detect a chessboard then it will not capture an image and will not emit a beep. Try repositioning the chessboard until NextStage can clearly see the entire board. If it still won’t detect the board make sure the inputs match the size of the board.

Once you have captured enough positions, stop the external camera.

Loading Images

You will need to load the recording from the external camera onto your computer and export each position as a still image.

If your camera records sound there should be a beep at each moment where NextStage captured an infrared frame. Save the frame closest to the beginning of each of these sound as a still image. The beta of NextStage Pro currently supports the import of .jpg and .png images.

If you open up the “Cailibration” folder in the main NextStage folder you will see a folder for each calibration that has been created. Inside each calibration folder is a “Color” folder. This is where the images from the external camera should be saved.

Once the images are in this folder press the “Load Images” button. NextStage will import all of the images from this folder.

You need to name/number these images alphanumerically, so that they are imported in the order that they were captured.

If you are using a stills/video camera it is not recommended that you use “photo mode” to capture these images. Often hybrid cameras will have a different crop factor while taking pictures vs taking videos. It is important that you capture these images exactly as your final video will be captured.

Deselecting Images

NextStage will compare each infrared image to the color image below it. Images can be deselected if the corresponding image was not captured, or the columns are out of order.

Each imported infrared and color image has a semi transparent “X” button in the upper right corner. Pressing this will temporarily deselect that image and it will not be included in the calibration. It will also push all of the images on the opposite row “forward” once spot.

Calibrate

Once all of the images have been imported and are in the correct order, press the “Calibrate” button. NextStage will go through all of the images and calibrate them. Once finished the viewport on the bottom will return to the infrared view.

Calibrations are saved as a plain text .txt file saved inside the current calibration folder. These can be easily opened, read and modified if necessary.

Calibrating after a shoot

It is common that you will not be able to go through the entire calibration process for each camera setup while on a shoot.

In that event, you can finish the camera calibration after the fact, as long as you created a calibration file for each camera setup and captured the chessboard at different positions.

The tracking data will be exported with no offset and a default 50mm field of view. After finishing the calibrations you will need to manually enter the calibration data in your 3D software.

The “Calibration.txt” file has 13 numbers listed as 13 separate lines. They are:

X Position in millimeters
Y Position in millimeters
Z Position in millimeters
X Rotation in degrees
Y Rotation in degrees
Z Rotation in degrees
Horiziontal Field of View in degrees
Vertical Field of View in degrees
Camera Distortion Values

Calibration