Skip to content

FreeD

Overview

FreeD is an open camera-tracking protocol, originally introduced by Vinten and now supported by most broadcast pan/tilt heads and camera-tracking systems. The tracker sends a small UDP packet, several times per video frame, containing the camera's pan, tilt and roll, its position in space, and the raw readings of the lens' zoom and focus encoders.

Onlyview uses it to drive a virtual camera that follows a real broadcast camera — for augmented reality, set extension and XR, where the rendered image has to match what the camera actually sees. To do that, the raw tracking data is not enough: Onlyview also needs to know where the camera's sensor sits relative to the tracker, and how the lens behaves at every zoom and focus position. That second part is the Lens configuration described below.

This is camera tracking. It is unrelated to the Tracking study preview mode, which helps design a rig of optical cameras to track performers.

Setup

  • FreeD port: The UDP port the tracker sends to. The usual FreeD port is 6301, which is the default.

Every machine of the show listens on this port itself, rather than receiving the data relayed by the Producer — this avoids adding a network hop of latency to the tracking. The tracker must therefore send its packets to an address all the machines of the show receive, typically a broadcast address of the show network.

Info

The camera identifier carried in the FreeD packets is ignored: the device uses every valid packet arriving on its port. If several trackers are on the network, give each one its own port and create one FreeD device per port.

Device Widget

The FreeD device widget can be placed on a UserScreen. It shows an on/off icon — lit while packets are arriving, off after one second without data — and the live values received: position, rotation in degrees, and the Zoom and Focus encoder readings. Those last two are the raw encoder values, which is what you need when filling in the encoder ranges of the lens.

Lens Configuration

The lens is configured on the FreeD Tracker node, not in the device setup: this lets several nodes read the same tracker with different lenses. Click Edit on the node's Lens property to open the Lens dialog. Changes apply immediately and are undoable; Cancel restores the lens as it was when the dialog was opened.

Camera and mounting

  • Sensor width (mm): Width of the camera's sensor.
  • Aspect ratio: Aspect ratio of the sensor; the sensor height is derived from it.
  • CCD position: Where the camera's sensor sits relative to the point the tracker reports, in metres. Use +X if the sensor is to the right of the FreeD tracker, +Y if it is above it, and +Z if it is behind it.
  • CCD orientation: Rotation of the sensor relative to the tracker, in degrees. Usually a small correction for a head that is not perfectly aligned.
  • Center shift: Offset, in millimetres, between the optical axis of the lens and the centre of the sensor. Zero on a theoretical lens, a fraction of a millimetre in practice. It applies at every zoom and focus position; the calibration table can add a small extra shift on top of it.

Encoder ranges

FreeD reports zoom and focus as raw encoder counts, in a range that depends on the lens. These four fields convert them to a 0–100 % travel:

  • Min/max zoom encoder: The raw zoom values read at full wide and at full tele.
  • Min/max focus encoder: The raw focus values read at both ends of the focus ring.

To fill them in, drive the lens to each extreme and read the raw Zoom and Focus values on the device widget. Getting these wrong shifts the whole calibration, since every entry of the table is addressed through these percentages.

The calibration table

A zoom lens does not have a single field of view: it changes with zoom, and slightly with focus too. Onlyview therefore stores a grid of measurements — the lens is characterised at a number of zoom positions and focus positions, and the values in between are interpolated linearly.

  • # Zoom stops: How many zoom positions the lens is measured at. They are assumed to be evenly spread over the zoom travel: 3 stops means 0 %, 50 % and 100 %.
  • # Focus stops: How many focus positions the lens is measured at.
  • Focus stops table: The focus travel percentage each focus stop corresponds to, one row per stop, in increasing order. Unlike zoom stops these are given explicitly, because many lenses focus beyond infinity — infinity may sit at 90 % of the ring rather than at 100 %. Two stops (near and infinity) are a typical setup.
  • Lens table: One row per zoom/focus combination. The number of rows follows the two counts above; you cannot add or remove rows by hand. The #Zoom and #Focus columns identify the combination each row describes, and you fill in the measured characteristics:
    • Horiz FoV (°): The horizontal field of view of the lens at that stop.
    • Dist to sensor (m): Distance from the sensor to the lens' nodal point. Positive means the nodal point is in front of the sensor. This is the point the virtual camera is placed at, so a wrong value shows up as parallax error when the camera pans.
    • k1, k2: The lens distortion coefficients — barrel or pincushion — used to deform the rendered image the way the real lens does.
    • extra shift X (mm), extra shift Y (mm): A per-stop correction added to Center shift. Should stay close to zero.

Filling it in

Set the two stop counts first: changing them rebuilds the table, and values already entered end up describing a different zoom/focus combination.

Then, for each stop, drive the real lens to that position and measure. The node itself helps here: it draws the current focus and zoom as percentages, with a tick mark for each calibrated stop, and the live marker turns red when you are within 1 % of a stop — so you can park the lens exactly on a stop before measuring it.

Info

Interpolation between focus stops currently only uses the first two rows of the Focus stops table. Calibrating more than two focus positions has no further effect.

ActionGraph Nodes

FreeD Tracker

Reads a FreeD device and turns it into a tracked camera. Available under Devices/FreeD in the node palette.

Properties:

  • Device: The FreeD device to read.
  • Lens: The lens characteristics — click Edit to open the Lens dialog described above.

Input Slots:

  • Oversize: Float — a widening factor applied to the field of view, used to render slightly more than the camera sees. Defaults to 1.25.

Output Slots:

  • Position: 3D Vector — world position of the camera's nodal point, tracker position plus the sensor offset and the nodal distance of the current lens stop.
  • Rotation: Quaternion — orientation of the sensor.
  • Actual FoV: FoV — the field of view of the real camera, interpolated from the lens table.
  • Oversized FoV: FoV — the same field of view widened by Oversize.
  • Aspect ratio: Float — the sensor aspect ratio from the lens.
  • Focus: Float — focus travel, 0 to 1, from the raw encoder value.
  • Zoom: Float — zoom travel, 0 to 1, from the raw encoder value.
  • Lens deformation (K1 & K2): 2D Vector — the distortion coefficients at the current zoom and focus.
  • Sensor size (mm): 2D Vector — sensor width and height.
  • Lens shift (mm): 2D Vector — Center shift plus the extra shift of the current stop.
  • Full tracking data: Camera tracking data — all of the above bundled into a single value, which is what most downstream nodes expect.

All outputs are invalid while no valid packet has been received, or if the selected device no longer exists.

Info

Tracking data always arrives a little late compared to the camera's video. To align the render with the video feed, use the delay nodes, or reduce the perceived latency with the Camera tracking predictor node — see the Delay section of the nodes reference.