Skip to content

Stype

Overview

The Stype device receives real-time camera tracking data from a Stype system (RedSpy, Follower, and other Stype trackers). Each packet carries the camera's position and orientation, together with the live lens data read from the lens encoders: field of view, aspect ratio, focus, zoom, lens distortion and lens shift.

In Onlyview, this data is fed into the ActionGraph and used to drive a 3D object — typically a camera or a projector component — so that the virtual scene follows the real camera. This is the basis of augmented reality and virtual production setups: the tracked camera object can be watched moving live in the 3D preview.

Setup

Create one Stype device per Stype system sending data to the show.

  • Stype port: The UDP port Onlyview listens on for the Stype stream (default: 6301). It must match the port the Stype system is configured to send to.

There is nothing else to configure: the packet format is fixed, and Onlyview listens on every network interface of the machine.

Info

Every machine of the show — the Producer and the Media Servers — listens on this port itself, rather than relaying the data from the Producer. This avoids one network hop of latency, but it means the Stype system must broadcast its stream (or send it to each machine) so that all of them actually receive it.

Malformed packets are discarded, so a mismatched port or another application sending unrelated traffic to the same port simply results in no tracking data rather than in erratic movement.

Device widget

The Stype device can be placed on a UserScreen. It shows the device name and an indicator that lights up while tracking packets are arriving, and goes off if nothing has been received for one second. This is the quickest way to check on site that the stream is reaching a given machine.

ActionGraph Nodes

Stype Tracker

Reads the last tracking data received by a Stype device. Available under Devices/Stype in the node palette.

The node has no events: it simply exposes the latest received values, so its outputs are normally read every frame (from an OnFrame event elsewhere in the graph).

Properties:

  • Device: The Stype device to read from.

Input Slots:

  • Oversize: Float — the enlargement factor used to compute Oversized FoV (default 1.25). See below.

Output Slots:

  • Position: 3D Vector — the camera position, relative to the Stype system's own origin. See Calibrating the Stype origin below.
  • Rotation: Quaternion — the camera orientation, converted from the pan / tilt / roll sent by Stype into Onlyview's coordinate system.
  • Actual FoV: FoV — the real horizontal field of view of the camera.
  • Oversized FoV: FoV — the same field of view widened by the Oversize factor.
  • Aspect ratio: Float — the camera's aspect ratio.
  • Focus: Float — the focus value reported by the lens encoders.
  • Zoom: Float — the zoom value reported by the lens encoders.
  • Lens deformation (K1 & K2): 2D Vector — the two lens distortion coefficients, to be fed to the AR LensDeformer effect.
  • Sensor size (mm): 2D Vector — the size of the camera sensor, in millimetres. The height is derived from the width and the aspect ratio.
  • Lens shift (mm): 2D Vector — the optical centre shift of the lens, in millimetres.
  • Full tracking data: Camera tracking data — all of the above bundled into a single value, which is much more convenient to route across a large graph. Use the Split camera tracking data node to unpack it again where it is needed.

All outputs are invalid until the first valid packet has been received. If the stream stops, the outputs keep the last received values.

About the oversized FoV

Lens distortion is applied by a shader that pushes the image outwards, which clips pixels along the borders. To avoid this, the scene must be rendered into a TextureArea larger than the final output, and with a correspondingly wider field of view — that is what Oversized FoV gives you.

With the default Oversize of 1.25, a 1920x1080 output must be rendered into a 2400x1350 TextureArea. If the TextureArea size and the Oversize value do not match, the distortion will not line up. Larger values are safe but cost performance; smaller values may not leave enough margin for a strongly distorted lens.

Calibrating the Stype origin

Stype reports the camera position relative to its own origin — in practice, wherever the crane or the tracker base happens to be. Before the tracked camera lines up with the real stage, that origin has to be located in the Onlyview scene. The usual method is to place the camera at rest, calibrate a "fake" projector component onto it with the manual calibration tool to obtain its world-space position, then let the Compute pivot node derive the origin from the difference between that world position and the local position given by Stype. In the scene, the tracked camera is then a child of an object representing the Stype origin.

The full procedure, and the objects the node expects (Calibrated cam, Stype cam, Stype origin), are described in the Compute pivot section of the nodes reference.

Warning

The manual calibration cannot follow a moving object. Trigger Compute pivot once, with the camera held still, and do not run it again while the camera is moving.

Latency compensation

Tracked data always arrives slightly late compared to the camera's real position, and the video feed it will be composited with has its own delay. Two families of nodes, documented in the Delay section of the nodes reference, deal with this:

  • The Delay nodes hold a value back by a fixed number of frames, which is how you align the tracking data with a video feed that arrives later.
  • The Camera tracking predictor node does the opposite: it extrapolates the camera's position and rotation a few milliseconds into the future to reduce the perceived latency of the tracking itself.