Protocol messages (v5)

Generated from protocol/imuninn.proto. Comments are the ones in the file; the protocol spec explains the flow, the raw file is imuninn.proto.

Header (message)

field	type	#	notes
stream_id	string	1	Stream this sample belongs to (e.g. “rgb”, “imu”, “ar_pose”, “depth”).
seq	uint64	2	Monotonic per-stream sequence number; gaps mean drops.
device_time_ns	uint64	3	Device monotonic clock, nanoseconds. Same time base for all streams on a device (iOS: ARFrame.timestamp / CoreMotion; see §6.5).
protocol_version	uint32	4	Protocol version this message was produced with (protocol/VERSION).
config_version	uint32	5	Config version in effect when this sample was captured (§6.8).

Platform (enum)

value	#	notes
PLATFORM_UNSPECIFIED	0
PLATFORM_IOS	1
PLATFORM_ANDROID	2

DeviceInfo (message)

field	type	#	notes
device_id	string	1	stable per install
device_name	string	2	user-visible
model	string	3	e.g. “iPhone17,1”
platform	Platform	4
os_version	string	5
protocol_version	uint32	6
app_version	string	7
has_lidar	bool	8

ParamSpec (message)

Self-describing parameter (§6.8 “参数自描述”). PC tools and the phone UI generate controls from this; adding a parameter needs no UI change.

field	type	#	notes
key	string	1	dotted path, e.g. “rgb.exposure.mode”
label	string	2
unit	string	3

ParamSpec.Kind (enum)

value	#	notes
KIND_UNSPECIFIED	0
KIND_BOOL	1
KIND_INT	2
KIND_FLOAT	3
KIND_ENUM	4
KIND_STRING	5

| kind | Kind | 4 | |

ParamSpec.Apply (enum)

When a change takes effect.

value	#	notes
APPLY_UNSPECIFIED	0
APPLY_IMMEDIATE	1	no interruption
APPLY_RESTART_SESSION	2	AR / capture session restarted; streams emit a Gap

| apply | Apply | 5 | | | min | double | 6 | Range for numeric kinds. | | max | double | 7 | | | step | double | 8 | | | choices | repeated string | 9 | Choices for KIND_ENUM. | | current | Value | 10 | | | default | Value | 11 | | | read_only | bool | 12 | |

Value (message)

field	type	#	notes
oneof v
b	bool	1
i	int64	2
f	double	3
s	string	4	enum choice or free string

StreamKind (enum)

value	#	notes
STREAM_KIND_UNSPECIFIED	0
STREAM_KIND_IMU	1
STREAM_KIND_POSE	2
STREAM_KIND_VIDEO	3
STREAM_KIND_DEPTH	4
STREAM_KIND_CONFIG	5	config-change events (recorded as its own MCAP channel)
STREAM_KIND_HEALTH	6	DeviceHealth: thermal, battery, phone-side rates

StreamDescriptor (message)

field	type	#	notes
stream_id	string	1
kind	StreamKind	2
transport	Transport	3
params	repeated ParamSpec	4	parameters scoped to this stream

Transport (enum)

value	#	notes
TRANSPORT_UNSPECIFIED	0
TRANSPORT_UDP	1	IMU / pose: allow loss, latency first
TRANSPORT_TCP	2	video / depth v1
TRANSPORT_WEBSOCKET	3	control; also video/depth for web clients

Capabilities (message)

field	type	#	notes
device	DeviceInfo	1
streams	repeated StreamDescriptor	2
session_params	repeated ParamSpec	3	e.g. “session.mode”
config	DeviceConfig	4	current effective config

SessionMode (enum)

value	#	notes
SESSION_MODE_UNSPECIFIED	0
SESSION_MODE_AR	1	pose available, main camera only
SESSION_MODE_CAMERA	2	full camera control, no pose

DeviceConfig (message)

field	type	#	notes
config_version	uint32	1
mode	SessionMode	2
streams	repeated StreamConfig	3
network	NetworkConfig	4
session_params	map<string, Value>	5	Session-level parameters other than mode, keyed by ParamSpec.key from Capabilities.session_params (e.g. “black_screen”). In a SetConfig patch only the listed keys change. v4.

StreamConfig (message)

field	type	#	notes
stream_id	string	1
enabled	optional bool	2	optional: absent in a SetConfig patch means “unchanged”
params	map<string, Value>	3	keyed by ParamSpec.key (relative)
encoder	EncoderConfig	4
delivery	DeliveryConfig	5

EncoderConfig (message)

field	type	#	notes
codec	string	1	“h264”, “hevc”, “none”, “lz4”, “zstd”
bitrate_bps	uint32	2
keyframe_interval	uint32	3

DeliveryConfig (message)

field	type	#	notes
transport	Transport	1

DeliveryConfig.DropPolicy (enum)

value	#	notes
DROP_POLICY_UNSPECIFIED	0
DROP_POLICY_LATEST_ONLY	1	keep newest frame, drop older (video)
DROP_POLICY_NONE	2	never drop (control / depth)

| drop_policy | DropPolicy | 2 | | | max_rate_hz | uint32 | 3 | 0 = unlimited | | priority | uint32 | 4 | |

NetworkConfig (message)

field	type	#	notes
host	string	1
control_port	uint32	2
data_port	uint32	3
bandwidth_limit_bps	uint32	4	0 = unlimited

ControlMessage (message)

Control channel RPC-style envelopes (WebSocket, reliable).

field	type	#	notes
request_id	uint64	1	echoed in responses; 0 for broadcasts
oneof body
hello	Hello	10	phone -> pc on connect
welcome	Welcome	11	pc -> phone
get_capabilities	GetCapabilities	20
capabilities	Capabilities	21
get_config	GetConfig	22
config	DeviceConfig	23
set_config	SetConfig	24
config_result	ConfigResult	25
config_changed	ConfigChanged	26	broadcast to all clients
subscribe	Subscribe	30
session_control	SessionControl	31
time_sync_ping	TimeSyncPing	40
time_sync_pong	TimeSyncPong	41
lease	Lease	50
error	Error	90

ClientInfo (message)

Direction: the PHONE is the server. It advertises _imuninn._tcp over Bonjour and listens for control (WebSocket), data-TCP connections; clients (PCs) connect. A client opens the control WebSocket and sends Hello; the phone answers Welcome.

field	type	#	notes
name	string	1	e.g. hostname or script name
version	string	2	client library version

Hello (message)

field	type	#	notes
client	ClientInfo	1
udp_port	uint32	2	where this client receives DataMessage datagrams (0 = none)
protocol_version	uint32	3
data_transport	Transport	4	v5: how this client wants its data streams. TCP (default) = the length-prefixed stream on Welcome.tcp_port plus UDP for pose / IMU / health when udp_port > 0. WEBSOCKET = the client opens a second WebSocket to the control port and sends “IMDATA01” + uint32 LE client_id as its first binary frame; the phone then sends every subscribed stream (video, depth, pose, IMU, health) on it as binary DataMessage frames. Browsers use this; nothing else changes.

Welcome (message)

field	type	#	notes
device	DeviceInfo	1
capabilities	Capabilities	2
protocol_version	uint32	3
tcp_port	uint32	4	connect here for the length-prefixed DataMessage stream
client_id	uint32	5	first 4 bytes (LE) the client writes on that TCP connection

GetCapabilities (message)

field	type	#	notes

GetCapabilities.GetConfig (message)

field	type	#	notes

GetCapabilities.GetCapabilities.GetConfig.SetConfig (message)

Partial update: only fields present in patch change.

field	type	#	notes
base_config_version	uint32	1	what the client saw; mismatch -> rejected
patch	DeviceConfig	2

GetCapabilities.GetCapabilities.GetConfig.ConfigResult (message)

field	type	#	notes
config_version	uint32	1

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigResult.FieldResult (message)

field	type	#	notes
key	string	1

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigResult.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigResult.FieldResult.Status (enum)

value	#	notes
STATUS_UNSPECIFIED	0
STATUS_APPLIED	1
STATUS_CLAMPED	2
STATUS_REJECTED	3

| status | Status | 2 | | | effective | Value | 3 | | | reason | string | 4 | |

| fields | repeated FieldResult | 2 | | | session_restarted | bool | 3 | |

GetCapabilities.GetCapabilities.GetConfig.ConfigChanged (message)

field	type	#	notes

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.Subscribe (message)

field	type	#	notes
stream_ids	repeated string	1
unsubscribe	bool	2

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.SessionControl (message)

field	type	#	notes

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.SessionControl.Action (enum)

value	#	notes
ACTION_UNSPECIFIED	0
ACTION_START	1
ACTION_STOP	2
ACTION_START_RECORDING	3	local recording on the phone (FileSink)
ACTION_STOP_RECORDING	4

| action | Action | 1 | |

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing (message)

field	type	#	notes

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.TimeSyncPong (message)

field	type	#	notes
client_send_ns	uint64	1
device_recv_ns	uint64	2
device_send_ns	uint64	3

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Lease (message)

Write lease (§6.8 控制权): one writer at a time, auto-released on disconnect.

field	type	#	notes

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Lease.Action (enum)

value	#	notes
ACTION_UNSPECIFIED	0
ACTION_REQUEST	1
ACTION_RELEASE	2
ACTION_GRANTED	3
ACTION_DENIED	4
ACTION_REVOKED	5	phone UI took over

| action | Action | 1 | | | holder | string | 2 | |

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error (message)

field	type	#	notes

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.DeviceHealth (message)

Periodic health report, stream “health” (§11 thermal throttling visibility).

field	type	#	notes
header	Header	1

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.DeviceHealth.Thermal (enum)

value	#	notes
THERMAL_UNSPECIFIED	0
THERMAL_NOMINAL	1
THERMAL_FAIR	2
THERMAL_SERIOUS	3
THERMAL_CRITICAL	4

| thermal | Thermal | 2 | | | battery_level | float | 3 | 0..1, -1 unknown |

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.DeviceHealth.BatteryState (enum)

value	#	notes
BATTERY_UNSPECIFIED	0
BATTERY_UNPLUGGED	1
BATTERY_CHARGING	2
BATTERY_FULL	3

| battery_state | BatteryState | 4 | | | low_power_mode | bool | 5 | | | pose_hz | double | 6 | phone-side rates (what the sources produce) | | imu_hz | double | 7 | | | video_fps | double | 8 | | | depth_hz | double | 9 | | | video_kbps | double | 10 | | | encoder_dropped | uint64 | 11 | | | uptime_s | double | 12 | since the app started |

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3 (message)

field	type	#	notes

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat (message)

field	type	#	notes

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.ImuSample (message)

field	type	#	notes
header	Header	1
accel	Vec3	2	m/s^2, device frame
gyro	Vec3	3	rad/s, device frame
mag	Vec3	4	uT, optional (all zero if absent)

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.Intrinsics (message)

Camera intrinsics are sent with EVERY frame (§5.4), not only at handshake.

field	type	#	notes
width	uint32	1
height	uint32	2
fx	double	3
fy	double	4
cx	double	5
cy	double	6
distortion	repeated double	7	empty for ARKit (already rectified)

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.TrackingState (enum)

value	#	notes
TRACKING_STATE_UNSPECIFIED	0
TRACKING_STATE_NOT_AVAILABLE	1
TRACKING_STATE_LIMITED	2
TRACKING_STATE_NORMAL	3

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.PoseSample (message)

Pose in the native AR convention: right-handed, Y up, camera looks -Z (OpenGL). PC SDK converts to OpenCV / ROS (§6.6).

field	type	#	notes
header	Header	1
position	Vec3	2
orientation	Quat	3
tracking	TrackingState	4
intrinsics	Intrinsics	5	intrinsics of the frame this pose belongs to

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.PixelFormat (enum)

value	#	notes
PIXEL_FORMAT_UNSPECIFIED	0
PIXEL_FORMAT_H264_NAL	1	Annex-B NAL units
PIXEL_FORMAT_HEVC_NAL	2
PIXEL_FORMAT_NV12	3	raw, for future zero-copy path

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.VideoFrame (message)

Image orientation: ARKit captured images are sensor-native landscape and do not rotate with the device (§5.4). rotation_deg says how to rotate to make the image upright for display; data is always sent as captured. rotation_deg: rotate the image clockwise by this much to view it upright when the phone is held in portrait. On iPhone, every camera (back lenses and the front camera) delivers landscape sensor frames that need 90; the field exists so other devices can say otherwise. Intrinsics are in the sensor frame.

field	type	#	notes
header	Header	1
format	PixelFormat	2
width	uint32	3
height	uint32	4
keyframe	bool	5
data	bytes	6	v1: payload inline (§6.1)
intrinsics	Intrinsics	7
rotation_deg	uint32	8	0 / 90 / 180 / 270

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.DepthCompression (enum)

value	#	notes
DEPTH_COMPRESSION_NONE	0
DEPTH_COMPRESSION_LZ4	1
DEPTH_COMPRESSION_ZSTD	2

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.DepthFrame (message)

Depth is uint16 millimetres, row-major, little-endian (§5.4), plus an optional confidence plane (uint8, 0 = low, 2 = high on ARKit).

field	type	#	notes
header	Header	1
width	uint32	2
height	uint32	3
compression	DepthCompression	4
depth_mm	bytes	5
confidence	bytes	6	empty if unavailable
intrinsics	Intrinsics	7	depth-camera intrinsics (may differ from rgb)
rotation_deg	uint32	8

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.Gap (message)

Emitted on a stream when a restart interrupts it (§6.8 step 4).

field	type	#	notes
header	Header	1
reason	string	2

GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.GetCapabilities.GetCapabilities.GetConfig.GetCapabilities.GetCapabilities.GetConfig.ConfigChanged.TimeSyncPing.Error.Vec3.Quat.DataMessage (message)

Envelope for data channels so a single socket can carry mixed streams.

field	type	#	notes
oneof body
imu	ImuSample	1
pose	PoseSample	2
video	VideoFrame	3
depth	DepthFrame	4
gap	Gap	5
config_changed	ConfigChanged	6
health	DeviceHealth	7