Capabilities

Every capability lists: priority (which milestone), consumer kinds, and acceptance criteria — the demo-visible behavior that defines “done”. Wire details live in docs/08; APIs in docs/09.

fjarr.camera — camera video (M1, reference implementation)

Peer consumer. Ports the proven camera-streamer v3 model (prior art).

• Multiple tracks per robot (e.g. front/rear/arm), each announced in the track manifest with a stable track_id and display label.

• Per-track enable/disable without renegotiation (valve + keyframe request); disabled tracks cost ~0 bandwidth.

• Active/inactive quality tiers per budgets; adaptive bitrate from day one (the camera-streamer gap).

• Per-second bandwidth-stats on the control DC, per track.

• Any source, by config. Tracks reference sources through the video source contract: a GStreamer description string covers every camera with a plugin, a registered type covers SDK-backed or multi-output devices, and the track id is the config key (stable across replug):

  [capabilities."fjarr.camera".tracks.front]
  label  = "Front"
  source = "v4l2src device=/dev/v4l/by-id/usb-Acme_Cam-video-index0 ! image/jpeg,width=1280,height=720,framerate=30/1 ! jpegdec"
  
  [capabilities."fjarr.camera".tracks.stereo-left]
  label  = "Stereo left"
  source = "zedsrc camera-resolution=2 camera-fps=30 stream-type=0"    # a vendor plugin (Stereolabs ZED), tier 1
  
  [capabilities."fjarr.camera".tracks.arm]
  label  = "Arm"
  source = { type = "acme.stereo", serial = "0123" }                    # registered in-process, tier 2
  output = "left"                                                       # which of the source's outputs (default "src")
  
  [capabilities."fjarr.camera".tracks.rear]
  label    = "Rear"
  source   = { type = "v4l2", device = "usb-Acme_Rear-video-index0", format = "mjpeg", width = 1280, height = 720, fps = 30 }
  required = true                                                       # a missing camera is a startup error (ADR-0020)
  
  [capabilities."fjarr.camera".tracks.gate]
  label  = "Gate camera"
  # passthrough: the camera's own H.264 is sent as it is — no decode, no encoder on the robot.
  # `thumbnail_url` is the camera's substream and becomes the track's thumbnail tier; without one
  # the track has a single tier and cannot adapt to a viewer's link (slice 6b).
  source = { type = "rtsp", url = "rtsp://10.0.0.7/stream1", thumbnail_url = "rtsp://10.0.0.7/stream2", passthrough = true, latency = 200, protocols = "tcp" }
  
  [capabilities."fjarr.camera".tracks.pattern]
  label  = "Test"
  source = { type = "test", pattern = "smpte", width = 1280, height = 720, fps = 30 }

  Unplugging a camera removes its track through the same renegotiation as a monitor hot-plug; replugging restores the same track_id. (A monitor keeps its track_id because it is keyed by the monitor’s EDID identity, not its connector name, which a replug can change: docs/08.)

Passthrough (passthrough = true in the source, slice 6b): for cameras with an on-board encoder the track carries the source’s elementary stream untouched — the agent parses and packetizes, never decodes or encodes, so the track costs no encoder and keeps the camera’s quality. It is a source setting, not a track setting, because it changes how the source’s pipeline is built (depayload and parse instead of decode); the rtsp type takes it, a gst description declares it (source = { type = "gst", description = "… ! h264parse", passthrough = true }), and a registered type declares video/x-h264 output caps. The thumbnail tier exists only if the source provides a lower stream (rtsp’s thumbnail_url, a registered type’s thumbnail output); otherwise the track has one tier, cannot adapt to a viewer’s link, and says so (adaptive: false in bandwidth-stats, docs/16). The camera decides its keyframes: a joining viewer starts from the retained one, a keyframe request is a counted no-op. --probe-source reports the codec, profile and level a browser must accept.

Control messages: select-tracks and bandwidth-stats are the generic track-control messages the core serves for every track-owning capability; fjarr.camera adds nothing of its own. The client folds all consumers’ demand into one select-tracks per capability (docs/21); nothing is enabled until something on screen asks.

Accepted when: 3 browsers watch 2 tracks of the demo-robot concurrently; toggling a track takes effect < 500 ms without renegotiation; kill/restore of the network recovers the stream without page reload; budgets hold on the NUC.

fjarr.desktop — remote desktop (M3; backend spikes M2)

Peer consumer. The capability that started the project (the original design discussion is archived in the gitignored inspiration/ folder; the full detail is incorporated here).

• One video track per monitor (multi-monitor = multiple tracks, dashboard picks; never one huge stitched frame). Track identity derives from the monitor’s stable connector id (docs/08), never from an index.
• Hot-plug is a first-class requirement, on both sides. Monitors may be connected, disconnected, re-plugged, or change mode at any time during a session: the agent’s backend reports changes (on_monitors_changed), the capability updates its track set and the core renegotiates (docs/08) while every other monitor’s video keeps flowing; a monitors event precedes the renegotiation so the UI reacts instantly. Same-monitor mode/DPI changes update the track in place (new geometry, keyframe). Zero monitors (headless robot, everything unplugged) is a valid state the session survives; a re-plugged monitor returns under its old identity. Web side: docs/22.
• Separate windows: “Open in separate window” puts one robot monitor in a browser window of its own, which the operator places and makes fullscreen with Keyboard Lock; one click per window, the same in every browser; one session and one desktop control claim behind all windows (docs/22).
• Pointer: absolute normalized coordinates per monitor, lossy channel; buttons/wheel reliable. Keyboard: physical KeyboardEvent.code → Linux keycodes, reliable channel; layout handling per backend (docs/07).
• Clipboard, both ways (text and PNG images; files with fjarr.files, M4): the robot offers what it copied and a client reads it on demand; a client writes what the operator pastes, before the paste keystroke reaches the robot (docs/08, docs/22). Writing is input in the desktop control domain; a view_only session sees no clipboard at all (docs/10). Accepted when text copied on the robot pastes in the browser and text pasted in the browser pastes on the robot, non-ASCII included, in the CI fixture and on the mini-PC.
• Unattended access: works after reboot with nobody logged in — the defining industrial requirement; backend chosen accordingly (ADR-0006).
• Privilege separation for injection (ADR-0009).
• Browser-side design — input pipeline, focus, browser-reserved shortcuts, cursor strategy, latency knobs, clipboard UX — is docs/22; the same select-tracks message as the camera capability carries preference: "sharpness" for text.
• Desktop audio (M3.5 slice 3.5.2): what the robot’s speakers play, the default output’s monitor, as the kind: "audio" track desktop-audio. On GNOME the session helper hands the agent a PipeWire connection narrowed to that output, as it hands the screen (docs/23), and on an X11 kiosk its session program does the same (ADR-0034). It follows the desktop grant, and stops when the robot stops sharing. Same demand model and autoplay handling as fjarr.audio.

Accepted when: operator controls the robot-sim desktop end-to-end (input-to-photon within budgets); reboot of the sim brings the desktop back with no local interaction; a stuck-modifier can never persist after disconnect (input state reset on session end); hot-plug: with two monitors streaming, adding a third shows it in the UI within 2 s with zero dropped frames on the other two, unplugging one leaves a placeholder and the rest untouched, re-plugging restores it under the same track_id, a mode change keeps input coordinates correct, and unplugging everything then plugging one back recovers without a reconnect; separate windows: one click opens a robot monitor in a window of its own, which streams, takes input, and goes fullscreen with Keyboard Lock so Alt+Tab typed there reaches the robot; closing the dashboard tab closes it; the same on Firefox.

fjarr.telemetry — sensor/telemetry streaming (M4)

Peer + backend consumers.

• Envelope messages on a dedicated DC; change-triggered with a rate floor (heartbeat every T even when quiet, min-gap when chatty — fleet-daemon pattern).
• Typed values with units; schema declared by the integrator via TelemetrySource adapters (ROS 2 adapter first).
• Store-and-forward for the backend consumer: whitelist of durable types, bounded queue with oldest-first eviction and a drop counter (fleet-daemon lessons, including the ones they got wrong).

Accepted when: demo-robot streams battery/temperature/pose to the dashboard live; a 10-minute signaling outage replays durable telemetry on reconnect, newest-first lane before backfill, without unbounded growth.

fjarr.files — file transfer (M4, greenfield)

Peer + backend consumers. No prior art in the inspiration projects (fleet-daemon had literally zero bytes fields) — designed fresh in docs/08.

• Manifest → blob frames (chunks ≤ 256 KiB and ≤ sctp.maxMessageSize, one blob per transfer) on a dedicated reliable DC; SHA-256 whole-file integrity.

• Backpressure both directions (bufferedAmount low-water pumping — never queue-unbounded).

• Resume by received-ranges after reconnect; a 4 GB diagnostic dump over LTE must survive connection churn.

• Bulk traffic isolated from interactive channels (own DC; separate PeerConnection if measurements demand — budgets).

• Explicit direction grants: files:read, files:write, per-path allow-lists in agent config.

• Designed to back a drive (ADR-0029): fjarr-connect will serve the allowed paths as a WebDAV folder for Nautilus, Finder and Explorer, so the protocol also carries:

  • listings with metadata (name, kind, size, mtime, a change tag), one request per directory;
  • stat of one path;
  • byte-range reads at any offset, without transferring the prefix;
  • writes as whole-file uploads with the existing resume;
  • mkdir, delete and rename under files:write.

  Each allow-listed root is a top-level folder, and nothing outside the allow-lists is reachable, even by name.

Accepted when: 1 GB transfers both ways with a mid-transfer network kill resumed to a verified hash; interactive video latency unaffected during bulk; a byte-range read from the middle of a 1 GB file that transfers no prefix; a path outside the allow-lists refused for list, stat, read and write alike.

fjarr.terminal — remote terminal (M2)

Peer consumer. Deliberately the second capability implemented: media-free, so it proves the extension API generalizes beyond video.

• PTY on the robot over the bulk byte channel in both directions, raw framing, one per session (docs/08); xterm.js component in @fjarr/react. Resize, UTF-8, scrollback client-side. The second client is native: fjarr-connect shell attaches the operator’s own terminal to the same pty (docs/27).

• Off unless configured, and it names its user. No pty exists until fjarr.toml says which account the shell runs as — the capability reports unavailable otherwise, which is a deployment choice rather than a fault. There is deliberately no default: the agent’s own unprivileged user is chosen for running the agent, not for being a useful or a safe shell, and silently picking it would be a decision made by omission (docs/10).

• user is verified, not switched to. The agent runs unprivileged (ADR-0009) and cannot become another account, so naming one it is not running as reports unavailable with that reason rather than quietly starting a shell as the wrong user. To give operators a different account, run the agent as it. Switching users needs a privileged path, which is a separate decision: it is M4’s, with fjarr.files (open question #42).

• A shell that refuses logins is no shell. The agent’s own account is a system account whose login shell is nologin; started as the terminal, it prints “This account is currently not available.” and exits (mini-PC, 2026-10-04). So a resolved shell named nologin or false reports unavailable naming it and shell as the fix, and fjarr-agent setup writes shell = "/bin/bash" (or /bin/sh) beside user when the account it picks has such a login shell.

  [capabilities."fjarr.terminal"]
  enabled = true
  user    = "service"   # required; the shell runs as this account
  shell   = ""          # default: that user's login shell

• The pty dies with the session, always — release_all_input closes it, so a dropped connection leaves no orphan shell. Input-bearing, but in no control domain: each session has its own pty. Every open and close is audited with the operator identity; an I/O recording hook exists from day one but stores nothing by itself (open question #4).

Accepted when: interactive shell round-trip < 150 ms LAN; disconnect kills the PTY (no orphan shells); every session start/end is audit-logged.

fjarr.net — network tunnel (M4.5)

Peer consumer, and the only capability whose peer is not a browser.

A session-scoped point-to-point IP link between one operator machine and one robot, so the tools a developer already owns work unchanged: ssh, scp, a UDP bridge onto a CAN bus, ros2 topic list against the robot’s graph. It exists so that the list of things nobody will write a protocol for stops being a roadmap. Full design in docs/27; the shape decision in ADR-0023.

• A persistent TUN interface at each end, created by the installer before the robot’s software starts — DDS binds its interfaces at participant creation, so a tunnel that appears at connect time is invisible to a ROS stack that is already running.
• Packets on fjarr:stream:fjarr.net with raw framing, unordered and never retransmitted (docs/08); MTU 1184, one SCTP chunk (ADR-0027); bounded queue with tail-drop.
• The operator’s address is fixed, the robot’s is derived from its robot id. No allocator and no state in the signaling server.
• Isolation is structural: every link terminates at the operator and nothing is forwarded between links, so two robots attached at once cannot reach or see each other.
• Off by default; an explicit net claim in the session grant; audited at open and close. The grant is as consequential as fjarr.terminal (docs/10).
• The operator end is fjarr-connect, a native binary (ADR-0024).

Accepted when: fjarr-connect robot-42 brings a link up in under 3 s and ssh logs in; a 1 GB scp completes to a verified hash; ros2 topic list shows the robot’s topics with Fast DDS unconfigured and with Cyclone DDS and the documented file (the latter amended out of the M4.5 gate as #29, then met the same day); a second robot attached at the same time is unreachable from the first, in both directions; the agent restarts without the robot’s ROS stack restarting; every open and close is audit-logged.

fjarr.observability — fleet observability (M7, flagship paid tier)

Backend consumer.

• Release/version reporting: agent, OS, customer software versions (via adapter), reported on change + on connect.
• Generic host metrics out of the box: CPU temp, load, mean per-process CPU, disk usage/health, memory, network counters.
• User-defined metrics and error events through the same adapter seam: counters, gauges, and error reports with severity + fingerprint (for fleet-wide error-rate views).
• Transport: telemetry envelope, backend stream, store-and-forward; server aggregates (Cloud: retention, dashboards, alerting).

Accepted when: a 10-robot simulated fleet renders version spread, error rate, and host-metric trends in the fleet view; one robot going dark raises its liveness state within 30 s.

fjarr.ota — fleet OTA updates (M8)

Backend consumer; the design is docs/32. The fleet daemon’s deploy.py is the documented anti-pattern this replaces (no A/B, no rollback, abort-unsafe).

• A release is one signed version pinning every component (OS image, apps, configuration, peripherals), each with a rollback class and hardware variants (ADR-0036). A release may carry a new image or only lighter components on top of it.
• fjarr-update orchestrates fetch, readiness, prepare, install, health, peripherals and commit; the OS slot is written by a narrow applier per recipe (ADR-0035). A/B atomic install with rollback where the recipe has slots.
• Readiness: conditions evaluated on the robot (time window, charger, idle, holds from the robot’s own stack over a local API) with a reported reason, and a customer-registered prepare action confirmed by a predicate; teleoperation is refused from prepare until commit.
• Peripherals through handlers with a fixed contract (identify, diff, prepare, install, verify by read-back, commit, rollback); reference handlers for u-blox, CAN flashing and router configuration with commit-confirm.
• Run logs like a CI job: a heading per step with its status and duration, the step’s output streamed line by line, kept per robot.
• Content over HTTPS as resumable, content-addressed chunks under a per-robot budget per time interval (ADR-0037).
• Campaigns (Cloud/paid): stages promoted by a person, parallel and batched within a stage, abortable; the fleet’s policy decides whether the robot’s owner is asked (with the open <UpdatePrompt>); security releases follow the EU Cyber Resilience Act’s rules.
• Recipes, CI-built: Yocto and NixOS on a ZED Box Mini (Jetson Orin) and on x86, an Ubuntu A/B image on x86, and plain Ubuntu with apt (not atomic).

Accepted when: on every recipe (Yocto and NixOS on the ZED Box Mini; Yocto, NixOS and Ubuntu A/B on the second x86 mini-PC), a power cut mid-write leaves the robot booting the old slot, and a release whose health check fails rolls back by itself with the failed step in its run log. Once, across the fleet: a staged campaign of one, then several robots, is aborted midway; a release downloads across several days within a daily budget, through link drops; and a robot’s owner schedules an update for tonight in the demo dashboard, which then waits for the charger and says so.

fjarr.audio — two-way audio (planned) {#fjarraudio—two-way-audio-planned}

Peer consumer. Opus over native WebRTC audio tracks — no new dependencies on either side (GStreamer opusenc/opusdec, browser getUserMedia).

• Downlink: one or more kind: "audio" tracks in the manifest (robot microphones — hear motors, alarms, people). Demand-driven like video: default off, per-consumer mute, browser autoplay policy surfaced (docs/21). Only when the grant allows it (params.listen = true): a session without it is offered no microphone track at all. A remote operator listening to the people near a robot is the privacy question of open question #3, closed so 2026-10-11.
• Uplink (talk to a person at the robot): granted per session with params.talk = true; the agent pre-allocates a recvonly transceiver in its offer so the browser attaches its mic via replaceTrack() without renegotiation. Played through a configured output device on the robot with a hard volume cap in agent config.
• Push-to-talk by default, open mic as the operator’s choice: one grant, talk, allows the uplink either way (open question #17 closed 2026-10-11; a separate grant for an open mic added complication for no value). The agent plays the operator’s audio only between the client’s talk-start and talk-stop (docs/08), which are audited audio-uplink session events (docs/10 — a live microphone is as sensitive as a terminal). One operator talks at a time; a second is told who holds the speaker.
• Desktop audio (what the robot’s desktop plays) is not a microphone: it travels on fjarr.desktop and follows that grant, so whoever sees the screen hears it.
• Not in scope: alert sounds/beeps in the dashboard (host UI concern), desktop audio (it is fjarr.desktop’s, same track model).

Accepted when: an operator hears the robot-sim’s synthetic audio source within budget (docs/16); PTT delivers speech to the sim’s sink with echo cancellation on; releasing PTT stops the uplink within 200 ms; audit events recorded; a session without listen is offered no microphone and one without talk gets no uplink; the robot desktop’s sound is heard with its screen. Milestone: M3.5 (roadmap).

fjarr.logs — live log tailing (planned)

Peer + backend consumers. journalctl/application log streams over a reliable-ordered channel with server-side filtering (unit, level, regex), backpressured so a chatty robot can’t flood the operator; the same source feeds observability (M7) in the backend mode. Cheap, and the single most requested support feature after “show me the screen”. Milestone: peer mode M4, backend mode M7 (roadmap).

fjarr.test — the built-in test capability (slice 3) {#fjarrtest—the-built-in-test-capability-slice-3}

Peer consumer, built into libfjarr as the public class fjarr::TestCapability and enabled by config. It exists for two reasons: it proves the core end to end before any real capability exists (docs/23), and it is the customer’s install smoke test — after wget … | sudo sh, a test pattern in the dashboard proves the robot, the server and the grant flow work before a single camera is configured (docs/26). It ships in the fjarr-agent package for that reason; its hooks are inert unless enabled.

Tracks (all codec: "H264", active tier 1280×720@30, thumbnail 640×360@5, frame stamp drawn per docs/25):

track_id	label	source	present
test-pattern	“Test pattern”	videotestsrc pattern=smpte is-live=true	always
test-second	“Second pattern”	videotestsrc pattern=ball is-live=true	only while the hot-plug hook has it “plugged” (default: unplugged)

Messages (envelopes, cap: "fjarr.test"):

type	kind	channel	payload	semantics
echo	request → result	control	request {any} → result {ok:true, echo:<the request payload>, t_agent: ms}	correlation round trip
hotplug	request → result	control	{"plugged": bool}	adds/removes test-second through update_tracks → docs/08 renegotiation; result{ok:true, manifest_version}; requires test_hooks = true
silence	request → result	control	{"pings": bool, "media": bool, "ms": number}	for ms: stop answering ping and/or close all valves — the operator-visible “agent went silent” faults; requires test_hooks = true
drive	event	realtime	{"v": number, "seq": number}	the deadman-armed consumer: the capability arms SessionContext::arm_deadman(500 ms) on the first drive; each drive feeds it
deadman	event (agent → operator)	control	{"state": "armed" | "expired" | "fed", "ms_since_feed"}	emitted on every transition, so a test can assert that drive silence of 500 ms expires the deadman and that session end expires it too

fjarr.test is input-bearing and its drive is the demo’s motion control domain (one driver at a time; echo, hotplug and silence claim nothing). It implements release_all_input by expiring the deadman and emitting deadman{state:"expired"} — which is how the docs/15 “release on session end” regression test observes the behaviour without a real actuator.

Config: [capabilities."fjarr.test"] enabled = true, test_hooks = false (the demo and CI set it true; a production install leaves the hooks off, and the grant must list fjarr.test for an operator to reach it at all).

Accepted when: the demo dashboard shows the pattern within the docs/16 startup budget; select-tracks toggles it; hotplug adds the second track with zero dropped frames on the first (frame-stamp counter); silence{pings} makes the operator’s ladder climb and recover; drive silence expires the deadman in ≤ 600 ms and so does session-close.

fjarr.introspect — pipeline introspection (slice 3 core, slice 5 UI) {#fjarrintrospect—pipeline-introspection-slice-3-core-slice-5-ui}

Peer consumer, built in. Exposes the agent’s live pipeline snapshots (DOT, JSON, summary text, with history) over the session so a robot’s media plane can be inspected from the dashboard without shell access — the product feature specified in docs/24. Grant-gated ({"name":"fjarr.introspect"}, developer/support roles); json and dot bodies ride fjarr:bulk:fjarr.introspect as blob references.

Accepted when: from the demo dashboard’s Diagnostics tab, an operator with the introspect grant watches the session pipeline update live while toggling tracks and triggering the hot-plug test hook; the same snapshots are readable with one curl on the robot; an operator without the grant gets capability-denied.

Sensor data transport: video track or stream class? {#sensor-transport}

Dense sensor data (depth, thermal, disparity, lidar point clouds) can travel either packed into a video track or as stream-class binary frames (ADR-0018). Neither is universally better; the choice is made per sensor, per capability with this matrix, confirmed by measurement in the first sensor capability’s spike (open question #16).

Concern	Video track	Stream class
Bandwidth	Temporal + spatial compression: 1080p30 dense depth in a few Mbps	Raw size unless we compress ourselves (640×480×16-bit ≈ 147 Mbps at 30 fps — must decimate/voxelize or add Draco/zstd)
Fidelity	Lossy: 8-bit 4:2:0 codecs; 16-bit metric data needs a packing profile (split planes, or 10/12-bit HEVC/AV1 where the browser decodes them in hardware); edge ringing → “flying pixels”	Bit-exact: 16-bit depth, float32 XYZ, intensities, per-point attributes
Data shape	Dense 2-D grids only	Anything: sparse/variable point counts, structs, protobuf/CBOR
Metadata & sync	Out-of-band (frame seq via time-sync + telemetry) — a real sync problem	Inline: timestamps, intrinsics, frame id in the header
Rate control	Free: GCC/TWCC adaptive bitrate, FEC/NACK, jitter buffer, demand tiers	Ours: drop frames on bufferedAmount growth; no FEC; whole frame or nothing
CPU / HW	HW encode (VA-API) and HW decode in the browser	Agent: cheap unless compressing; browser: WASM decode if compressed
Consumer access	<video> → WebGL texture is cheap; numeric readback is expensive (canvas getImageData / WebCodecs VideoFrame copy)	ArrayBuffer straight into a three.js/WebGL buffer; trivial numeric access
Latency	Encoder pipeline adds ~1–2 frames	Serialization only
Fits when	dense, high-rate (≥ 15 fps), visualization-grade, resolution matters	sparse or exact, ≤ ~10 Hz after decimation, ≤ docs/16 stream budget, metadata-rich

Rules of thumb: a depth camera meant for seeing obstacles → video (with a documented packing profile); a lidar cloud meant for a 3-D map or for numbers → stream class after decimation; and hybrid is legitimate — a depth video for the overview plus a decimated cloud on the stream class for the 3-D view, both stamped with time-sync. The measurements that decide each case: end-to-end latency, bandwidth, agent and browser CPU, and fidelity error (RMS depth error after the round trip).

Conventions on existing capabilities (planned additions)

• fjarr.camera/snapshot — request → result carrying one full-resolution still (JPEG/PNG) via the bulk channel, for tickets and reports without screen-grabbing a compressed video frame.
• fjarr.telemetry/alert — a conventional event shape {severity, code, message, data?} so hosts route robot-originated notifications to their own toast/notification system uniformly (the fleet dashboard’s /rtc/toastMessage, generalized).
• fjarr.files/list — directory listing within the configured allow-lists, a prerequisite for any file-browser UI.
• fjarr.core — reserved namespace for session-level messages that belong to no capability: ping/pong (docs/08 heartbeat) and time-sync (operator↔agent clock offset + RTT, needed for stamping teleop commands and for the docs/15 latency harness).
• Multi-operator presence (planned, M5): the server emits session-peers (docs/08) to every party on a robot — who is connected, with their ownership role from the docs/10 leases (owner of input, viewer) — so UIs can show “Anna is driving, Björn is watching” and request/hand over control explicitly. Web: useSessionPeers(session).
• Haptic feedback (planned): any capability may emit haptic {intensity, duration_ms, pattern?} events on the realtime class (collision proximity, end-stop, terrain); the web library maps them to the Gamepad API’s vibrationActuator via useGamepadHaptics(session, cap) and ignores them where unsupported. Lossy by design — a missed rumble is fine, a late one is worse.

Later / explicitly deferred

Session recording/replay, mobile operator apps — open questions.

Stress test: com.example.arm-teach (never to be built) {#stress-test}

Fictional third-party capability used to pressure-test the extension model: a robot-arm teach pendant needing (a) bidirectional low-latency joint-state streaming (lossy DC, 100 Hz), (b) a per-robot persistent waypoint file read/written through the file capability’s API as a dependency, (c) a custom dashboard panel with its own state, (d) a privileged actuator interlock requiring an explicit integrator grant. Every extension-API change must keep arm-teach buildable without core patches.