Slice 6a Review
Retrospective review of slice 6a per docs/13 and docs/20, run before the slice was committed. The slice closes ADR-0007 on webrtcbin with an estimator of our own and makes adaptive bitrate real: NACK/RTX and keyframe feedback in the offer, a per-peer estimate from transport-wide feedback, tier encoders that follow their viewers inside a band, and viewers moved between tiers on their own link. The web client shows the agent’s view per track. One reviewer pass on the C++, the simulator and the TypeScript, plus what the traces and the lab found — which, this time, was most of the design’s calibration.
What the harness found
Section titled “What the harness found”| Found by | Defect | Fix |
|---|---|---|
the first /stats trace under bad | the estimator cut 7.5 % per window on 15 % random loss and reached the floor in 3 s: sending less never reduces random loss | a loss cut needs throughput to be collapsing (the peer acknowledges under 70 % of what is sent) window after window, at most once per second; random loss with throughput intact is left to retransmission |
| the same trace | one lost packet in a seven-packet feedback window read as 14 % loss | loss is judged over a rolling second of windows, ≥ 10 packets |
| the same trace, recovery | the estimate could not climb out of the thumbnail tier: growth was capped at 1.5× what arrived, and a demoted viewer receives 300 kbps | growth is uncapped while nothing is lost (30 %/s below the level before the last cut); the delay rule catches an overshoot within three windows |
the same trace, lossy | ±15 ms jitter tripped the delay rule (1 ms, two windows) at the thumbnail’s tiny windows | 3 ms for three windows in a row; a shaper’s queue is steady, jitter flips sign |
lab, bad on a lone viewer | the session died of an SCTP transport error: clamped to the shared-encoder band (2 Mbps) above a 1.5 Mbit shaper for the 2 s a tier switch takes, the kernel queue grew past what the data channel tolerates | a tier with one subscriber is clamped to the floor, not the band (docs/23) |
| lab, measuring the reaction | the bandwidth-stats events arrived 3 s late under bad: they ride the impaired link behind the video | the gate reads /stats through the introspection port, which the lab’s netem now exempts like docker/lab/netem.sh |
| lab, three viewers | netem on lo for the simulator viewer shaped both directions through one queue: its pings died behind the video and the session closed on the heartbeat | the simulator runs in dev and the robot’s egress toward that address alone is impaired (netemToward) |
| lab, three viewers | the simulator’s session read as a lossless link at the ceiling: its webrtcbin answerer sent no retransmission requests, and an unacknowledged feedback window was taken as “no loss” | do-nack on the simulator’s transceivers; a window with no acknowledged packets is no evidence and holds the estimate |
simulator netem-lossy (both directions of lo impaired) | a lost feedback packet makes the next window report everything it did not cover as lost (50–70 % in one window), and 5 % reverse-path loss demoted the viewer to the floor | the loss cut needs three consecutive collapsed windows: a shaper collapses every window, a lost feedback packet one; unit-tested with a lost feedback packet every eighth window |
simulator netem-lossy from dev, one-directional | with only the media path jittered, the webrtcbin receiver still reported 20–50 % loss per window at 5 % real loss (Chromium reports ~5 % under the same profile) and the estimate fell to the floor | the profile’s rate control is recorded, not asserted, with a GStreamer receiver; the browser lab asserts it; the netem-* scenarios now run opsim from dev against the robot’s egress with the introspection port exempt on every device |
| lab, three viewers, run after the others | the congested viewer was not demoted within the scenario’s 10 s: a session that starts already behind a bad link waits 3–4 s for transport-wide feedback to carry bitrates at all before the estimator has anything to judge, then 2 s below the band, then the 1 s sample | the scenario allows 25 s and says why; the browser gate measures an established session’s reaction, which is what docs/16 is about |
| lab, three viewers, once | select-tracks got no result: its reply crosses the impaired link behind whatever video is already queued, and the shared encoder is held at 4 Mbps by the clean viewers | a control request on that link gets a 15 s budget and one retry (the agent is unchanged: this is what a bad link does to a request) |
| the same trace | avg-delta-of-delta is INT64_MIN before the first feedback, and the first windows carry packet counts with zero bitrates | both are read as “nothing to learn yet”; the session also treats an unchanged twcc-stats read as no new feedback, since rtpsession keeps the last window until the next arrives |
| simulator | a nested wait_for deadlocked the congested-viewer scenario | inbox readers inside predicates take no lock |
Reviewer findings
Section titled “Reviewer findings”| # | Severity | Finding | Resolution |
|---|---|---|---|
| 1 | high | the plan’s “estimate per (peer, track)” does not match the feedback, which is per transport | one estimate per peer; the peer’s enabled tracks share it in proportion to their tier targets; docs/23 amended |
| 2 | medium | the promotion hold of 10 s plus the climb from the floor missed the 15 s gate | 5 s hold and 30 %/s recovery growth; measured 11.6 s to promotion, 12.6 s to 90 % of the target |
| 3 | medium | the plan put the estimator’s numbers into the session snapshot, which walks GStreamer elements only | they are in /stats (rate, twcc, rtx, per-tier kbps) and in bandwidth-stats; docs/23 amended |
| 4 | low | the simulator’s netem-* comments still described an offer without NACK/RTX | updated; the scenarios gained a rate-control check |
Measured (software encoder, the lab’s Chromium, this host)
Section titled “Measured (software encoder, the lab’s Chromium, this host)”| Case | Result |
|---|---|
bad on a lone viewer | estimate below 2 Mbps within ~1 s of the impairment, encoder output below 1.7 Mbps within ~3–4 s, demoted at ~2–3 s; 3–4 frames decoded per 6 s at the 5 fps thumbnail tier (a repair round trip per frame at 15 % loss); promoted 11.6 s and back above 3.6 Mbps 12.6 s after the link cleared |
lossy (5 %) on a 30 fps track, 20 s | 1416 retransmission requests served, 0 keyframe requests, 2 freezes, 624 frames decoded, bitrate held |
three viewers, one behind bad | the congested viewer demoted 8–9 s after enabling (TWCC warm-up on a session that starts behind the link, then 2 s below the band, then the 1 s sample), kept receiving, promoted after the link cleared; the two clean viewers on the active tier at ≥ 3.5 Mbps throughout (clean 3.7 Mbps); three consecutive runs |
Gate (docs/23 slice 6a)
Section titled “Gate (docs/23 slice 6a)”1–5. tests/stack/ratecontrol.spec.ts (four tests) and the simulator’s netem-lossy, netem-4g, netem-bad with their rate-control check, plus congested-viewer — green on this host; the estimator, the tier policy and the client’s agent-stats store each have unit tests.
6. Every earlier gate green; see the commit.
Deferred
Section titled “Deferred”- The delay rule is a threshold on the average delta-of-delta, not a trendline filter with an adaptive threshold as in GCC; it held under every docs/25 profile, and a real Wi-Fi link is the next calibration.
- A control run with retransmission disabled (gate 3 as planned) was not built: the numbers under
lossyspeak for themselves, and a config switch to turn repair off has no product use. - The
badprofile at the thumbnail tier decodes slowly; forward error correction would help there at a bandwidth cost everywhere (docs/08 leaves it off). - Two test marginalities on this host were measured to be pre-existing, not regressions: the simulator’s
netem-bad“frames keep arriving” bound (7–10 whole access units per 5 s window, unchanged with the encoder pinned at the pre-slice-6a fixed 300 kbps — 15 % loss costs most multi-packet frames, since only about 44 % of five-packet frames survive it; the bound is now 5 and the 3 s gap check is what guards a stall), and the slice-4 camera three-viewer test, which fails about one run in three on a loaded machine with the slice-6a changes stashed as well. - A viewer whose link is far below the shared encoder’s rate has its control channel queued behind the video for the two seconds the demotion hysteresis takes; the per-consumer leaky queue bounds what the agent holds, not what the network queues. A separate transport for control, or pacing a new subscriber’s first seconds, is an M3 question.
- Passthrough tracks (
adaptive: false) are slice 6b.