Voice 14: apps/fluxer voice runtime wiring (#75) #131

Manually merged
charles merged 0 commits from voice-14-app-voice-runtime into voice 2026-11-11 23:43:27 +00:00
Owner

Stacked on: voice-10-core-soundboard

What this builds

  • apps/fluxer/src/voice_rt.rs: VoiceBackend (room + sound system; new(room, host), paced_null(room, NullHost, NullControl)) and VoiceRuntime, which spawns the VoiceSession, forwards VoiceEvent -> CoreEvent::Voice, pumps remote_audio() -> AudioEngine::push_remote, and executes Effect::Voice. UnavailableRoom is used for real instances until the LiveKit room (Voice 13) lands: every join fails with a reason the voice panel shows.
  • apps/fluxer/src/audio_rt.rs: executes every AudioCommand on the AudioEngine (CaptureConfig -> ChainConfig, DeviceId "default" -> None, PlaybackSource -> SourceId, chimes at the output volume). StartCapture starts a thread that feeds the engine's processed frames to VoiceHandle::push_microphone. EngineEvent -> AudioEvent, with LevelThrottle holding local/remote levels to 20 Hz each (the newest one waits). It also holds the downloaded sound bytes for PlayClip, and a pacer that advances a NullHost clock in real time.
  • runtime.rs: Runtime::with_voice_backend. FetchMedia skips the download when AudioEngine::has_clip. Leaving the room stops the clips and drops the bytes nobody took. The status API (fluxer-status-api) starts disabled with the runtime (port 21863, or a free port with --devserver), follows the "Companion apps" switch and token, and gets selectors::voice_status(&app) after every apply. Handle::status_api_addr(), plus Handle::voice_script() with --devserver. In devserver mode the default backend is FakeRoom + paced NullHost. The room follows the scenario's voice script: speakers are added with a microphone track and set as active speakers, and leavers are removed.
  • config.rs: --audio-null. record.rs: e2ee_key is redacted (Review Focus 7).

How it was tested

apps/fluxer/tests/voice_runtime.rs runs against the voice devserver scenario with a FakeRoom and a paced NullHost:

  • join_reaches_connected_and_mic_frames_reach_room
  • remote_tone_reaches_mixer_with_user_gain: peak at 50 % = 8192 x 10^-0.6 +/- 15 %
  • deafen_stops_remote_audio: unsubscribed, mic muted, silence
  • soundboard_event_plays_clip
  • status_api_reports_connected_channel: a raw GET /status
  • record_mode_never_contains_token_or_e2ee_key
  • leaving_stops_capture

There are also unit tests for the level throttle, the chain mapping, the default device, UnavailableRoom and e2ee_key redaction. FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci is green locally, and clippy passes with --no-default-features.

Deviations

  • No Apm stage: fluxer-audio has no echo canceller or AGC (spec §12 Q3). echo_cancellation and auto_gain are ignored.
  • "FetchMedia with the Foundation disk cache": the Foundation cache only holds READY and messages, so there is no disk cache for media. Bytes are kept in memory until PlayClip, and the engine's 32-clip decoded cache covers repeats.
  • No LiveKit room yet (Voice 13). Real instances get UnavailableRoom.
  • The devserver fake room does not play tones for speakers. It only sets active speakers.
  • lib.rs was touched to declare and export the new modules. README.md was updated.

Closes #75

🤖 Generated with Claude Code

Stacked on: voice-10-core-soundboard ## What this builds - `apps/fluxer/src/voice_rt.rs`: `VoiceBackend` (room + sound system; `new(room, host)`, `paced_null(room, NullHost, NullControl)`) and `VoiceRuntime`, which spawns the `VoiceSession`, forwards `VoiceEvent` -> `CoreEvent::Voice`, pumps `remote_audio()` -> `AudioEngine::push_remote`, and executes `Effect::Voice`. `UnavailableRoom` is used for real instances until the LiveKit room (Voice 13) lands: every join fails with a reason the voice panel shows. - `apps/fluxer/src/audio_rt.rs`: executes every `AudioCommand` on the `AudioEngine` (`CaptureConfig` -> `ChainConfig`, `DeviceId` "default" -> `None`, `PlaybackSource` -> `SourceId`, chimes at the output volume). `StartCapture` starts a thread that feeds the engine's processed frames to `VoiceHandle::push_microphone`. `EngineEvent` -> `AudioEvent`, with `LevelThrottle` holding local/remote levels to 20 Hz each (the newest one waits). It also holds the downloaded sound bytes for `PlayClip`, and a pacer that advances a `NullHost` clock in real time. - `runtime.rs`: `Runtime::with_voice_backend`. `FetchMedia` skips the download when `AudioEngine::has_clip`. Leaving the room stops the clips and drops the bytes nobody took. The status API (`fluxer-status-api`) starts disabled with the runtime (port 21863, or a free port with `--devserver`), follows the "Companion apps" switch and token, and gets `selectors::voice_status(&app)` after every `apply`. `Handle::status_api_addr()`, plus `Handle::voice_script()` with `--devserver`. In devserver mode the default backend is `FakeRoom` + paced `NullHost`. The room follows the scenario's voice script: speakers are added with a microphone track and set as active speakers, and leavers are removed. - `config.rs`: `--audio-null`. `record.rs`: `e2ee_key` is redacted (Review Focus 7). ## How it was tested `apps/fluxer/tests/voice_runtime.rs` runs against the `voice` devserver scenario with a `FakeRoom` and a paced `NullHost`: - `join_reaches_connected_and_mic_frames_reach_room` - `remote_tone_reaches_mixer_with_user_gain`: peak at 50 % = 8192 x 10^-0.6 +/- 15 % - `deafen_stops_remote_audio`: unsubscribed, mic muted, silence - `soundboard_event_plays_clip` - `status_api_reports_connected_channel`: a raw GET `/status` - `record_mode_never_contains_token_or_e2ee_key` - `leaving_stops_capture` There are also unit tests for the level throttle, the chain mapping, the default device, `UnavailableRoom` and `e2ee_key` redaction. `FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci` is green locally, and clippy passes with `--no-default-features`. ## Deviations - No `Apm` stage: `fluxer-audio` has no echo canceller or AGC (spec §12 Q3). `echo_cancellation` and `auto_gain` are ignored. - "FetchMedia with the Foundation disk cache": the Foundation cache only holds READY and messages, so there is no disk cache for media. Bytes are kept in memory until `PlayClip`, and the engine's 32-clip decoded cache covers repeats. - No LiveKit room yet (Voice 13). Real instances get `UnavailableRoom`. - The devserver fake room does not play tones for speakers. It only sets active speakers. - `lib.rs` was touched to declare and export the new modules. `README.md` was updated. Closes #75 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(core): soundboard and entrance sounds
All checks were successful
ci / Linux (x86_64) (push) Successful in 8m43s
ci / Windows (x64) (push) Successful in 8m5s
add3ec139a
Refs #71

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #75

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(app): voice and audio runtime
Some checks failed
ci / Windows (x64) (push) Failing after 3m47s
ci / Linux (x86_64) (push) Has been cancelled
e0e3168fb3
Effect::Voice drives the fluxer-voice session, Effect::Audio the
fluxer-audio engine; capture frames go to the room, remote audio to the
mixer, engine events come back throttled to 20 Hz. FetchMedia keeps the
bytes for the PlayClip that follows. The status API publishes the voice
status after every event. --devserver uses FakeRoom and NullHost,
--audio-null forces NullHost. Recordings redact e2ee_key.

Refs #75

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
Owner

Independent review: Voice 14 (app-voice-runtime)

Reviewed only add3ec1..origin/voice-14-app-voice-runtime (e0e3168).

Tests (plan Step 1): all 7 named tests are in apps/fluxer/tests/voice_runtime.rs and assert what the plan asks for. The user-gain check uses 10^-0.6 for 50 %, matching the original curve. Deafen asserts that the track is unsubscribed (Review Focus 1), that the mic is muted, and that nothing is played.
Interfaces: Effect::Voice and Effect::Audio are executed. The pipeline is capture → CaptureChain → push_microphone, then remote_audio → push_remote. Levels are throttled to 20 Hz and coalesced (with unit tests). StatusApi::publish(voice_status) runs after every apply. --devserver uses FakeRoom and a paced NullHost. --audio-null is added.
Invariants: no core/ui crate touched, no secrets in fixtures, no new third-party dependency (bytes is already in the workspace). The session token, e2ee_key and status token are all secrets and are never logged.

Findings

  • minor: in record_mode_never_contains_token_or_e2ee_key, the e2ee_key assert is vacuous because the devserver voice scenario never sends an e2ee_key. Review Focus 7 is still covered by the new unit test voice_server_update_loses_its_token_and_e2ee_key in record.rs. Nothing checks the status-API token against logs; it is only ever passed around as a SecretString.
  • minor: in audio_rt::Media, bytes fetched for a PlayClip that never arrives stay in memory until the room is left. There is no disk cache, which is a documented deviation from the plan. The bound is one entry per clip key.
  • minor: FetchMedia skips the download when engine.has_clip. If the engine evicts that clip before PlayClip, the sound is skipped. ClipFinished is still emitted, so the store does not hang.
  • minor (documented deviations): there is no Apm stage, since fluxer-audio and fluxer-voice have none (spec §12 Q3). Real instances get UnavailableRoom until Voice 13 lands. lib.rs was touched to export the new modules. Make sure a follow-up issue tracks the Apm stage.
  • minor: the timing-based tests sleep up to 5 s, and one test allows a 15 % peak tolerance. They may flake on slow runners.

CI

Local cargo xtask ci in a clean review worktree: green (exit 0). All 7 voice_runtime tests passed.

Verdict: APPROVE (no blocking findings).

## Independent review: Voice 14 (app-voice-runtime) Reviewed only `add3ec1..origin/voice-14-app-voice-runtime` (e0e3168). **Tests (plan Step 1):** all 7 named tests are in `apps/fluxer/tests/voice_runtime.rs` and assert what the plan asks for. The user-gain check uses 10^-0.6 for 50 %, matching the original curve. Deafen asserts that the track is unsubscribed (Review Focus 1), that the mic is muted, and that nothing is played. **Interfaces:** Effect::Voice and Effect::Audio are executed. The pipeline is capture → CaptureChain → push_microphone, then remote_audio → push_remote. Levels are throttled to 20 Hz and coalesced (with unit tests). `StatusApi::publish(voice_status)` runs after every apply. `--devserver` uses FakeRoom and a paced NullHost. `--audio-null` is added. **Invariants:** no core/ui crate touched, no secrets in fixtures, no new third-party dependency (`bytes` is already in the workspace). The session token, `e2ee_key` and status token are all secrets and are never logged. ### Findings - **minor**: in `record_mode_never_contains_token_or_e2ee_key`, the `e2ee_key` assert is vacuous because the devserver voice scenario never sends an `e2ee_key`. Review Focus 7 is still covered by the new unit test `voice_server_update_loses_its_token_and_e2ee_key` in record.rs. Nothing checks the status-API token against logs; it is only ever passed around as a `SecretString`. - **minor**: in `audio_rt::Media`, bytes fetched for a `PlayClip` that never arrives stay in memory until the room is left. There is no disk cache, which is a documented deviation from the plan. The bound is one entry per clip key. - **minor**: `FetchMedia` skips the download when `engine.has_clip`. If the engine evicts that clip before `PlayClip`, the sound is skipped. ClipFinished is still emitted, so the store does not hang. - **minor (documented deviations)**: there is no Apm stage, since fluxer-audio and fluxer-voice have none (spec §12 Q3). Real instances get `UnavailableRoom` until Voice 13 lands. lib.rs was touched to export the new modules. Make sure a follow-up issue tracks the Apm stage. - **minor**: the timing-based tests sleep up to 5 s, and one test allows a 15 % peak tolerance. They may flake on slow runners. ### CI Local `cargo xtask ci` in a clean review worktree: **green** (exit 0). All 7 voice_runtime tests passed. **Verdict: APPROVE** (no blocking findings).
test(app): poll for capture to open instead of asserting immediately
All checks were successful
ci / Windows (x64) (push) Successful in 12m56s
ci / Linux (x86_64) (push) Successful in 23m49s
27cd4bd67a
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
charles manually merged commit 09de2137ea into voice 2026-11-11 23:43:27 +00:00
Sign in to join this conversation.
No description provided.