Voice 4: fluxer-audio devices, capture and playback mixer (#65) #106
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "voice-4-audio-devices-mixer"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Stacked on: voice-3-audio-dsp
What this adds (
crates/fluxer-audio)host:AudioHost(list devices per direction, open input/output streams with callbacks, one listener for device and default-device changes),DeviceInfo { id, name, is_default },HostError,StreamError.host_cpal:CpalHostoncpal0.18 (ALSA / WASAPI / CoreAudio). Sample formats are converted toi16in a buffer allocated when the stream is built.cpalhas no device notifications, so a small thread compares the device lists every 3 s.host_null:NullHost::new() -> (NullHost, NullControl)withadvance,feed_input,take_output,unplug_input, plusunplug_output,plug,set_default,fail_stream,set_tone,is_open,open_device, andNullHost::with_configfor other rates and channel counts.mixer:Mixerwithset_gain(SourceId, f32),push(SourceId, Frame),remove(&SourceId), 120 ms cap per source, soft clip on the sum, per-source levels.SourceId::{Remote(String), Clip(u64), MicTest}.capture:Capture, the device stream → 48 kHz mono/stereo 10 ms frames step in front of the chain.engine:AudioEngine::start(host, events)withset_input_device,set_output_device,start_capture(ChainConfig) -> Receiver<Frame>,stop_capture,set_chain_config,push_remote,remove_remote,set_gain,devices(),start_mic_test/stop_mic_test;EngineEvent::{Devices, DeviceLost { input }, LocalLevel { rms, gate_open }, RemoteLevels, Error}, levels once per 50 ms of audio.cpal = "0.18",ringbuf = "0.6". No CI change (libasound2-devis already installed on the Linux runner).cargo denyis green without new licences or ignores.Design: the device callbacks only copy between the device buffer and a
ringbufring. One worker thread (fluxer-audio) does the rest: channel layout, resampling, the chain, the mixer. It wakes every 5 ms while a stream is open and sleeps on its command queue otherwise. Devices are opened on demand (microphone while capturing or testing; loudspeaker while the mixer has a source, plus 2 s).Review Focus 6 (device loss): a lost stream is reported as
DeviceLost, the selection falls back to the default device, the stream is reopened in the same worker pass, and the capture receiver and the mixer's sources are kept, so the call goes on. Tests:unplugging_input_falls_back_to_default_and_reports_device_lost,unplugging_output_falls_back_to_default_and_keeps_playing,losing_every_input_device_reports_an_error_and_recovers_when_one_returns,a_new_default_device_takes_over_streams_that_follow_the_default,a_device_that_dies_on_every_open_is_not_reopened_forever.How it was tested
tests/mixer.rs(5) andtests/engine.rs(14), all onNullHost, including the seven tests the plan names.callbacks_do_not_allocateinstalls a counting global allocator and asserts zero allocations on the thread runningNullControl::advanceover 2 s of capture + mic test + two remote participants on 44.1 kHz devices (resampling both ways), after warm-up.FLUXER_SCREENSHOT_TESTS=1 cargo xtask ciis green locally;cargo check --all-targets --target x86_64-pc-windows-msvc -p fluxer-audiotype-checks.CpalHostwas exercised once on a real machine (FLUXER_AUDIO_DEVICE_TESTS=1 cargo test -p fluxer-audio --lib): default input and output opened at 48 kHz stereo and delivered audio. Without that variable the test returns at once, so CI needs no sound card.Deviations from the plan and things to look at
std::sync::mpsc, not tokio: the crate has no async runtime and no task names one. Voice 14 bridges them.AudioEngine::sync()is an addition: it blocks until the worker has caught up. Tests on the virtual clock need it to be deterministic; the application does not.set_gaintakes aSourceId(the mixer's key) rather than an identity string, so Voice 5's clips can use the same call.remove_remote. A participant whose track comes back after a deafen or reconnect keeps their volume without the caller re-sending it.devices()returnsResult<Devices, HostError>withDevices { inputs, outputs }.voice/4-audio-devices-mixernext to thevoicebranch (ref name conflict), so it is pushed asvoice-4-audio-devices-mixer, like Voice 1 to 3.cpal's default (ALSA) backend the lists show ALSA PCM names (default,pipewire,sysdefault:CARD=…), not PipeWire node names, and a default-device change is invisible because the id staysalsa:default(PipeWire reroutes it itself).cpal0.18 has an optional native PipeWire backend; using it would be a new decision for the settings UI task (Voice 21), not made here.NullHostsimulates it.Closes #65
🤖 Generated with Claude Code
Independent review — Voice 4 (audio devices, capture, mixer) @
0743d38Scope:
git diff fc38bd2..0743d38(13 files, cratefluxer-audio+ workspaceCargo.toml/Cargo.lock).1. Named tests — all seven exist and assert what the plan says:
mixer_sums_sources_with_gain_and_soft_clips,mixer_drops_backlog_beyond_120ms(200 ms pushed, exactly 120 ms kept, frames 9..=20 played in order),capture_delivers_10ms_frames_through_chain,unplugging_input_falls_back_to_default_and_reports_device_lost,remote_levels_reported_per_identity,mic_test_loops_input_to_output,callbacks_do_not_allocate(counting global allocator, thread-local to the thread runningNullControl::advance, 2 s after a 0.5 s warm-up, with a self-check that the counter counts). All onNullHost.2. Interfaces —
AudioHost,CpalHost,NullHost::new() -> (NullHost, NullControl),NullControl::{advance, feed_input, take_output, unplug_input},DeviceInfo { id, name, is_default },Mixer,AudioEngine::start(Arc<dyn AudioHost>, mpsc::Sender<EngineEvent>)and every listed method, and all fiveEngineEventvariants are present. Deviations from the letter of the plan are listed under findings (minor).3. Constraints —
fluxer-audiodepends on neitherfluxer-corenor LiveKit; onlycpalandringbufwere added (both named by the task); no logging in non-test code; no secrets. Real-time rule: the engine's callbacks only touch aringbufhalf and atomics; thecpalwrappers convert through a scratch buffer allocated at stream build. I checked the implementer's claim that the 3 s device-list poll opens no device: true for cpal 0.18.2 ALSA (supports_input/outputare answered from the hint direction, not by opening the PCM).4. Review Focus 6 (device loss keeps the call) — holds. Loss sets a flag in the error callback;
devices_changedrunscheck_streams(false)before listing, so the order is alwaysDeviceLostthenDevices, whichever of the flag orHostChangedthe worker sees first (the other order only produces a duplicateDevices). The stream is reopened on the default in the same worker pass; the captureReceiverand the mixer sources survive. Covered for input, output, loss of every device + re-plug, a device that dies on every open, and a default change.5. CI —
cargo xtask ciin the review worktree (own target dir): all steps passed (screenshot step skipped; this PR has no UI). Remote runners not checked by me; Windows/WASAPI paths have never executed.Findings
crates/fluxer-audio/src/engine.rs: signatures differ from the plan's Interfaces block; Voice 5 and Voice 14 must build against the code, not the plan: channels arestd::sync::mpsc(Voice 14 needs a bridging thread to its async runtime),AudioEngine::set_gain(SourceId, f32)rather than an identity string,devices() -> Result<Devices { inputs, outputs }, HostError>, andMixer::remove(&SourceId)(plan: by value).engine.rspump_playback/reconcile_output: the mixer only runs while an output stream is open. On a machine with no usable loudspeaker (output_blocked) remote audio is never mixed, soRemoteLevelsis never emitted and level-based speaking indicators stay dark.engine.rsstart_capture: the frame channel is unbounded; a consumer that stalls (LiveKit publish blocked) grows memory at 100 frames/s with no drop policy, unlike the 120 ms cap on the playback side.engine.rsdevices_changed: a stream opened on an explicitly chosen device is only replaced when the backend reports a stream error; if a backend drops the device from the list without erroring the stream, the engine keeps a dead stream. Also, after any loss the selection is reset to default and not restored when the device returns (documented; the core has to re-send the choice).host_cpal.rs: on Linux the lists are ALSA PCM names and a default-device change is invisible (alsa:defaultnever changes id); WASAPI code is type-checked only. Per CLAUDE.md these deferred items need afollow-upissue linked from this PR before merge.Verdict: APPROVE — no blocking findings, local ci green.