Voice 3: fluxer-audio crate — frames and DSP (#64) #103
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "voice-3-audio-dsp"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What this builds
New crate
crates/fluxer-audio(no devices, no network, no dependency onfluxer-coreor LiveKit), with the constants of the Electron client:Frame { samples: Vec<i16>, channels: u8 }(48 kHz, 10 ms),SAMPLE_RATE,FRAME_MS,FRAME_LEN.Resampler::new(from_hz, to_hz, channels) -> Result<_, ResampleError>: streaming over rubato's FFT resampler;process(&[i16], &mut Vec<i16>),flush,reset. The filter delay is trimmed andflushends a stream with exactly the input's duration.gain::user_volume_gain(percent)(per-user curve) andgain::percent(p)(linear 0-200 % microphone volume).Gate::new(GateConfig { mode: Auto | Manual { threshold_rms }, attack_ms: 5, release_ms: 20, hold_ms: 180, lookahead_ms: 10 }): a port ofvoiceGateProcessor.worklet.js(floor = 10th percentile of 2 s x 1.5, auto threshold = 3 x floor capped at 0.04, 300 ms seeding, release at 70 % of the threshold).gate::threshold_rms_for_slider.LevelDetector::new(DetectorConfig)(30 ms attack, 220 ms release),level::remote_threshold_rms_for_slider,level::rms.soft_clip(&mut [f32])(knee 0.8, tanh shoulder).Denoiser::{Rnnoise, None}(a plain selector, soChainConfigisCopy + PartialEq) anddenoise::DenoiseProcessorholding the RNNoise state per channel.CaptureChain::new(ChainConfig { denoise, gate, input_volume_percent, stereo }),process(&mut Frame) -> ChainOutput { rms, gate_open }, plusset_config(keeps the gate's noise floor when only the volume or threshold changes) for Voice 4'sset_chain_config.Workspace
Cargo.tomlgainsnnnoiseless(no default features: the defaults build its CLI) andrubato.deny.tomlis unchanged: every new licence was already allowed.How it was tested
crates/fluxer-audio/tests/dsp.rs, synthetic signals only: the nine tests the plan names plusgate_attack_ramp_takes_5ms,rms_of_reference_signalsandchain_never_gates_stereo_capture. Written first against a signatures-only stub (12 failed), then implemented (12 pass).FLUXER_SCREENSHOT_TESTS=1 cargo xtask ciis green locally.Deviations from the plan
user_volume_gain(0.0)is0.0, not10^-1.2. The plan's reference point came from the facts report, which left out the original's special case (VoiceVolumeUtils.ts:24:if (clamped === 0) return 0). Review focus 5 asks to match the original, so 0 % is silence; the test checks the10^-1.2limit just above zero (0.001 %) and exact zero separately. Voice 8/10 should not expect a non-zero gain at 0 %.rnnoise_reduces_white_noise_energyasserts at least 3 dB, measured over seconds 3-4 of white noise at RMS 0.02. RNNoise's attenuation of pure white noise varies a lot with level and seed (measured: 3 % to 12 % of the energy left at this level, but 55-75 % left at RMS 0.05 or 0.2), so a stricter bound would be fragile.voice-3-audio-dsp. The remote has a branchvoice, so git refusesvoice/3-audio-dspthere (refname conflict); same as Voice 1 (#102). The local branch keeps the plan's name.f32(the original clamps and tolerates non-finite values; so does this).README.mdcrate table is not updated (outside this task's file list).For the reviewer
Closes #64
🤖 Generated with Claude Code
Independent review: Voice 3 (audio-dsp), commit
fc38bd2Reviewed
8b37da4..origin/voice-3-audio-dspcold, against the plan (Task 3, Global Constraints, Review Focus 5), spec §6, facts §3/§4 and the fork sources.1. Tests named by the plan — all nine exist in
crates/fluxer-audio/tests/dsp.rsand assert the spec values: gate 5/20/180/10 ms, slider points 0.0003 / 0.008 / 0.04 (log below 50) and 0.002 / 0.006 / 0.035, detector 30/220 ms, soft clip bound, 44.1 k → 48 k duration (exactly 48 000 frames), RNNoise energy drop, volume applied after the gate. Three extra tests (gate_attack_ramp_takes_5ms,rms_of_reference_signals,chain_never_gates_stereo_capture).2. Interfaces block — every name exists as written:
Frame { samples, channels },Resampler::new(from_hz, to_hz, channels)(returnsResult),gain::user_volume_gain,gain::percent,Gate::new(GateConfig { mode: GateMode::{Auto, Manual { threshold_rms }}, attack_ms, release_ms, hold_ms, lookahead_ms })with defaults 5/20/180/10,gate::threshold_rms_for_slider,LevelDetector::new(DetectorConfig),level::remote_threshold_rms_for_slider,soft_clip(&mut [f32]),Denoiser::{Rnnoise, None},CaptureChain::new(ChainConfig { denoise, gate, input_volume_percent, stereo }),process(&mut self, &mut Frame) -> ChainOutput { rms, gate_open }.3. Constraints / invariants —
fluxer-audiodepends only onnnnoiseless,rubato,thiserror(nofluxer-core, no LiveKit); no other crate touched; no secrets or fixtures; malformed frames (0/1/479/481/5000 samples, channels 0/3/255, NaN config) do not panic in my probe. Gate, thresholds, volume curve and soft clip compared line by line withvoiceGateProcessor.worklet.js,VoiceSpeakingThreshold.ts,VoiceVolumeUtils.ts,VoiceSharedAudioContext.ts: faithful port (only difference: 120-sample analysis blocks instead of 128).4. Review Focus 5 — curve matches the original at 50/100/150/200 % to 1e-4. At 0 % the original returns 0 (
VoiceVolumeUtils.ts:if (clamped === 0) return 0), so the implementation is right and the plan's0 → 10^-1.2is wrong (see finding 1). The soundboard cap of 3 is Voice 10's.5.
cargo xtask ciin the review worktree: green (fmt, clippy, deny, core-isolation, parity, test; 12/12 indsp.rs; screenshot step skipped becauseFLUXER_SCREENSHOT_TESTSwas not set). Remote runners not checked.Findings
gain.rs/ plan:user_volume_gain(0.0) == 0.0deviates from the plan's test text but matches the fork. Voice 8 and Voice 10 (and the plan / facts §3) must expect silence at 0 %, not10^-1.2.resample.rs: only rate pairs with a large common divisor work. Probe (1 s of input, thenflush): 8 k, 11.025 k, 16 k, 22.05 k, 32 k, 44.1 k, 88.2 k, 96 k, 176.4 k, 192 k → exactly 48 000 frames, mono and stereo. ButResampler::new(47_999, 48_000, _)delivers nothing while streaming and only 24 000 of 48 000 frames afterflush(the doc promise "exactly the input's duration" does not hold), andResampler::new(u32::MAX, 48_000, 1)did not finish within 60 s instead of returning an error. Rates come from the local device, not the server, and real devices use the rates that work, so not blocking; suggest a follow-up issue to reject rates outside a sane range before Voice 4 passes device rates in.level.rs:LevelDetector::updateaccumulatesdt_ms, so speaking turns on one tick earlier than the original's timestamp difference (advanceSpeakingHysteresis): on the 3rd 10 ms frame instead of the 4th, and on the first 50 ms window instead of the second. Not audible; note it if Voice 8 compares indicator timing with the original.rnnoise_reduces_white_noise_energyasserts only 3 dB. Acceptable: fixed seed, pure-Rust network, passes locally; I did not verify the Windows runner.README.mdcrate table has nofluxer-audiorow (outside this task's file list; the crate README itself is complete).Verdict: APPROVE — zero blocking findings, local ci green.