Voice: apply automatic gain control to the microphone (#587) #595

Merged
charles merged 1 commit from fix/587-quiet-voice into textchat 2026-11-15 06:15:40 +00:00
Owner

Closes #587

Root cause

Automatic gain control was never applied. The setting is on by default
(fluxer-core CaptureConfig::auto_gain), and fluxer_voice::apm::Apm wraps WebRTC's audio
processing module, but nothing used it. apps/fluxer/src/audio_rt.rs chain_config dropped
auto_gain ("Echo cancellation and automatic gain … are left out"), and the LiveKit source is
created with auto_gain_control: false (crates/fluxer-voice/src/rtc.rs:439). So each Flaxative
microphone was sent at whatever level the device captured: about −30 to −40 dBFS for typical
speech. The original's Chromium brings the same microphone to about −18 to −23 dBFS. When
everyone in the call uses Flaxative, everyone sounds 7 to 17 dB too quiet.

Measurements (levels in dBFS RMS)

Path Input Before After
Playback: push_remote → mixer → resample → device ring (NullHost; 48/44.1 kHz × 1/2/6 ch; mono and stereo frames) −12.00 (1 kHz sine) −12.00 in all 20 combinations unchanged
Playback: i16 → f32 device conversion (cpal from_sample) full scale 1.0 (same scale as i16) unchanged
Receive: LiveKit interop, Opus → server → NativeAudioStream (mono) → RoomEvent::Audio −15.05 (440 Hz tone) −15.05 unchanged
Capture chain, defaults (RNNoise, auto gate, 100 %, soft clip); real speech −30 / −40 / −20 −30.0 / −40.0 / −20.0 (unity) −23.2 / −22.9 / −20.4 (settled after 12 s; peaks −1.1 / −8.9 / −0.2)
Capture chain, defaults; synthetic speech (regression test) −30 / −40 −30.4 / not run (the test stops at the first failure) −20.0 / −20.5 (peaks −1.2 / −4.7)
WebRTC APM alone (AGC2 adaptive digital, as the livekit FFI configures it), real speech −30 / −40 — −23.0 / −22.8 settled

The real speech is the ALSA test recordings (/usr/share/sounds/alsa/*.wav), measured locally
and not committed. The level is the RMS over the 10 ms speech frames (those within 30 dB of the
loudest frame). RNNoise, the gate and the soft clip change speech by less than 0.1 dB. The
playback and receive paths lose nothing.

Change

  • fluxer-audio: adds a CaptureProcessor trait. CaptureChain::set_processor and
    AudioEngine::set_capture_processor run it first in the chain. This is where Chromium applies
    getUserMedia processing in the original, before its RNNoise → gate → volume → soft clip
    graph. As a result, the gate, the denoiser and the level meter all see the levelled signal.
  • apps/fluxer: when auto_gain is on, the runtime puts fluxer_voice::apm::Apm (AGC2
    adaptive digital plus high-pass filter) in front of the chain. The processor is replaced only
    when its configuration changes, so moving sliders keeps the AGC's state. Echo cancellation is
    still not wired, because it needs the playback signal (#90).

Tests

  • apps/fluxer audio_rt::tests::default_capture_levels_quiet_speech_like_the_original: with
    the default settings, synthetic speech at −30 and −40 dBFS leaves the chain within −25 to
    −18 dBFS and never reaches full scale. Without the processor it fails (−30.4 dBFS).
  • fluxer-audio engine::playback_keeps_the_level_of_a_remote_voice (±0.5 dB),
    f32_devices_get_the_same_scale_as_i16_devices, and
    dsp::chain_runs_the_processor_first_and_keeps_it_across_settings.
  • fluxer-voice livekit_interop::a_tone_arrives_at_the_level_it_was_sent (±0.5 dB;
    FLUXER_LIVEKIT_TESTS=1, passed locally).
  • FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci is green locally.

Not measured here

  • Real devices, including PipeWire stream volume and Windows WASAPI session volume. The NullHost
    has no session volume, and the cpal host does not set one.
  • Chromium's own AGC on the same recording. The −18 to −23 dBFS reference is the usual figure.
    Chromium also uses AGC1's analog input-volume controller, which raises the OS microphone
    volume. This change does not do that, so a very quiet microphone may still need its system
    volume raised.
  • A Flaxative listener with an Electron speaker (#587 item 4). The receive path is shown to be
    lossless with a Flaxative speaker.

🤖 Generated with Claude Code

Closes #587 ## Root cause Automatic gain control was never applied. The setting is on by default (`fluxer-core` `CaptureConfig::auto_gain`), and `fluxer_voice::apm::Apm` wraps WebRTC's audio processing module, but nothing used it. `apps/fluxer/src/audio_rt.rs` `chain_config` dropped `auto_gain` ("Echo cancellation and automatic gain … are left out"), and the LiveKit source is created with `auto_gain_control: false` (`crates/fluxer-voice/src/rtc.rs:439`). So each Flaxative microphone was sent at whatever level the device captured: about −30 to −40 dBFS for typical speech. The original's Chromium brings the same microphone to about −18 to −23 dBFS. When everyone in the call uses Flaxative, everyone sounds 7 to 17 dB too quiet. ## Measurements (levels in dBFS RMS) | Path | Input | Before | After | |---|---|---|---| | Playback: `push_remote` → mixer → resample → device ring (NullHost; 48/44.1 kHz × 1/2/6 ch; mono and stereo frames) | −12.00 (1 kHz sine) | −12.00 in all 20 combinations | unchanged | | Playback: `i16` → `f32` device conversion (cpal `from_sample`) | full scale | 1.0 (same scale as i16) | unchanged | | Receive: LiveKit interop, Opus → server → `NativeAudioStream` (mono) → `RoomEvent::Audio` | −15.05 (440 Hz tone) | −15.05 | unchanged | | Capture chain, defaults (RNNoise, auto gate, 100 %, soft clip); real speech | −30 / −40 / −20 | −30.0 / −40.0 / −20.0 (unity) | −23.2 / −22.9 / −20.4 (settled after 12 s; peaks −1.1 / −8.9 / −0.2) | | Capture chain, defaults; synthetic speech (regression test) | −30 / −40 | −30.4 / not run (the test stops at the first failure) | −20.0 / −20.5 (peaks −1.2 / −4.7) | | WebRTC APM alone (AGC2 adaptive digital, as the livekit FFI configures it), real speech | −30 / −40 | — | −23.0 / −22.8 settled | The real speech is the ALSA test recordings (`/usr/share/sounds/alsa/*.wav`), measured locally and not committed. The level is the RMS over the 10 ms speech frames (those within 30 dB of the loudest frame). RNNoise, the gate and the soft clip change speech by less than 0.1 dB. The playback and receive paths lose nothing. ## Change - `fluxer-audio`: adds a `CaptureProcessor` trait. `CaptureChain::set_processor` and `AudioEngine::set_capture_processor` run it first in the chain. This is where Chromium applies `getUserMedia` processing in the original, before its RNNoise → gate → volume → soft clip graph. As a result, the gate, the denoiser and the level meter all see the levelled signal. - `apps/fluxer`: when `auto_gain` is on, the runtime puts `fluxer_voice::apm::Apm` (AGC2 adaptive digital plus high-pass filter) in front of the chain. The processor is replaced only when its configuration changes, so moving sliders keeps the AGC's state. Echo cancellation is still not wired, because it needs the playback signal (#90). ## Tests - `apps/fluxer` `audio_rt::tests::default_capture_levels_quiet_speech_like_the_original`: with the default settings, synthetic speech at −30 and −40 dBFS leaves the chain within −25 to −18 dBFS and never reaches full scale. Without the processor it fails (−30.4 dBFS). - `fluxer-audio` `engine::playback_keeps_the_level_of_a_remote_voice` (±0.5 dB), `f32_devices_get_the_same_scale_as_i16_devices`, and `dsp::chain_runs_the_processor_first_and_keeps_it_across_settings`. - `fluxer-voice` `livekit_interop::a_tone_arrives_at_the_level_it_was_sent` (±0.5 dB; `FLUXER_LIVEKIT_TESTS=1`, passed locally). - `FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci` is green locally. ## Not measured here - Real devices, including PipeWire stream volume and Windows WASAPI session volume. The NullHost has no session volume, and the cpal host does not set one. - Chromium's own AGC on the same recording. The −18 to −23 dBFS reference is the usual figure. Chromium also uses AGC1's analog input-volume controller, which raises the OS microphone volume. This change does not do that, so a very quiet microphone may still need its system volume raised. - A Flaxative listener with an Electron speaker (#587 item 4). The receive path is shown to be lossless with a Flaxative speaker. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
The default "automatic gain control" setting was never applied: the
runtime dropped `auto_gain` and the WebRTC APM wrapper went unused, so
each microphone went out at its raw level (typical speech -30 to
-40 dBFS). Chromium brings the original's microphone to about -18 to
-23 dBFS.

fluxer-audio gains a `CaptureProcessor` stage that runs first in the
capture chain (`CaptureChain::set_processor`,
`AudioEngine::set_capture_processor`), where Chromium's getUserMedia
processing sits in the original. When auto_gain is on, apps/fluxer puts
WebRTC's AGC2 adaptive digital gain and high-pass filter there. Echo
cancellation is still not wired (#90).

Measured: speech at -30 / -40 dBFS now settles at -23.2 / -22.9 dBFS.
Before, it stayed at -30.0 / -40.0. Playback (-12.00 -> -12.00 for
48/44.1 kHz x 1/2/6 ch) and receive (-15.05 -> -15.05 through LiveKit)
lose nothing; they now have level regression tests.

Refs #587

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
charles merged commit dc426c1e8f into textchat 2026-11-15 06:15:40 +00:00
Sign in to join this conversation.
No description provided.