Screen share 15: Windows stream audio capture (WASAPI process loopback) (#162) #199

Manually merged
charles merged 0 commits from screenshare-15-audio-capture-windows into screenshare 2026-11-12 05:17:19 +00:00
Owner

What

fluxer_capture::wasapi (spec §9, Windows): WasapiAudioCapturer: StreamAudioCapturer on WASAPI process loopback, ported from the fork's fluxer_desktop/native/win-process-loopback (activation, format negotiation, packet loop, kernel32 build check, process-tree walk) with an AGPL attribution header, without napi, the sample pool or the session mixer.

  • AudioMode::System → PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE on exclude_pid_tree (the client never hears itself).
  • AudioMode::Applications → INCLUDE_TARGET_PROCESS_TREE on the first AppMatch::Pid; a pid in the client's own tree is refused, a rule without a pid is Unsupported.
  • Float 48 kHz stereo (16-bit PCM fallback) → FrameAssembler → exactly 10 ms i16 frames, whatever the packet sizes; silent packets are zeros.
  • supported() / unsupported_reason(): kernel32 build ≥ 20348 (PROCESS_LOOPBACK_MIN_BUILD), cached; always unsupported off Windows (the module compiles everywhere so the unit tests run on the Linux runner).
  • start blocks until the stream runs or fails (activation timeout 5 s → Backend, e.g. no audio service on Server Core); set_rule on a running session reopens it with the same sink; stop is idempotent and joins the thread; drop stops.
  • default_backends() returns it on Windows when supported, else Unsupported(reason).

Tests

  • Unit (all OSes): pcm16_and_f32_convert_to_i16_frames, frames_are_exactly_10ms, plus system_mode_excludes_the_given_tree, application_mode_includes_the_first_pid_tree_never_our_own, pid_tree_walk_survives_cycles_and_gaps, support_follows_the_build. Seen failing against stubbed converter/assembler, then passing.
  • CI: tests/wasapi_real.rs::server_core_without_audio_is_an_error_not_a_panic (runs on both runners).
  • Real (FLUXER_CAPTURE_TESTS=1, Windows only): system_mode_hears_other_process_not_self, application_mode_hears_only_target — tones from WAV files, the other processes started through WMI so they are not our descendants, Goertzel detection. Not run: no Windows desktop available to the implementer (Linux machine). They compile and pass clippy for x86_64-pc-windows-msvc (cargo clippy --target x86_64-pc-windows-msvc -p fluxer-capture --all-targets -- -D warnings).
  • FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci green locally (Linux).

Deviations

  • windows-core 0.62 added next to windows 0.62 (the #[implement] macro of the activation completion handler expands to ::windows_core paths; same version family, already in the graph via cpal). Win32_Security feature is needed for CreateEventW.
  • tests/fake.rs::unsupported_backends_return_errors (Task 3's file): the stream-audio half is skipped on Windows when process loopback is supported, since default_backends() now returns the real backend there.
  • Applications mode captures one process tree (the first pid); multiple applications would need the fork's session mixer, not in scope.
  • No error channel exists on StreamAudioCapturer; a mid-session WASAPI failure just ends the frames. The capture thread does not raise its priority (the fork uses its rt-thread helper).
  • Commit trailer uses the actual model (Claude Opus 5.5) rather than the Claude Fable 5.2 line in CLAUDE.md.

Stacked on: screenshare-3-capture-crate

Closes #162

🤖 Generated with Claude Code

## What `fluxer_capture::wasapi` (spec §9, Windows): `WasapiAudioCapturer: StreamAudioCapturer` on WASAPI process loopback, ported from the fork's `fluxer_desktop/native/win-process-loopback` (activation, format negotiation, packet loop, kernel32 build check, process-tree walk) with an AGPL attribution header, without napi, the sample pool or the session mixer. - `AudioMode::System` → `PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE` on `exclude_pid_tree` (the client never hears itself). - `AudioMode::Applications` → `INCLUDE_TARGET_PROCESS_TREE` on the first `AppMatch::Pid`; a pid in the client's own tree is refused, a rule without a pid is `Unsupported`. - Float 48 kHz stereo (16-bit PCM fallback) → `FrameAssembler` → exactly 10 ms `i16` frames, whatever the packet sizes; silent packets are zeros. - `supported()` / `unsupported_reason()`: kernel32 build ≥ 20348 (`PROCESS_LOOPBACK_MIN_BUILD`), cached; always unsupported off Windows (the module compiles everywhere so the unit tests run on the Linux runner). - `start` blocks until the stream runs or fails (activation timeout 5 s → `Backend`, e.g. no audio service on Server Core); `set_rule` on a running session reopens it with the same sink; `stop` is idempotent and joins the thread; drop stops. - `default_backends()` returns it on Windows when supported, else `Unsupported(reason)`. ## Tests - Unit (all OSes): `pcm16_and_f32_convert_to_i16_frames`, `frames_are_exactly_10ms`, plus `system_mode_excludes_the_given_tree`, `application_mode_includes_the_first_pid_tree_never_our_own`, `pid_tree_walk_survives_cycles_and_gaps`, `support_follows_the_build`. Seen failing against stubbed converter/assembler, then passing. - CI: `tests/wasapi_real.rs::server_core_without_audio_is_an_error_not_a_panic` (runs on both runners). - Real (`FLUXER_CAPTURE_TESTS=1`, Windows only): `system_mode_hears_other_process_not_self`, `application_mode_hears_only_target` — tones from WAV files, the other processes started through WMI so they are not our descendants, Goertzel detection. **Not run**: no Windows desktop available to the implementer (Linux machine). They compile and pass clippy for `x86_64-pc-windows-msvc` (`cargo clippy --target x86_64-pc-windows-msvc -p fluxer-capture --all-targets -- -D warnings`). - `FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci` green locally (Linux). ## Deviations - `windows-core` 0.62 added next to `windows` 0.62 (the `#[implement]` macro of the activation completion handler expands to `::windows_core` paths; same version family, already in the graph via cpal). `Win32_Security` feature is needed for `CreateEventW`. - `tests/fake.rs::unsupported_backends_return_errors` (Task 3's file): the stream-audio half is skipped on Windows when process loopback is supported, since `default_backends()` now returns the real backend there. - Applications mode captures one process tree (the first pid); multiple applications would need the fork's session mixer, not in scope. - No error channel exists on `StreamAudioCapturer`; a mid-session WASAPI failure just ends the frames. The capture thread does not raise its priority (the fork uses its rt-thread helper). - Commit trailer uses the actual model (Claude Opus 5.5) rather than the `Claude Fable 5.2` line in `CLAUDE.md`. Stacked on: screenshare-3-capture-crate Closes #162 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(video): frames, conversions, frame bus, test pattern and preview JPEG
All checks were successful
ci / Linux (x86_64) (push) Successful in 15m16s
ci / Windows (x64) (push) Successful in 16m21s
8b8a359082
Refs #148

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(capture): capture traits and fake backends
All checks were successful
ci / Windows (x64) (push) Successful in 16m9s
ci / Linux (x86_64) (push) Successful in 35m35s
d821f19097
Refs #150

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(capture): Windows stream audio
All checks were successful
ci / Windows (x64) (push) Successful in 16m14s
ci / Linux (x86_64) (push) Successful in 35m48s
9cbb0b4b27
WASAPI process loopback behind StreamAudioCapturer: system mode excludes
the client's process tree, application mode includes the picked pid's
tree; float or 16-bit PCM cut into exact 10 ms 48 kHz stereo i16 frames;
supported() checks for build 20348. Ported from the fork's
win-process-loopback without napi.

Refs #162

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
Owner

Independent review: Screen share 15 (Windows stream audio, WASAPI process loopback)

Reviewed d821f19..9cbb0b4 cold; the implementer's notes were checked against the code.

Checks

  1. Plan tests. All five exist. pcm16_and_f32_convert_to_i16_frames and frames_are_exactly_10ms (unit, in src/wasapi/mod.rs) check clamping, the 32767 scale, NaN as silence, PCM pass-through, 960-sample (480 × 2) frames across awkward packet lengths, and that nothing is lost or reordered. server_core_without_audio_is_an_error_not_a_panic runs on CI. system_mode_hears_other_process_not_self and application_mode_hears_only_target sit behind FLUXER_CAPTURE_TESTS=1 (Goertzel on 440/1000 Hz tones, players started through WMI so they fall outside our tree). There are also extra unit tests for the target rules, the pid-tree walk and the build gate.
  2. Interfaces. WasapiAudioCapturer: StreamAudioCapturer (start/set_rule/stop with Task 3's merged signatures) and wasapi::supported() -> bool are present. unsupported_reason() and PROCESS_LOOPBACK_MIN_BUILD = 20_348 are additive. default_backends() keeps its signature and returns WASAPI on supported Windows.
  3. Invariants. fluxer-capture gains only windows/windows-core 0.62, target-gated to cfg(windows) (the version cpal already locks; Cargo.lock adds no new package). It has no fluxer-core or livekit dependency, no libwebrtc types, and no public signature exposes a Windows type. No secrets.
  4. Review Focus 7 (own output excluded). System mode uses PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE on exclude_pid_tree. Application mode uses INCLUDE on the picked pid and refuses any pid inside our own tree (snapshot walk, safe against cycles). This matches spec §9 and the fork (lib.rs:1129-1245,1391-1487). The retry HRESULTs (0x80040155/0x80070002/0x80070490), the backoff and the float→PCM16 fallback are faithful ports. The unsafe code holds up: the PROPVARIANT VT_BLOB is in ManuallyDrop so PropVariantClear never frees the stack params; the handler is IAgileObject and the result goes through Arc<Completion> + Mutex + event, so a late completion after the 5 s timeout writes into memory that is still alive; COM objects drop before CoUninitialize; buffers are released with their own frame count. Send/Sync on Completion is justified: the IAudioClient comes from the MTA and is consumed on an MTA thread.
  5. CI. FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci in the review worktree passed ("all steps passed"). cargo clippy -p fluxer-capture --all-targets --target x86_64-pc-windows-msvc -- -D warnings is clean too. I did not check the remote Windows runner.

Findings

  • minor: The real-device tests (Review Focus 7's Windows check) have never run, because no Windows desktop was available. The code reads correctly, but self-exclusion on Windows is unverified until someone runs FLUXER_CAPTURE_TESTS=1 cargo test -p fluxer-capture --test wasapi_real -- --test-threads=1 on a Win10 20348+/Win11 desktop. Suggest a follow-up issue linked from this PR, or putting it on the Screen share 31 acceptance checklist.
  • minor: server_core_without_audio_is_an_error_not_a_panic asserts no error when supported() is true (for example, a Server 2022 runner is build 20348). On such a runner it only proves "no panic, and AlreadyStarted while running". That is acceptable, but the name promises more. If activation can hang, start() also blocks the test for up to 5 s.
  • minor: set_rule compares the whole AudioRule. In System mode, a change to only the Linux-only flags (only_speakers, ignore_*) tears down the loopback stream and reopens it, which causes a short gap in stream audio for no effect. Comparing the LoopbackTarget instead would avoid that.
  • minor: If reopening fails in set_rule, the session is gone and the caller gets Err, while the sink has been dropped. That is consistent with the trait, which has no error channel, but Screen share 21 must treat set_rule Err as "stream audio ended".
  • note: The kernel32 version check hardcodes C:\Windows\System32 (same as the fork, lib.rs:797). On a Windows installed elsewhere, supported() is false and the reason is a clear message, so nothing panics.

No blocking findings.

Verdict: APPROVE. No blocking findings and local CI is green. Please open a follow-up for the unrun real-device tests.

## Independent review: Screen share 15 (Windows stream audio, WASAPI process loopback) Reviewed `d821f19..9cbb0b4` cold; the implementer's notes were checked against the code. **Checks** 1. **Plan tests.** All five exist. `pcm16_and_f32_convert_to_i16_frames` and `frames_are_exactly_10ms` (unit, in `src/wasapi/mod.rs`) check clamping, the 32767 scale, NaN as silence, PCM pass-through, 960-sample (480 × 2) frames across awkward packet lengths, and that nothing is lost or reordered. `server_core_without_audio_is_an_error_not_a_panic` runs on CI. `system_mode_hears_other_process_not_self` and `application_mode_hears_only_target` sit behind `FLUXER_CAPTURE_TESTS=1` (Goertzel on 440/1000 Hz tones, players started through WMI so they fall outside our tree). There are also extra unit tests for the target rules, the pid-tree walk and the build gate. 2. **Interfaces.** `WasapiAudioCapturer: StreamAudioCapturer` (start/set_rule/stop with Task 3's merged signatures) and `wasapi::supported() -> bool` are present. `unsupported_reason()` and `PROCESS_LOOPBACK_MIN_BUILD = 20_348` are additive. `default_backends()` keeps its signature and returns WASAPI on supported Windows. 3. **Invariants.** fluxer-capture gains only `windows`/`windows-core` 0.62, target-gated to `cfg(windows)` (the version cpal already locks; Cargo.lock adds no new package). It has no fluxer-core or livekit dependency, no libwebrtc types, and no public signature exposes a Windows type. No secrets. 4. **Review Focus 7 (own output excluded).** System mode uses `PROCESS_LOOPBACK_MODE_EXCLUDE_TARGET_PROCESS_TREE` on `exclude_pid_tree`. Application mode uses INCLUDE on the picked pid and refuses any pid inside our own tree (snapshot walk, safe against cycles). This matches spec §9 and the fork (`lib.rs:1129-1245,1391-1487`). The retry HRESULTs (0x80040155/0x80070002/0x80070490), the backoff and the float→PCM16 fallback are faithful ports. The unsafe code holds up: the PROPVARIANT VT_BLOB is in `ManuallyDrop` so `PropVariantClear` never frees the stack params; the handler is `IAgileObject` and the result goes through `Arc<Completion>` + Mutex + event, so a late completion after the 5 s timeout writes into memory that is still alive; COM objects drop before `CoUninitialize`; buffers are released with their own frame count. `Send/Sync` on `Completion` is justified: the IAudioClient comes from the MTA and is consumed on an MTA thread. 5. **CI.** `FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci` in the review worktree passed ("all steps passed"). `cargo clippy -p fluxer-capture --all-targets --target x86_64-pc-windows-msvc -- -D warnings` is clean too. I did not check the remote Windows runner. **Findings** - **minor**: The real-device tests (Review Focus 7's Windows check) have never run, because no Windows desktop was available. The code reads correctly, but self-exclusion on Windows is unverified until someone runs `FLUXER_CAPTURE_TESTS=1 cargo test -p fluxer-capture --test wasapi_real -- --test-threads=1` on a Win10 20348+/Win11 desktop. Suggest a `follow-up` issue linked from this PR, or putting it on the Screen share 31 acceptance checklist. - **minor**: `server_core_without_audio_is_an_error_not_a_panic` asserts no error when `supported()` is true (for example, a Server 2022 runner is build 20348). On such a runner it only proves "no panic, and AlreadyStarted while running". That is acceptable, but the name promises more. If activation can hang, start() also blocks the test for up to 5 s. - **minor**: `set_rule` compares the whole `AudioRule`. In System mode, a change to only the Linux-only flags (`only_speakers`, `ignore_*`) tears down the loopback stream and reopens it, which causes a short gap in stream audio for no effect. Comparing the `LoopbackTarget` instead would avoid that. - **minor**: If reopening fails in `set_rule`, the session is gone and the caller gets `Err`, while the sink has been dropped. That is consistent with the trait, which has no error channel, but Screen share 21 must treat `set_rule` Err as "stream audio ended". - **note**: The kernel32 version check hardcodes `C:\Windows\System32` (same as the fork, `lib.rs:797`). On a Windows installed elsewhere, `supported()` is false and the reason is a clear message, so nothing panics. No blocking findings. **Verdict: APPROVE.** No blocking findings and local CI is green. Please open a follow-up for the unrun real-device tests.
charles manually merged commit 69edce3d34 into screenshare 2026-11-12 05:17:19 +00:00
Sign in to join this conversation.
No description provided.