Screen share 14: Linux stream audio capture (PipeWire) (#161) #198
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "screenshare-14-audio-capture-linux"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What this builds
Linux stream audio for screen share (spec §9), ported from the original's
fluxer_desktop/native/linux-audio-capture(AGPL, attribution headers in each file), without napi:fluxer_capture::pipewire_audio::PipeWireAudioCapturer: StreamAudioCapturer.new()probes the daemon (none →CaptureError::Unsupported);with_remote(name)for tests.startspawns one PipeWire thread per session: hidden private null sink (fluxer-native-stream-audio-<pid>-<n>), registry watch, passive L/R links from every node the rule selects (mono feeds both), and once the sink is in the registry a capture stream on its monitor (F32 48 kHz stereo) →FrameChunker→ 10 ms stereoi16frames to the sink.set_rulere-routes live;stop/drop tears down links, stream and sink and joins the thread (sink dropped beforestopreturns).routing.rs(pure): the original'sshould_route_node,matches_pattern,targets_default_sink,parse_default_sink_name,pick_node_ports, client enrichment, desired-link computation;RoutingRule::from_audio_rulebuilds the rule asScreenShareStartFlow.ts+NativeAudio.ts normalizeLinuxRoutingRuledo (System: only/default speakers or every playback stream; Applications:application.name/application.process.binary/ PID tree, any output; empty Applications falls back to System like the original;ignore_input_media,ignore_virtual,ignore_devices).self_identity.rs: PID tree ofexclude_pid_treefrom/proc, plus a parent-chain walk so descendants started afterstartare excluded too; binary names from/proc/<pid>/{comm,exe}; product namesfluxer-native,Fluxer Native,Fluxer,Fluxer Canaryand the original's prefixes.list_applications() -> Vec<AudioApp { name, pid: Option<u32>, binary: Option<String> }>(one-shot registry enumeration, client hidden).AudioApplives inaudio.rs(OS-neutral) and is re-exported.default_backends()on Linux returns the PipeWire capturer when a daemon answers, elseUnsupportedwith the reason.fluxer-audio's identity in PipeWire (verified, recorded in the README)
On PipeWire 1.6.11, an ALSA (cpal) playback stream shows as node
alsa_playback.<binary>,application.name=PipeWire ALSA [<binary>], with the client'sapplication.process.id= PID andapplication.process.binary= the executable name. cpal cannot setapplication.nametofluxer-native, so the PID tree and binary name exclude the client's own audio (the spec's stated fallback). No change tofluxer-audio.Tests
rule_matches_like_the_fork(table: default sink by name/serial, other sink, non-playback classes, only-speakers off, applications by name/binary/PID tree, empty applications, ignore_virtual, pattern details),self_identity_excludes_own_tree_and_names,input_media_and_devices_ignored_by_default,default_sink_name_parses_like_the_fork,graph_links_left_and_right_into_the_sink_and_enriches_from_the_client,stat_parsing_and_tree_walk,f32_chunks_become_10ms_stereo_i16_frames.tests/pipewire_real.rs::no_pipewire_daemon_is_an_error.FLUXER_CAPTURE_TESTS=1), ran on this machine (CachyOS, KDE Wayland session, PipeWire 1.6.11 + WirePlumber, other audio playing in Firefox at the time), 3 runs, all green:system_capture_hears_a_test_tone_from_another_process(1234 Hz tone from apw-playre-parented outside our tree, Goertzel amplitude > 0.1),own_playback_not_captured(apw-playchild of the test is silent even when the rule names it; control: the same rule hears the detached one),list_applications_shows_a_playing_stream_but_not_ourselves.FLUXER_SCREENSHOT_TESTS=1 cargo xtask cigreen locally.Deviations / notes for the reviewer
PipeWireAudioCapturerfirst.processruns on the session's own (non-real-time) PipeWire thread instead of the original'sRT_PROCESSdata thread, so the sink may allocate a frame; nothing runs on PipeWire's RT thread.tests/fake.rs::unsupported_backends_return_errorsnow checks the audio side onUnsupporteditself, because on a desktop with PipeWiredefault_backends()returns the real backend.deny.toml: license exception fortarget-lexicon(Apache-2.0 WITH LLVM-exception, build-time only viasystem-deps).pipewire0.10 (default features, works with Ubuntu 22.04's libpipewire 0.3.48); bindgen needs libclang, which the Linux runner already has (clang-21depends onlibclang1-21, checked in an ubuntu:22.04 container), so no CI workflow change.apps/fluxerdoes not linkfluxer-captureyet; when Screen share 21 wires it, the AppImage needslibpipewire-0.3.so.0(system library, present on every PipeWire desktop) — Screen share 30's concern.crates/fluxer-capture/README.md(the task's Files list), not in the spikes doc.Update: merged origin/screenshare (Screen share 1-8, 15)
fluxer-capture(Cargo.toml,README.md,src/lib.rs,tests/fake.rs),deny.tomlandCargo.lock.default_backends()now returnsDesktopScreenCapturer(Linux),PipeWireAudioCapturer(Linux, daemon present) andWasapiAudioCapturer(Windows with process loopback), each falling back toUnsupportedwith the reason.tests/fake.rs::unsupported_backends_return_errorsbuildsdefault_backends()(must not panic) and then tests bothUnsupportedstubs directly.fix(capture): initialise libwebrtc's PipeWire stubs before pw_init): once libwebrtc is linked into the same binary (thedesktopbackend from Screen share 4, andfluxer-voice), thepipewirecrate'spw_*calls bind to libwebrtc's generatedpipewire_stubs.o, whose function pointers are null until libwebrtc dlopens libpipewire, soPipeWireAudioCapturer::new()segfaulted inpw_init(gdb: jump to 0x0 frompipewire::init). Every PipeWire entry point now callswebrtc::InitializePipeWire()first (extern by its mangled name_ZN6webrtc18InitializePipeWireEv; no arguments, no libwebrtc type crosses the boundary), which fills the stubs once per process; iflibpipewire-0.3.so.0cannot be loaded the capturer isUnsupported. This matters for Screen share 21:apps/fluxerlinks both.FLUXER_SCREENSHOT_TESTS=1 cargo xtask cigreen locally;FLUXER_CAPTURE_TESTS=1 cargo test -p fluxer-capture --test pipewire_real -- --test-threads 1re-ran on this machine (CachyOS, KDE Wayland, PipeWire 1.6.11), 4/4 green, now with libwebrtc in the test binary.Update: merged origin/screenshare again (Screen share 4, 5, 9, 16)
fluxer-capture(Cargo.toml,README.md,src/lib.rs,tests/fake.rs).desktop.rsis nowsrc/desktop/and runs on Linux and Windows.default_backends()still returns the PipeWire capturer on Linux when a daemon answers, and WASAPI on Windows.FLUXER_SCREENSHOT_TESTS=1 cargo xtask ciis green locally.FLUXER_CAPTURE_TESTS=1 cargo test -p fluxer-capture --test pipewire_real -- --test-threads 1also passes 4/4 on this machine (CachyOS, KDE Wayland, PipeWire 1.6.11).Closes #161
🤖 Generated with Claude Code
Independent review: Screen share 14 (Linux stream audio, PipeWire)
Reviewed
d821f19..136a1a6cold, against the plan's Task 14, spec §9 and the fork'slinux-audio-capture.Plan tests (1): all named tests exist and assert what the plan says:
rule_matches_like_the_fork,self_identity_excludes_own_tree_and_names,input_media_and_devices_ignored_by_default(unit);no_pipewire_daemon_is_an_error(CI);system_capture_hears_a_test_tone_from_another_process,own_playback_not_captured(gated onFLUXER_CAPTURE_TESTS=1).should_route_node,targets_default_sink,matches_patternandparse_default_sink_namematch the fork line for line. The defaults come from Task 3'sAudioRule::default(), which matches spec §9.Interfaces (2):
PipeWireAudioCapturer: StreamAudioCapturer,list_applications() -> Vec<AudioApp { name, pid, binary }>, hidden private null sink, passive links, F32 to i16 10 ms stereo 48 kHz frames, and exclusion by PID tree, binary and name are all present. Attribution headers are present.Invariants (3):
fluxer-capturegains onlypipewire(Linux only) andtracing. It has nofluxer-coreor livekit dependency, and no PipeWire type appears in a public signature. With no daemon, every failure path returnsCaptureError::UnsupportedorBackendand never panics. Buffer slicing is bounds-checked. There are no secrets.Review Focus 7: own-output exclusion is by PID tree (including descendants started later, through the ppid walk), by binary name and by product names. The README records that cpal/ALSA cannot set
application.name, so the spec's PID-only fallback applies. That is acceptable. Note that the real test uses apw-playchild, notfluxer-audio's cpal path. The ALSA plug-in's client carries ourpipewire.sec.pid, so the PID match covers it.CI (5):
FLUXER_SCREENSHOT_TESTS=1 cargo xtask ciis green in the review worktree. All task tests ran. I did not re-run the real tests, because they play an audible tone on the desktop. The implementer reports 3 passing runs.Findings
list_applicationsis an inherent method ofPipeWireAudioCapturer, butdefault_backends()returns aBox<dyn StreamAudioCapturer>. Task 21 therefore cannot reach it through the default backend. It must keep a concretePipeWireAudioCapturer(cfg linux) or add a trait method. The Interfaces block does not specify which, so Task 21 should be told.SessionState::recomputeruns on every registry global, ports included. Each run callsSelfIdentity::matchesfor every node, and for unknown PIDs that walks up to 64/proc/<pid>/statreads. On a busy graph this is O(globals × nodes × depth) file reads on the PipeWire thread before the syncdonearrives, which eats into the 2 sREADY_TIMEOUT. Suggested fix: cache ancestry results per PID, or defer recompute until after the initial sync.AppMatch::Pidis expanded into a PID set once, atstart/set_rule. Children the target app spawns later (browser audio processes) are not routed, unlike self-exclusion, which walks ancestry. The fork resolves the tree once too, so this is not a regression.global_removeofsink_id) or the capture stream errors, the session silently stops producing frames, and the trait gives no way to report it. This is worth a follow-up issue for Task 21 (e.g. treat a long gap as "stream audio lost").processcallback allocates aVec<f32>per buffer. It runs on the session's non-RT thread (noRT_PROCESS), so this does not break the real-time rule. The deviation is documented, and the allocation is avoidable.No blocking findings. Electron names are not excluded, which is a justified deviation. The
tests/fake.rschange is justified, and thedeny.tomlexception fortarget-lexiconis build-time only and fine.Verdict: approve. No blocking findings; local CI is green.
Independent re-review: Screen share 14 after the merge of screenshare (f3fd2a0..2e267f7)
I read the diff
f3fd2a0..origin/screenshare-14-audio-capture-linuxcold, focusing on what changed since the approving review (comment 3961): the merge resolution and theInitializePipeWirefix.Plan tests (1): every named test exists and asserts what the plan says. Unit tests:
rule_matches_like_the_fork,self_identity_excludes_own_tree_and_names,input_media_and_devices_ignored_by_default. CI test:no_pipewire_daemon_is_an_error. Real tests, which needFLUXER_CAPTURE_TESTS=1:system_capture_hears_a_test_tone_from_another_processandown_playback_not_captured. The defaults match spec §9: System, true, true, true, true, false.Interfaces (2):
PipeWireAudioCapturer: StreamAudioCapturer,list_applications() -> Vec<AudioApp { name, pid, binary }>andAudioApp(re-exported from the crate root) exist as written.default_backends()keeps its signature. Linux and Windows each get acfg-gateddefault_audio_backend, and each falls back toUnsupportedwith the reason.Invariants (3):
fluxer-capturegains onlypipewire(Linux) andtracing. It has nofluxer-coreorlivekitdependency, and no PipeWire or libwebrtc type appears in a public signature. The merged Linux dependency table is correct:pipewirepluslibwebrtcwithglib-main-loop. Thedeny.tomltarget-lexiconnote is build-time only. Graph data from PipeWire cannot panic the client:parse_default_sink_nameonly slices at"bytes, and the buffer slicing is checked. There are no secrets.The
InitializePipeWirefix: I verified it. The prebuiltlibwebrtc.a(linux-x64) definesT webrtc::InitializePipeWire()and referencespw_initthrough its stubs. The function body is a function-local static, so it is thread-safe and runs once. Bothprobe,run_sessionandsnapshot_graphcall it beforepw::init(). The C++bool()signature with no arguments is ABI-compatible with the Rust declaration.Review Focus 7: own output is excluded by PID tree (with an ancestry walk for later descendants), by binary name and by product names. The README records what cpal/ALSA reports to PipeWire. This is unchanged from the first review.
CI (5):
FLUXER_SCREENSHOT_TESTS=1 cargo xtask ciis green in the review worktree (all steps passed), and all of the task's unit tests and the CI test ran. I did not run the real tests, because they play an audible tone. The implementer reports 4/4 passing on PipeWire 1.6.11.Findings
pipewire_audio/mod.rsdeclares the libwebrtc C++ symbol_ZN6webrtc18InitializePipeWireEvitself. No libwebrtc type crosses the boundary, so the Global Constraint (types confined todesktop.rs/rtc.rs) holds as written. It does add a second place that depends on libwebrtc's ABI, and the mangled name ties it to the pinned 0.4.0 build. Moving theexternintodesktop.rsas apub(crate) fn initialize_pipewire() -> boolwould keep every libwebrtc coupling in one file. If the symbol ever disappears, the link fails loudly, so nothing breaks silently.list_applicationsis inherent to the concrete type, anddefault_backends()returnsBox<dyn StreamAudioCapturer>. Task 21 must constructPipeWireAudioCaptureritself on Linux to fill the "Audio sources" submenu. It should also callInitializePipeWire-dependent code only through this crate, which it does by construction.list_applicationsgives up after 2 s, but the spawnedfluxer-pipewire-listthread keeps waiting inmainloop.run()for adonethat may never come, for example from a wedged daemon. That thread leaks. Each call would leak at most one thread, so this is not blocking.follow-upissue linked from this PR before merge, asCLAUDE.mdrequires.No blocking findings.
Verdict: approve. There are no blocking findings and local CI is green.
Independent review: Screen share 14 (Linux stream audio, PipeWire), head
2e75321I reviewed only
61a073c..2e75321and focused on how the merge conflicts were resolved.Merge resolution
src/lib.rs:default_audio_backendis now three cfg-gated functions. Linux usesPipeWireAudioCapturer::new()orUnsupported(err). Windows usesWasapiAudioCapturerorUnsupported(reason). Every other OS getsUnsupported. The doc comment matches the code, and nothing from Screen share 4/5 (thedesktopre-export,default_screen) was lost. OK.Cargo.toml: the Linux target block now holds bothpipewireandlibwebrtcwithglib-main-loop, with no duplicate table.tempfileis a dev-dependency. OK.desktop,wasapiandpipewire_audiotogether. OK.tests/fake.rs:unsupported_backends_return_errorsnow checks thatdefault_backends()builds without panicking, and tests the stubs directly. That fits now that every desktop OS has a real backend. OK.deny.toml: thetarget-lexiconexception comment now names both users. OK.Plan checks
rule_matches_like_the_fork,self_identity_excludes_own_tree_and_namesandinput_media_and_devices_ignored_by_default, the CI testno_pipewire_daemon_is_an_error, and the real testssystem_capture_hears_a_test_tone_from_another_processandown_playback_not_captured(the last with a control) are all present. Frames are checked at 480×2 samples, 2 channels.PipeWireAudioCapturer: StreamAudioCapturer,list_applications() -> Vec<AudioApp { name, pid, binary }>andAudioAppre-exported from the crate root are all present as written.fluxer-capturedoes not pull influxer-coreorlivekit. No libwebrtc or LiveKit type appears in a public signature. Server data, such as chunk offset and size, PIDs and prop strings, is handled with checked arithmetic,getand parse-or-ignore, so it cannot panic the client. The diff contains no secrets.application.nameandapplication.process.id, as the plan asks.FLUXER_SCREENSHOT_TESTS=1 cargo xtask ciin a fresh review worktree: all steps passed.Findings
init_pipewiredeclares anextern "C"to libwebrtc'swebrtc::InitializePipeWire(by mangled name) inpipewire_audio/mod.rs, outsidesrc/desktop/. No libwebrtc type crosses the boundary, so the constraint on types is met, but the libwebrtc coupling sits outside the sanctioned module. This is already tracked in #225.list_applicationsis an inherent method, so the runtime cannot reach it through theBox<dyn StreamAudioCapturer>thatdefault_backends()returns. Screen share 21/26 will need a way to reach it. This is tracked in #225, and it does not break any interface written in the plan.Verdict: approve. No blocking findings, and local CI is green.