Screen share 21: apps/fluxer screen-share runtime wiring (#168) #245

Manually merged
charles merged 3 commits from screenshare-21-app-screenshare-runtime into screenshare 2026-11-12 10:46:00 +00:00
Owner

Stacked on: screenshare-13-core-previews (screenshare-14-audio-capture-linux, screenshare-17-voice-livekit-video and screenshare-20-devserver-screenshare were already on screenshare at 2e06bfe).

What this builds

apps/fluxer now executes the screen-share half of the store:

  • capture_rt.rs: Effect::Capture runs on a worker thread that owns the ScreenCapturer and StreamAudioCapturer (the system backend is made on that thread, so PipeWire/X11 probing never holds up startup). Port types are converted both ways. Captured frames go to VoiceHandle::push_video and FrameBus[LocalCapture], gated to the share's frame rate (FrameGate); stream audio (own PID tree excluded) to push_screen_audio; picker thumbnails to FrameBus[Source { id }]. Started/Stalled/Resumed/Ended become CoreEvent::Capture; a failed start becomes Error, a cancelled system picker Ended { Cancelled }. CaptureBackend::{system, fake, unsupported, new}, Runtime::with_capture_backend, Handle::frame_bus().
  • preview_rt.rs: UploadStreamPreview encodes the latest LocalCapture frame with preview_jpeg, gets the presigned URL (cached in memory, renewed 60 s before expiry, dropped on a failed PUT) and PUTs it. The privacy switch is checked again just before the PUT and drops the cached URL. FetchStreamPreview decodes the JPEG into FrameBus[Preview] (404 removes the key).
  • voice_rt.rs: remote video of the screen shares the store subscribed to goes to FrameBus[Remote { sid }], and is taken off when the store unsubscribes (late frames dropped), the track ends, the participant leaves or the room is left. Remote screen-share sound goes to the new mixer source Stream(identity), so SetGain { Stream } (stream volume) now applies.
  • --capture-fake (feature capture-fake, on with devserver); --devserver screenshare uses the fake capture, and its fake room now plays every other member's stream from VoiceScript::streams() (test pattern at 15 fps and a 440 Hz stereo tone, only while subscribed).
  • --debug-info lists the capture backends (screen, stream audio, in-app picker, window audio), encodable codecs, hardware encoders and decodable codecs (MediaInfo).
  • --record also redacts upload_url and thumbnail (defence in depth; the preview requests are already unobserved in fluxer-api).
  • Chimes ScreenShareStart/Stop now play stream-start.mp3 / stream-stop.mp3 (copied from the fork's fluxer_app/src/media/sounds/).

Tests

apps/fluxer/tests/screenshare_runtime.rs (devserver screenshare + FakeRoom + NullHost + fake capture): go_live_publishes_and_sends_op4_sequence (self_stream:true only after publication, own key after it, every later op 4 streaming; devserver lists the stream), local_frames_reach_room_and_bus (rate ceiling, own tile removed on stop), watching_fake_streamer_puts_frames_on_bus, stream_audio_reaches_mixer_with_stream_gain (user volume 0 does not mute the stream; 50 % gives the store's curve), preview_uploaded_after_go_live, record_mode_never_contains_upload_url_or_token (no token, no presigned path, no upload_url/thumbnail, no JPEG bytes), capture_end_stops_share, debug_info_lists_capture_and_codecs. Unit tests for the frame gate, rule conversion, URL renewal, JPEG round trip/broken JPEG, privacy switch, video routing, stream/chime mapping, record redaction, --capture-fake.

FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci green locally (Linux). No real-device test is part of this task; no FLUXER_CAPTURE_TESTS run was needed (the backends are Screen share 4/5/14/15's).

Deviations

  • Outside the Files list, kept minimal: fluxer-voice/src/session.rs (RemoteAudio gains source: TrackSource, the only way to tell screen-share sound from the microphone), fluxer-audio/src/engine.rs (push_stream, SourceId::Stream in the level filter), fluxer-audio/tests/clips.rs (13 chimes), READMEs.
  • xtask/src/package/libs.rs: the app now links fluxer-capture, so the Linux binary needs libpipewire-0.3.so.0 and GLib (libglib/gobject/gio-2.0) at load time; they are added to LINUX_DECLARED (spec section 12 declares PipeWire) so cargo xtask package passes. AppRun, the release container and the README are follow-up #243 (Screen share 30).
  • A CaptureOutput::Error during a session is logged, not forwarded: the store ends the share on CaptureEvent::Error, while the capture contract says the session goes on (e.g. a failed switch) and Ended follows when it is over.
  • image (workspace dependency) added to apps/fluxer to decode fetched previews; fluxer-ui::run does not take the FrameBus yet, the UI tasks wire Handle::frame_bus().

Closes #168

🤖 Generated with Claude Code

Stacked on: screenshare-13-core-previews (screenshare-14-audio-capture-linux, screenshare-17-voice-livekit-video and screenshare-20-devserver-screenshare were already on screenshare at 2e06bfe). ## What this builds `apps/fluxer` now executes the screen-share half of the store: - **`capture_rt.rs`**: `Effect::Capture` runs on a worker thread that owns the `ScreenCapturer` and `StreamAudioCapturer` (the system backend is made on that thread, so PipeWire/X11 probing never holds up startup). Port types are converted both ways. Captured frames go to `VoiceHandle::push_video` and `FrameBus[LocalCapture]`, gated to the share's frame rate (`FrameGate`); stream audio (own PID tree excluded) to `push_screen_audio`; picker thumbnails to `FrameBus[Source { id }]`. `Started/Stalled/Resumed/Ended` become `CoreEvent::Capture`; a failed start becomes `Error`, a cancelled system picker `Ended { Cancelled }`. `CaptureBackend::{system, fake, unsupported, new}`, `Runtime::with_capture_backend`, `Handle::frame_bus()`. - **`preview_rt.rs`**: `UploadStreamPreview` encodes the latest `LocalCapture` frame with `preview_jpeg`, gets the presigned URL (cached in memory, renewed 60 s before expiry, dropped on a failed PUT) and PUTs it. The privacy switch is checked again just before the PUT and drops the cached URL. `FetchStreamPreview` decodes the JPEG into `FrameBus[Preview]` (404 removes the key). - **`voice_rt.rs`**: remote video of the screen shares the store subscribed to goes to `FrameBus[Remote { sid }]`, and is taken off when the store unsubscribes (late frames dropped), the track ends, the participant leaves or the room is left. Remote screen-share sound goes to the new mixer source `Stream(identity)`, so `SetGain { Stream }` (stream volume) now applies. - **`--capture-fake`** (feature `capture-fake`, on with `devserver`); `--devserver screenshare` uses the fake capture, and its fake room now plays every other member's stream from `VoiceScript::streams()` (test pattern at 15 fps and a 440 Hz stereo tone, only while subscribed). - **`--debug-info`** lists the capture backends (screen, stream audio, in-app picker, window audio), encodable codecs, hardware encoders and decodable codecs (`MediaInfo`). - **`--record`** also redacts `upload_url` and `thumbnail` (defence in depth; the preview requests are already unobserved in `fluxer-api`). - Chimes `ScreenShareStart/Stop` now play `stream-start.mp3` / `stream-stop.mp3` (copied from the fork's `fluxer_app/src/media/sounds/`). ## Tests `apps/fluxer/tests/screenshare_runtime.rs` (devserver `screenshare` + FakeRoom + NullHost + fake capture): `go_live_publishes_and_sends_op4_sequence` (self_stream:true only after publication, own key after it, every later op 4 streaming; devserver lists the stream), `local_frames_reach_room_and_bus` (rate ceiling, own tile removed on stop), `watching_fake_streamer_puts_frames_on_bus`, `stream_audio_reaches_mixer_with_stream_gain` (user volume 0 does not mute the stream; 50 % gives the store's curve), `preview_uploaded_after_go_live`, `record_mode_never_contains_upload_url_or_token` (no token, no presigned path, no `upload_url`/`thumbnail`, no JPEG bytes), `capture_end_stops_share`, `debug_info_lists_capture_and_codecs`. Unit tests for the frame gate, rule conversion, URL renewal, JPEG round trip/broken JPEG, privacy switch, video routing, stream/chime mapping, record redaction, `--capture-fake`. `FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci` green locally (Linux). No real-device test is part of this task; no `FLUXER_CAPTURE_TESTS` run was needed (the backends are Screen share 4/5/14/15's). ## Deviations - Outside the Files list, kept minimal: `fluxer-voice/src/session.rs` (`RemoteAudio` gains `source: TrackSource`, the only way to tell screen-share sound from the microphone), `fluxer-audio/src/engine.rs` (`push_stream`, `SourceId::Stream` in the level filter), `fluxer-audio/tests/clips.rs` (13 chimes), READMEs. - `xtask/src/package/libs.rs`: the app now links `fluxer-capture`, so the Linux binary needs `libpipewire-0.3.so.0` and GLib (`libglib/gobject/gio-2.0`) at load time; they are added to `LINUX_DECLARED` (spec section 12 declares PipeWire) so `cargo xtask package` passes. AppRun, the release container and the README are follow-up #243 (Screen share 30). - A `CaptureOutput::Error` during a session is logged, not forwarded: the store ends the share on `CaptureEvent::Error`, while the capture contract says the session goes on (e.g. a failed switch) and `Ended` follows when it is over. - `image` (workspace dependency) added to `apps/fluxer` to decode fetched previews; `fluxer-ui::run` does not take the `FrameBus` yet, the UI tasks wire `Handle::frame_bus()`. Closes #168 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Refs #168

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #168

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
Owner

Independent review: Screen share 21 (diff 16a3971..39786c5)

Tests (plan Step 1): all 8 named tests exist in apps/fluxer/tests/screenshare_runtime.rs and assert what the plan asks. go_live_publishes_and_sends_op4_sequence checks that no self_stream: true appears before the publication, then that the own key follows and later op 4s keep self_stream: true. local_frames_reach_room_and_bus checks the rate stays under the encode-fps ceiling. Stream gain is checked against stream_gain. The record test looks for the token, upload_url, thumbnail, stream-previews and JPEG magic. The unit tests also cover the 60 s URL renewal (540 s of a 600 s URL), the privacy switch dropping the cached URL, and broken JPEGs returning an error.

Interfaces: everything in Task 21's Produces list is present. Effect::Capture runs on a worker thread. Frames are gated and go to push_video and FrameBus[LocalCapture]. Stream audio goes to push_screen_audio, with our own PID excluded. Remote video goes to FrameBus[Remote { sid }] and remote stream audio to mixer SourceId::Stream(identity). Previews (upload URL cached until 60 s before expiry, then PUT) and FetchStreamPreview go to FrameBus[Preview]. Thumbnails go to FrameBus[Source { id }]. --capture-fake, --devserver and the fake backends are wired, and --debug-info has the MediaInfo section. The additive changes in other crates (RemoteAudio.source, AudioEngine::push_stream, SourceId::Stream, two new chimes) break no construction on other screenshare branches; I grepped 22, 25, 26 and 28.

Invariants: fluxer-core, fluxer-ui, fluxer-video and fluxer-capture are untouched. No LiveKit types appear in new public signatures. The upload URL stays a SecretString inside fluxer-api and is redacted in records. Server JPEGs are decoded without panicking. No secrets are in code or fixtures beyond the existing demo fixture token.

Review Focus 5: satisfied. The privacy switch is re-read before the upload-URL request and again before the PUT. Turning it on drops the cached URL. The record test shows no URL, token or JPEG reaches --record output.

Local CI: cargo xtask ci in a clean review worktree: green (all steps passed, screenshare_runtime 8/8).

Findings

  1. minor: preview_rt::decode_jpeg decodes server-supplied JPEGs with the image crate's default limits, which allow allocations of up to about 512 MiB. A hostile or corrupt preview with huge declared dimensions can cause a large allocation. It is not a panic. Consider image::ImageReader with explicit Limits sized to the preview budget (e.g. 4096×4096).
  2. minor: VideoRouting::participant_gone removes the participant's frames but keeps their identity in wanted. If they rejoin with a new share before the store unsubscribes, frames reach the bus without a new SetSubscribed. This is harmless today because the store re-issues subscriptions, but it is inconsistent with want(false).
  3. minor: CaptureRuntime::drop joins the worker thread. If the worker is blocked inside a backend call (portal list_sources/start waiting on the user), shutdown waits for it.
  4. minor: the Linux binary now has DT_NEEDED on libpipewire-0.3 and GLib. A system without them fails at load time, not with a message. This matches spec §12, which declares PipeWire as a runtime dependency. AppRun and the container are tracked in #243. Make sure #243 gives AppRun a readable check, like the libasound one.
  5. minor (noted, agreed): CaptureOutput::Error during a running session is only logged, not sent as CaptureEvent::Error. That is reasonable given the core ends the share on that event and the backend sends Ended. A short line in spikes or the spec would keep the next reader from treating it as a gap.

Verdict: APPROVED. No blocking findings and local CI is green; the remote runners are left to the integrator.

## Independent review: Screen share 21 (diff 16a3971..39786c5) **Tests (plan Step 1):** all 8 named tests exist in `apps/fluxer/tests/screenshare_runtime.rs` and assert what the plan asks. `go_live_publishes_and_sends_op4_sequence` checks that no `self_stream: true` appears before the publication, then that the own key follows and later op 4s keep `self_stream: true`. `local_frames_reach_room_and_bus` checks the rate stays under the encode-fps ceiling. Stream gain is checked against `stream_gain`. The record test looks for the token, `upload_url`, `thumbnail`, `stream-previews` and JPEG magic. The unit tests also cover the 60 s URL renewal (540 s of a 600 s URL), the privacy switch dropping the cached URL, and broken JPEGs returning an error. **Interfaces:** everything in Task 21's Produces list is present. `Effect::Capture` runs on a worker thread. Frames are gated and go to `push_video` and `FrameBus[LocalCapture]`. Stream audio goes to `push_screen_audio`, with our own PID excluded. Remote video goes to `FrameBus[Remote { sid }]` and remote stream audio to mixer `SourceId::Stream(identity)`. Previews (upload URL cached until 60 s before expiry, then PUT) and `FetchStreamPreview` go to `FrameBus[Preview]`. Thumbnails go to `FrameBus[Source { id }]`. `--capture-fake`, `--devserver` and the fake backends are wired, and `--debug-info` has the MediaInfo section. The additive changes in other crates (`RemoteAudio.source`, `AudioEngine::push_stream`, `SourceId::Stream`, two new chimes) break no construction on other screenshare branches; I grepped 22, 25, 26 and 28. **Invariants:** fluxer-core, fluxer-ui, fluxer-video and fluxer-capture are untouched. No LiveKit types appear in new public signatures. The upload URL stays a `SecretString` inside `fluxer-api` and is redacted in records. Server JPEGs are decoded without panicking. No secrets are in code or fixtures beyond the existing demo fixture token. **Review Focus 5:** satisfied. The privacy switch is re-read before the upload-URL request and again before the PUT. Turning it on drops the cached URL. The record test shows no URL, token or JPEG reaches `--record` output. **Local CI:** `cargo xtask ci` in a clean review worktree: **green** (`all steps passed`, screenshare_runtime 8/8). ### Findings 1. **minor**: `preview_rt::decode_jpeg` decodes server-supplied JPEGs with the `image` crate's default limits, which allow allocations of up to about 512 MiB. A hostile or corrupt preview with huge declared dimensions can cause a large allocation. It is not a panic. Consider `image::ImageReader` with explicit `Limits` sized to the preview budget (e.g. 4096×4096). 2. **minor**: `VideoRouting::participant_gone` removes the participant's frames but keeps their identity in `wanted`. If they rejoin with a new share before the store unsubscribes, frames reach the bus without a new `SetSubscribed`. This is harmless today because the store re-issues subscriptions, but it is inconsistent with `want(false)`. 3. **minor**: `CaptureRuntime::drop` joins the worker thread. If the worker is blocked inside a backend call (portal `list_sources`/`start` waiting on the user), shutdown waits for it. 4. **minor**: the Linux binary now has DT_NEEDED on libpipewire-0.3 and GLib. A system without them fails at load time, not with a message. This matches spec §12, which declares PipeWire as a runtime dependency. AppRun and the container are tracked in #243. Make sure #243 gives AppRun a readable check, like the libasound one. 5. **minor (noted, agreed)**: `CaptureOutput::Error` during a running session is only logged, not sent as `CaptureEvent::Error`. That is reasonable given the core ends the share on that event and the backend sends `Ended`. A short line in spikes or the spec would keep the next reader from treating it as a gap. **Verdict: APPROVED.** No blocking findings and local CI is green; the remote runners are left to the integrator.
charles manually merged commit e4e18fb99a into screenshare 2026-11-12 10:46:00 +00:00
Sign in to join this conversation.
No description provided.