Screen share 5: Windows screen capture backend (#152) #202
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "screenshare-5-capture-windows"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What
DesktopScreenCapturerfor Windows (spec §7.1) on libwebrtc'sDesktopCapturer, incrates/fluxer-capture/src/desktop.rs:list_sources: screens ("Screen 1", ..., idscreen:<n>) then other processes' windows (idwindow:<HWND>) with titles, owningpid(GetWindowThreadProcessId) andapp_name(executable name). Untitled windows and this process's own windows are left out. Thumbnails: one frame per source, at most 480x270, four sources at a time.Pacer), crop to even dimensions, area downscale to the target, BGRA to I420 (fluxer-video),Startedwith the first frame,Stalled/Resumed(StallDetector), permanent error =Ended(SourceClosed),switch/updateon the fly, synchronousstarterrors, stop/drop drop the sink.default_backends()returns it on Windows (orUnsupportedwith the reason when libwebrtc cannot create a screen capturer).capabilities(): in-app picker yes,window_audiofalse (Screen share 15's call).docs/superpowers/research/2026-11-11-screenshare-spikes.mdsection 3.How it was tested
desktop.rs: pacing, size/even crop, conversion, Started-first, Ended once, error-then-Ended, switch, update, sync start errors, thumbnails, drop).FLUXER_SCREENSHOT_TESTS=1 cargo xtask cigreen on Linux.cfgcode was type-checked and linted on Linux:cargo clippy --target x86_64-pc-windows-msvc -p fluxer-capture --all-targets -- -D warnings(withDOCS_RS=1and stub C compilers, so not linked). The Windows runner is its first real build.tests/windows_real.rs:server_core_reports_unsupported_or_lists_nothing_without_panickingruns on Windows CI.screen_capture_delivers_frames,window_list_has_titles_and_pids_and_excludes_self,closing_captured_window_reports_endedare#[ignore]+FLUXER_CAPTURE_TESTS=1: not run, no Windows desktop was available to the implementer.Deviations
desktop.rsholds the Windows part plus an OS-neutral session machine (Grabber/Opener/Lister) that Task 4 can reuse; the module is#[cfg(any(windows, test))]. Integrator merge is mechanical.src/lib.rs(module, export,default_backends) andtests/fake.rs(unsupported_backends_return_errorsnow tests theUnsupportedstub directly, since Windows has a real screen backend).libwebrtc0.4.0 (already in the graph vialivekit) andwindows0.62 (already in the graph; featuresWin32_Foundation,Win32_System_Threading,Win32_UI_WindowsAndMessaging). Both added to the workspace; Task 4 and 15 extend them.Stacked on: screenshare-3-capture-crate
Closes #152
🤖 Generated with Claude Code
Independent review: Screen share 5 (Windows screen capture), commit
8fd5f3bScope:
git diff d821f19..origin/screenshare-5-capture-windows. Local gate:FLUXER_SCREENSHOT_TESTS=1 cargo xtask ciin a fresh worktree FAILED at theteststep (details in finding 1). fmt, clippy, deny, core isolation and parity passed before it.Blocking
crates/fluxer-capture/src/desktop.rs:start()right afterEndedcan returnAlreadyStarted;closed_window_reports_ended_onceis flaky and turned CI red.capture_loopdrops the sink when it returns, soEndedarrives and the receiver disconnects before the OS thread has exited.reap()checksJoinHandle::is_finished(), which is still false in that window, so the nextstart()sees the old session and returnsCaptureError::AlreadyStarted. Local result:panicked at desktop.rs:887: a new session after Ended: AlreadyStarted; reran that test 20 times and it failed 2 of them. This is also a real contract bug, not only a test race: the runtime (Screen share 21) restarts as soon as it getsEnded(a window closed, then the user picks another source), and it would get a spuriousAlreadyStarted. Fix suggestion: have the session record that it has finished before the final sink call/drop (e.g. anArc<AtomicBool>set at the end ofcapture_loop, or a "done" message) and makereap()join when that flag is set; or havestart()join an existing session whose command channel is disconnected. Then make the test deterministic.Minor
tests/windows_real.rsserver_core_reports_unsupported_or_lists_nothing_without_panicking: whennew()succeeds, the test expectsstart(Source("window:0"))to returnSourceNotFound. Butwin::openfirst creates a window capturer and returnsCaptureError::Backendif libwebrtc can't create one. On Server Core the screen probe could succeed while the window capturer fails, and this test would then fail on the Windows runner. Consider acceptingSourceNotFound | Backend | Unsupportedthere, or returningSourceNotFoundfor an unparseable or unknown id before the capturer is created. The Windows runner's result for this PR decides this one. I did not poll CI.source_that_never_delivers_reports_error_then_endeduseswindow:3, which returnsPermanentstraight away, so it covers the "fails before first frame" branch and not theTemporary-for-10 s branch (FIRST_FRAME_TIMEOUT). There is also no unit test ofStalled/Resumedthrough this session machine (the fake covers the contract). The constant ispub(crate), so the tests could inject a short timeout.pacing_holds_target_fps(16..=22 frames in a wall-clock second) depends on timing and could flake on a loaded runner. It passed here.#[ignore]"), but it should become afollow-upissue (run the real tests on Windows 10/11, settle the border, minimise and CPU questions) linked from the PR before merge.Checked and OK
server_core_reports_unsupported_or_lists_nothing_without_panicking, and realscreen_capture_delivers_frames,window_list_has_titles_and_pids_and_excludes_self,closing_captured_window_reports_endedall exist,#[ignore]+FLUXER_CAPTURE_TESTS=1, and assert what the plan says. Thumbnails are ≤ 480×270 (THUMBNAIL_WIDTH/HEIGHT).DesktopScreenCapturer: ScreenCapturerexported on Windows; WGC (DXGI/GDI picked by libwebrtc);Screen/Windowlists with titles, ownerpid,app_name; own process's windows left out (list andopen);set_include_cursor(true);default_backends()returns it on Windows and falls back toUnsupportedwithout panicking.fluxer-capturegainslibwebrtc0.4.0 (single version inCargo.lock, the onelivekitpins) andwindows0.62, Windows-only. Nofluxer-core/livekitdep. libwebrtc types are confined todesktop.rs::winand appear in no public signature. Frame copy validates width/height/stride/length, so a bad frame isTemporaryand never a panic. No secrets.cfg(any(windows, test))module + the Task 4 merge note: mechanical, as the plan allows.Verdict: changes requested. One blocking finding (1): the reap race makes CI flaky and gives a real
AlreadyStartedafterEnded.Review round 1 addressed in
9214c92:desktop.rs): fixed. The capture thread sets anended: AtomicBool(Release) before its finalEndedsink call;reap()treats a session as over when that flag is set or the thread is finished, and joins it (which only waits for the sink call to return and the sink to drop). New teststart_right_after_ended_is_not_already_startedreproduces the race deterministically (a sink whose drop sleeps 200 ms); it failed before the fix and passes after. 15 back-to-back reruns of thedesktoptests were green.window:0expectation (windows_real.rs): the test now acceptsSourceNotFound,BackendorUnsupportedthere, with the reason in a comment.Timeouts { first_frame, stall }(defaultsFIRST_FRAME_TIMEOUTandSTALL_AFTER; the tests shorten them). New testssource_that_never_starts_times_out_with_error_then_ended(a source that only returns Temporary) andstalled_source_reports_stalled_then_resumed_before_its_next_frame(checks Started, Frame, Stalled, Resumed, Frame in that order).pacing_holds_target_fpsnow measures the elapsed time and asserts at mostelapsed x 20 + 2frames (pacing never overshoots) and at least half ofelapsed x 20(a loaded runner may deliver fewer).FLUXER_SCREENSHOT_TESTS=1 cargo xtask ciis green locally. The Windowscfgcode was re-checked withcargo clippy --target x86_64-pc-windows-msvc -p fluxer-capture --all-targets -- -D warnings, which passes.Independent review: Screen share 5 (Windows capture), head
9214c92Scope:
git diff d821f19..origin/screenshare-5-capture-windows. I read the code without relying on the implementer's notes.Plan tests. All four named tests are in
tests/windows_real.rs.server_core_reports_unsupported_or_lists_nothing_without_panickingruns on Windows CI. The three real-device tests are#[ignore]and also needFLUXER_CAPTURE_TESTS=1, which the plan allows because no Windows desktop was available. They assert what the plan asks: titles, pids, exclusion of this process,Ended(SourceClosed)when the window closes, non-black frames, and thumbnails no larger than 480x270. The unit tests of the session machine cover the spec values:STALL_AFTER= 3 s, thumbnails at 480x270, and cursor on.Interfaces. These match the plan:
DesktopScreenCapturer: ScreenCaptureris public, withnew() -> Result<Self, CaptureError>on Windows.default_backends()returns it on Windows and falls back toUnsupportedwhen it cannot be created.Screen/Windowlists carry the title, the owningpidandapp_name. This process's windows are filtered out by the SDK and again by pid.in_app_picker: true.Invariants.
fluxer-capturedepends on neitherfluxer-corenorlivekit.libwebrtcandwindowsare Windows-only target dependencies, and their types stay insrc/desktop.rs(privatewinmodule).u32).Review Focus. None of the items belongs to Task 5.
The reap() race fix. Verified.
endedis stored (Release) before the finalEndedsink call, andreap()loads it with Acquire.start_right_after_ended_is_not_already_startedcovers it with a sink that is slow to drop.Local CI.
FLUXER_SCREENSHOT_TESTS=1 cargo xtask cipassed: "all steps passed", and the 16desktop::testspassed. The first run failed with a transient cargo dep-info I/O error ("could not parse/generate dep info … tempfile-*.d"). It is unrelated to this change, and a clean rerun was green.Findings
capture_loop,Grab::Framearm. Whenfit_to_targetreturnsNone(for example a 1x1 window, or a frame shorter than its stride claims), the loop doescontinuebefore the first-frame timeout and the stall poll. A source that only ever delivers such frames never sendsStarted,ErrororEnded; the session stays silent untilstop. Suggested fix: treat a frame that cannot be used likeGrab::Temporary, so it falls through to the timeout and stall logic.cfgcode has only been type-checked (clippy with stub C compilers). It has not been linked or run anywhere yet. The Windows runner is its first real build, and the real-device facts in spikes section 3 are unverified. #204 tracks this, which is right; the integrator should confirm that the Windows runner is green before merging.docs/superpowers/research/2026-11-11-screenshare-spikes.mdis created as a new file here, and Screen share 4 (section 2) will most likely create it too. That is a mechanical merge conflict, so keep the sections in number order when merging.Verdict: APPROVED (no blocking findings; local CI green).