Screen share: copy Wayland frames before libwebrtc overwrites them (#589) #607

Merged
charles merged 1 commit from fix/589-stream-top-strip into textchat 2026-11-15 07:37:52 +00:00
Owner

Refs #589. This deliberately says Refs, not Closes: the "part of another frame" strip is found and fixed on the Linux/Wayland sender, but the black strip was not reproduced (see "Not covered"). Close #589 once FLUXER_FRAME_CHECK=1 confirms it on the real machines.

Root cause (measured)

On Wayland, libwebrtc's PipeWire capturer hands fluxer-capture a share of one of the two frames in its stream's queue, and its PipeWire thread writes a new picture into that same frame two buffers later whether or not the frame is still being read. It only logs a debug warning (shared_screencast_stream.cc at the pinned revision webrtc-a545de4: CaptureFrame returns latest_available_frame_->Share() with no copy, l. 698; ProcessBuffer runs queue_.MoveToNextFrame() and then logs "Overwriting frame that is still shared" but keeps writing, l. 915–917; the copy goes into that frame at l. 1040 (MemFd) and l. 1076 (DMA-BUF)). kQueueLength is 2 (screen_capture_frame_queue.h).

Our callback (crates/fluxer-capture/src/desktop/rtc.rs, RtcGrabber::start) converted frame.data() straight to I420 with libyuv. A 4K frame takes long enough to convert that sometimes the PipeWire thread starts rewriting it partway through. The writer starts at the top and is faster than the conversion, so it overtakes the read position. The result is a frame made of two pictures two buffers apart, with the boundary somewhere down the frame. That frame then gets encoded, so every viewer sees it, Electron included (Chromium uses the same capturer). It only lasts one frame, which is why a screenshot almost never catches it.

Mixed frames seen in the test, bands top to bottom, digit = palette colour of the frame (7 colours, frame n = colour n mod 7): 6666611111, 333?555555, 4444444446. The two colours are always two frames apart, which is the signature of the 2-frame queue.

Fix

rtc.rs now copies the BGRA pixels into a buffer that is reused from frame to frame, and compares the copy with the frame (desktop::stable_copy). If anything changed during the copy it pulls the frame again (pull_stable, at most 3 retries, then the tick is skipped), and it converts only the copy. On X11 and Windows (WGC/DXGI fill the frame inside capture_frame() on our thread, wgc_capture_session.cc GetFrame → ProcessFrame), the check never fires. It only costs the copy there.

Cost: one memcpy plus one memcmp of the BGRA frame per captured frame. At 4K→4K@60 on this loaded machine (load average about 50 on 32 threads) the session delivered 749 frames in 20 s, against 973 before.

Measurements

Capture side, headless: tools/capture-consistency/run.sh

This uses a private D-Bus, a private PipeWire daemon, a stand-in ScreenCast portal and a pw_stream source (examples/solid_source.rs) whose frame n is one solid colour. The real Wayland path runs end to end (libwebrtc portal capturer, PipeWire stream, our conversion), and every frame is checked by tests/portal_consistency.rs. Nothing touches the desktop. Each run is 20 s with the source at 60 fps (libwebrtc asks for at most 60).

source → target before: frames / mixed / black top after: frames / mixed / black top (torn pulls retried)
1080p → 1080p@30 601 / 0 / 0 601 / 0 / 0
1080p → 1080p@60 1201 / 0 / 0 1201 / 0 / 0
1080p → 1080p@60, 6 busy cores 1201 / 0 / 0 1201 / 0 / 0
4K → 1080p@30 601 / 6 / 0 601 / 0 / 0
4K → 1080p@60 1164 / 13 / 0 988 / 0 / 0 (48)
4K → 4K@60 973 / 20 / 0 749 / 0 / 0 (115)
4K → 1080p@60, 6 busy cores 1088 / 36 / 0 963 / 0 / 0 (67)

Total: 75 mixed frames in 6,829 before, and 0 in 6,304 after.

The first version of the source used GStreamer's pipewiresink. An independent pw_stream consumer showed that pipewiresink hands over buffers whose pixels keep changing for a few milliseconds: 33 of 467 buffers were mixed if read at once, and 0 of 294 if read 30 ms later. That source was dropped, and the table uses the pw_stream source, for which the same consumer saw 0 of 579 mixed at 4K.

Receive side: livekit_interop::received_screen_share_frames_are_whole_pictures

The test publishes a SolidSequence through LiveKit and checks every frame the viewer's session delivers (what goes onto the FrameBus). It ran with FLUXER_MEASURE=1, 10 s per case, for AV1, H.264, VP9 and VP8, at 1080p30, 1080p60 and 1080p30 switching to 720p mid-stream. The four configurations were NVDEC on and LK_DISABLE_NVDEC=1, each with and without 6 busy cores. Result: 56 runs, about 18,900 frames, 0 inconsistent, 0 black top. The receive path is clean.

Ownership map (audit)

  • Capture: libwebrtc owns the BGRA DesktopFrame (the 2-slot queue). It is written on the PipeWire thread and read in our callback: the race. Our I420 frame is a new Vec per frame, which becomes Arc<[u8]>.
  • Capture → bus and encoder (apps/fluxer/src/capture_rt.rs hand_on): immutable Arc planes are shared. fluxer-voice to_rtc_buffer copies them into a new I420Buffer per frame.
  • Receive (rtc.rs read_video): rtc_to_i420 produces an I420 buffer that we hold by reference. Pooled decoder buffers, including NVDEC's buffer_pool_, are reused only once nobody references them. from_rtc_i420 copies the buffer into new Arc planes, which go onto the FrameBus (two slots, never mutated, replaced by Arc swap).
  • Widget (fluxer-ui widgets/video.rs): prepare converts to I420 and writes all three planes with queue.write_texture, which copies the data at call time, before the render pass of the same submission. Textures are recreated only on a size change and are written in that same prepare, so no uninitialised texture is ever drawn.

Debug switch

FLUXER_FRAME_CHECK=1 logs at warn each frame whose top fifth is black while the rest is not, and totals every 600 frames after the first one (fluxer_video::check::FrameCheck). It covers both the captured frames (side = "captured", before encoding) and the decoded frames (side = "received", per track sid), so a real share shows which side produces it.

Not covered here

  • The black strip was not reproduced. No configuration produced a black top strip. The mixed frames reproduce "part of another frame". The DMA-BUF path, which KWin normally uses, cannot be exercised without a compositor. The audit shows libwebrtc imports the DMA-BUF and calls glReadPixels with no explicit sync: it negotiates no SPA_META_SyncTimeline (egl_dmabuf.cc ImageFromDmaBuf). On a driver without implicit sync this could read a buffer the compositor has not finished, which would be a separate, unfixed cause. Running the debug switch on the real machines should tell.
  • The real compositor (KWin), the sender's OS (not stated in the issue) and a Windows sender. Windows was audited only.
  • The widget's GPU upload was audited, not tested headlessly.

Gate

FLUXER_SCREENSHOT_TESTS=1 FLUXER_LIVEKIT_TESTS=1 cargo xtask ci: the first run failed only on the known flake watching_a_real_share_puts_its_frames_on_the_bus (#573), and the rerun passed every step ("all steps passed"). The new received_screen_share_frames_are_whole_pictures passed in both runs. tools/capture-consistency/run.sh 3840 2160 60 --target 3840x2160@60 --expect-clean passes on this branch (897 frames, 0 mixed); without the fix the same run fails (20 of 973 mixed).

🤖 Generated with Claude Code

Refs #589. This deliberately says Refs, not Closes: the "part of another frame" strip is found and fixed on the Linux/Wayland sender, but the black strip was not reproduced (see "Not covered"). Close #589 once `FLUXER_FRAME_CHECK=1` confirms it on the real machines. ## Root cause (measured) On Wayland, libwebrtc's PipeWire capturer hands `fluxer-capture` a *share* of one of the two frames in its stream's queue, and its PipeWire thread writes a new picture into that same frame two buffers later **whether or not the frame is still being read**. It only logs a debug warning (`shared_screencast_stream.cc` at the pinned revision `webrtc-a545de4`: `CaptureFrame` returns `latest_available_frame_->Share()` with no copy, l. 698; `ProcessBuffer` runs `queue_.MoveToNextFrame()` and then logs "Overwriting frame that is still shared" but keeps writing, l. 915–917; the copy goes into that frame at l. 1040 (MemFd) and l. 1076 (DMA-BUF)). `kQueueLength` is 2 (`screen_capture_frame_queue.h`). Our callback (`crates/fluxer-capture/src/desktop/rtc.rs`, `RtcGrabber::start`) converted `frame.data()` straight to I420 with libyuv. A 4K frame takes long enough to convert that sometimes the PipeWire thread starts rewriting it partway through. The writer starts at the top and is faster than the conversion, so it overtakes the read position. The result is a frame made of two pictures two buffers apart, with the boundary somewhere down the frame. That frame then gets encoded, so every viewer sees it, Electron included (Chromium uses the same capturer). It only lasts one frame, which is why a screenshot almost never catches it. Mixed frames seen in the test, bands top to bottom, digit = palette colour of the frame (7 colours, frame n = colour n mod 7): `6666611111`, `333?555555`, `4444444446`. The two colours are always two frames apart, which is the signature of the 2-frame queue. ## Fix `rtc.rs` now copies the BGRA pixels into a buffer that is reused from frame to frame, and compares the copy with the frame (`desktop::stable_copy`). If anything changed during the copy it pulls the frame again (`pull_stable`, at most 3 retries, then the tick is skipped), and it converts only the copy. On X11 and Windows (WGC/DXGI fill the frame inside `capture_frame()` on our thread, `wgc_capture_session.cc` `GetFrame` → `ProcessFrame`), the check never fires. It only costs the copy there. Cost: one memcpy plus one memcmp of the BGRA frame per captured frame. At 4K→4K@60 on this loaded machine (load average about 50 on 32 threads) the session delivered 749 frames in 20 s, against 973 before. ## Measurements ### Capture side, headless: `tools/capture-consistency/run.sh` This uses a private D-Bus, a private PipeWire daemon, a stand-in ScreenCast portal and a `pw_stream` source (`examples/solid_source.rs`) whose frame n is one solid colour. The real Wayland path runs end to end (libwebrtc portal capturer, PipeWire stream, our conversion), and every frame is checked by `tests/portal_consistency.rs`. Nothing touches the desktop. Each run is 20 s with the source at 60 fps (libwebrtc asks for at most 60). | source → target | before: frames / mixed / black top | after: frames / mixed / black top (torn pulls retried) | |---|---|---| | 1080p → 1080p@30 | 601 / 0 / 0 | 601 / 0 / 0 | | 1080p → 1080p@60 | 1201 / 0 / 0 | 1201 / 0 / 0 | | 1080p → 1080p@60, 6 busy cores | 1201 / 0 / 0 | 1201 / 0 / 0 | | 4K → 1080p@30 | 601 / **6** / 0 | 601 / 0 / 0 | | 4K → 1080p@60 | 1164 / **13** / 0 | 988 / 0 / 0 (48) | | 4K → 4K@60 | 973 / **20** / 0 | 749 / 0 / 0 (115) | | 4K → 1080p@60, 6 busy cores | 1088 / **36** / 0 | 963 / 0 / 0 (67) | Total: 75 mixed frames in 6,829 before, and 0 in 6,304 after. The first version of the source used GStreamer's `pipewiresink`. An independent `pw_stream` consumer showed that `pipewiresink` hands over buffers whose pixels keep changing for a few milliseconds: 33 of 467 buffers were mixed if read at once, and 0 of 294 if read 30 ms later. That source was dropped, and the table uses the `pw_stream` source, for which the same consumer saw 0 of 579 mixed at 4K. ### Receive side: `livekit_interop::received_screen_share_frames_are_whole_pictures` The test publishes a `SolidSequence` through LiveKit and checks every frame the viewer's session delivers (what goes onto the `FrameBus`). It ran with `FLUXER_MEASURE=1`, 10 s per case, for AV1, H.264, VP9 and VP8, at 1080p30, 1080p60 and 1080p30 switching to 720p mid-stream. The four configurations were NVDEC on and `LK_DISABLE_NVDEC=1`, each with and without 6 busy cores. Result: 56 runs, about 18,900 frames, **0 inconsistent, 0 black top**. The receive path is clean. ## Ownership map (audit) - **Capture:** libwebrtc owns the BGRA `DesktopFrame` (the 2-slot queue). It is written on the PipeWire thread and read in our callback: **the race**. Our I420 frame is a new `Vec` per frame, which becomes `Arc<[u8]>`. - **Capture → bus and encoder** (`apps/fluxer/src/capture_rt.rs` `hand_on`): immutable `Arc` planes are shared. `fluxer-voice` `to_rtc_buffer` copies them into a new `I420Buffer` per frame. - **Receive** (`rtc.rs` `read_video`): `rtc_to_i420` produces an I420 buffer that we hold by reference. Pooled decoder buffers, including NVDEC's `buffer_pool_`, are reused only once nobody references them. `from_rtc_i420` copies the buffer into new `Arc` planes, which go onto the `FrameBus` (two slots, never mutated, replaced by `Arc` swap). - **Widget** (`fluxer-ui` `widgets/video.rs`): `prepare` converts to I420 and writes all three planes with `queue.write_texture`, which copies the data at call time, before the render pass of the same submission. Textures are recreated only on a size change and are written in that same `prepare`, so no uninitialised texture is ever drawn. ## Debug switch `FLUXER_FRAME_CHECK=1` logs at `warn` each frame whose top fifth is black while the rest is not, and totals every 600 frames after the first one (`fluxer_video::check::FrameCheck`). It covers both the captured frames (`side = "captured"`, before encoding) and the decoded frames (`side = "received"`, per track sid), so a real share shows which side produces it. ## Not covered here - **The black strip was not reproduced.** No configuration produced a black top strip. The mixed frames reproduce "part of another frame". The DMA-BUF path, which KWin normally uses, cannot be exercised without a compositor. The audit shows libwebrtc imports the DMA-BUF and calls `glReadPixels` with no explicit sync: it negotiates no `SPA_META_SyncTimeline` (`egl_dmabuf.cc` `ImageFromDmaBuf`). On a driver without implicit sync this could read a buffer the compositor has not finished, which would be a separate, unfixed cause. Running the debug switch on the real machines should tell. - The real compositor (KWin), the sender's OS (not stated in the issue) and a Windows sender. Windows was audited only. - The widget's GPU upload was audited, not tested headlessly. ## Gate `FLUXER_SCREENSHOT_TESTS=1 FLUXER_LIVEKIT_TESTS=1 cargo xtask ci`: the first run failed only on the known flake `watching_a_real_share_puts_its_frames_on_the_bus` (#573), and the rerun passed every step ("all steps passed"). The new `received_screen_share_frames_are_whole_pictures` passed in both runs. `tools/capture-consistency/run.sh 3840 2160 60 --target 3840x2160@60 --expect-clean` passes on this branch (897 frames, 0 mixed); without the fix the same run fails (20 of 973 mixed). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
libwebrtc's PipeWire capturer hands over a share of one of the two frames
of its stream queue, and its PipeWire thread writes the picture two
buffers later into that same frame while it is still shared (it only logs
"Overwriting frame that is still shared"). The capture callback converted
that frame in place; a 4K frame took long enough that the writer overtook
the read position, so the encoded frame was the top of one picture over the
rest of another: the strip at the top of the share that viewers saw.

The callback now copies the pixels into a reused buffer, compares the copy
with the frame and pulls again when it changed (at most 3 retries), and
converts the copy.

Measured headless (tools/capture-consistency/run.sh: private D-Bus and
PipeWire, a stand-in ScreenCast portal, a solid-colour-per-frame source;
the real libwebrtc Wayland path): 4K sources had 6/601, 13/1164, 20/973
and 36/1088 mixed frames before, 0 of 6,304 after; 1080p had none. The
receive side (every codec, NVDEC on and off, 30/60 fps, size change, CPU
load; ~18,900 frames) had none either.

Also:
- fluxer-video `check`: SolidSequence, check_sequence_frame,
  top_strip_black and FrameCheck.
- FLUXER_FRAME_CHECK=1 logs frames whose top fifth is black while the rest
  is not, on capture (before encoding) and on receive (after decoding).
- livekit_interop received_screen_share_frames_are_whole_pictures (in CI
  with FLUXER_LIVEKIT_TESTS=1; FLUXER_MEASURE=1 for the matrix).
- tests/portal_consistency.rs + examples/solid_source.rs + the harness.

Refs #589

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
charles merged commit 5d1965ed27 into textchat 2026-11-15 07:37:52 +00:00
Sign in to join this conversation.
No description provided.