Camera 11: fluxer-capture camera on Windows (Media Foundation) (#963) #984

Closed
charles wants to merge 0 commits from camera-11-capture-camera-on-windows into camera
Owner

What this builds

fluxer_capture::camera::media_foundation::MfCamera, the Windows CameraCapturer over the Media Foundation source reader (Camera 2's choice, spikes §2), and default_camera() returning it on Windows where Media Foundation starts (an N edition without the Media Feature Pack gets Unsupported with the reason).

  • Split for testing (as Task 10): MfCamera<Io: MfIo> holds all the logic; MfIo / MfSource are the calls it needs (enumerate, COM on a thread, activate + reader, native types, select, NV12, output type, read a sample with its locked buffer, interrupt, register notifications). WindowsIo (media_foundation/win.rs, Windows only) makes the real calls; memory::MemoryIo (tests and feature fake) replays scripted devices, failures at any MfStep and reads, and counts COM threads, sources, locks and registrations alive.
  • list: MFEnumDeviceSources (VIDCAP: capture cards list too), names through names::display, id = symbolic link. Activates nothing (list_activates_nothing).
  • watch: CM_Register_Notification for KSCATEGORY_VIDEO_CAMERA and KSCATEGORY_CAPTURE. This is the windowless alternative that spikes §2 allows, and it is simpler than a message-only window. A burst of notifications calls on_change once, 250 ms after the last one.
  • modes: activates on its own thread, reads every native type (any size, aspect ratio or rate), and shuts down.
  • start: forbidden() first. Then, on the session thread: COM MTA + MFStartup; the device by link (case-insensitive; Default = first). The device is refused with Busy when another MfCamera of this process has it open. Then ActivateObject, a reader with advanced video processing, only the first video stream selected, the native type from format::choose, and NV12 asked for. If the reader refuses NV12, it re-selects the native type and decodes it with camera::decode (the spike's fallback). start returns after the first read, so E_ACCESSDENIED at the first read is still NotAllowed (the docs leave open which call returns it). A camera silent for 3 s starts anyway and the stall watchdog takes over. Samples are paced by their time stamps. They are converted by frame_from_buffer: the stride comes from IMF2DBuffer2::Lock2DSize (a negative pitch means bottom-up and is flipped), else from MF_MT_DEFAULT_STRIDE. Then they are fitted and never mirrored.
  • Endings: a code the table calls gone, MF_SOURCE_READERF_ERROR and end of stream all give Ended(DeviceLost). Any other read error (preempted: "in use") gives Error(text) and then Ended(DeviceLost). The source is released before Ended is sent.
  • stop: sets the flag and calls IMFMediaSource::Shutdown from the stopping thread, which unblocks a ReadSample. It then joins the threads: the session thread drops the interrupter, the reader and the source (Shutdown, ShutdownObject), releases the claim, and calls MFShutdown and CoUninitialize. After that it drops the sink and returns. Drop stops.
  • No frame bytes in logs: name, mode, sizes, counts, and the id as id_hash.

How it was tested

  • tests/camera_mf.rs (every OS, opens nothing): the plan's list_activates_nothing, nv12_with_stride_and_negative_height, access_denied_is_not_allowed (at activation and at the first read), preempted_is_busy (at start and mid-stream), removed_is_device_lost (also the error flag and end of stream; Ended arrives with 0 sources alive), stop_shuts_source_down_before_returning (a blocked read, the sink dropped before stop returns, Drop), forbidden_env_refuses_before_activation, com_wrappers_release_on_every_path (a failure at each step, NV12 refused, unknown read error, no usable type: everything back to 0, activations == shutdowns). Also: mode choice, fit (1080p60 capture card asked for 640×480 → 640×360), NV12 fallback, pacing 60 → 30 fps, stall once then resume, a silent camera, NotFound / AlreadyStarted / second consumer Busy, modes, and watch debounce and replacement.
  • The Windows code was type-checked and linted for x86_64-pc-windows-msvc on Linux. This used cargo clippy --target x86_64-pc-windows-msvc --all-targets -- -D warnings in a scratch crate that mounts this camera module (and screen.rs) against windows 0.62.2. It is clean, both with and without fake. A deliberate type error in win.rs was reported, which shows the block is really checked. The Windows runner checks it in the workspace after merge.
  • Local gate: FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci, all steps passed. It was run with TMPDIR set to a directory under the worktree's target dir, because /tmp was full of leaked /tmp/appimage_extracted_* directories (#983). No test needing a real display or device was run, and nothing was run with FLUXER_DESKTOP_TESTS or FLUXER_CAMERA_TESTS.

Deviations from the plan

  • The real-device test is tests/camera_real_win.rs (Windows only, #[ignore], FLUXER_CAMERA_TESTS=1) instead of a Windows case in tests/camera_real.rs. Task 10 creates that file in the same wave, and the change avoids an add/add conflict. The two can be merged later.
  • media_foundation.rs has two child files, media_foundation/win.rs (the real calls) and media_foundation/memory.rs (the in-memory I/O).
  • One added feature of the existing windows crate: Win32_Devices_DeviceAndDriverInstallation, for CM_Register_Notification. No new crate.
  • camera/mod.rs: default_camera() now dispatches to a platform_camera() per OS. Task 10 adds its Linux arm there. This is a registration file: keep both sides.

What Charles must check on Windows (this code has never run)

With a camera, set FLUXER_CAMERA_TESTS=1 then cargo test -p fluxer-capture --test camera_real_win -- --ignored --nocapture, and in the app once Task 12 lands:

  1. list does not turn the light on. Names are right and capture cards are listed.
  2. A picture arrives within 5 s, the right way up and in true colours: an MJPEG-only webcam (does NV12 conversion work, or does the fallback decode run?) and a YUY2 one.
  3. The light goes off within a second of stop. A covered or silent camera still stops promptly (the Shutdown from the stopping thread must unblock ReadSample).
  4. The Windows Camera app holding the camera gives "in use" (Busy), at start or mid-stream.
  5. Settings → Privacy → Camera off gives "not allowed" (NotAllowed), at activation or at the first read.
  6. Unplugging during capture gives Ended(DeviceLost) and the light goes off. Plugging in or unplugging a camera updates the list (watch).
  7. A capture card: its modes are all listed, and a 1080p60 mode can be selected.
  8. On an N edition without the Media Feature Pack, the camera button is disabled and its tooltip gives the reason.

Closes #963

🤖 Generated with Claude Code

https://claude.ai/code/session_0161qMQMsCyJhjYw4gqa6e4K

## What this builds `fluxer_capture::camera::media_foundation::MfCamera`, the Windows `CameraCapturer` over the Media Foundation source reader (Camera 2's choice, *spikes §2*), and `default_camera()` returning it on Windows where Media Foundation starts (an N edition without the Media Feature Pack gets `Unsupported` with the reason). - **Split for testing (as Task 10):** `MfCamera<Io: MfIo>` holds all the logic; `MfIo` / `MfSource` are the calls it needs (enumerate, COM on a thread, activate + reader, native types, select, NV12, output type, read a sample with its locked buffer, interrupt, register notifications). `WindowsIo` (`media_foundation/win.rs`, Windows only) makes the real calls; `memory::MemoryIo` (tests and feature `fake`) replays scripted devices, failures at any `MfStep` and reads, and counts COM threads, sources, locks and registrations alive. - **list:** `MFEnumDeviceSources` (VIDCAP: capture cards list too), names through `names::display`, id = symbolic link. Activates nothing (`list_activates_nothing`). - **watch:** `CM_Register_Notification` for `KSCATEGORY_VIDEO_CAMERA` and `KSCATEGORY_CAPTURE`. This is the windowless alternative that *spikes §2* allows, and it is simpler than a message-only window. A burst of notifications calls `on_change` once, 250 ms after the last one. - **modes:** activates on its own thread, reads every native type (any size, aspect ratio or rate), and shuts down. - **start:** `forbidden()` first. Then, on the session thread: COM MTA + `MFStartup`; the device by link (case-insensitive; `Default` = first). The device is refused with `Busy` when another `MfCamera` of this process has it open. Then `ActivateObject`, a reader with advanced video processing, only the first video stream selected, the native type from `format::choose`, and NV12 asked for. If the reader refuses NV12, it re-selects the native type and decodes it with `camera::decode` (the spike's fallback). `start` returns after the first read, so `E_ACCESSDENIED` at the first read is still `NotAllowed` (the docs leave open which call returns it). A camera silent for 3 s starts anyway and the stall watchdog takes over. Samples are paced by their time stamps. They are converted by `frame_from_buffer`: the stride comes from `IMF2DBuffer2::Lock2DSize` (a negative pitch means bottom-up and is flipped), else from `MF_MT_DEFAULT_STRIDE`. Then they are fitted and never mirrored. - **Endings:** a code the table calls gone, `MF_SOURCE_READERF_ERROR` and end of stream all give `Ended(DeviceLost)`. Any other read error (preempted: "in use") gives `Error(text)` and then `Ended(DeviceLost)`. The source is released before `Ended` is sent. - **stop:** sets the flag and calls `IMFMediaSource::Shutdown` from the stopping thread, which unblocks a `ReadSample`. It then joins the threads: the session thread drops the interrupter, the reader and the source (`Shutdown`, `ShutdownObject`), releases the claim, and calls `MFShutdown` and `CoUninitialize`. After that it drops the sink and returns. `Drop` stops. - No frame bytes in logs: name, mode, sizes, counts, and the id as `id_hash`. ## How it was tested - `tests/camera_mf.rs` (every OS, opens nothing): the plan's `list_activates_nothing`, `nv12_with_stride_and_negative_height`, `access_denied_is_not_allowed` (at activation and at the first read), `preempted_is_busy` (at start and mid-stream), `removed_is_device_lost` (also the error flag and end of stream; `Ended` arrives with 0 sources alive), `stop_shuts_source_down_before_returning` (a blocked read, the sink dropped before `stop` returns, `Drop`), `forbidden_env_refuses_before_activation`, `com_wrappers_release_on_every_path` (a failure at each step, NV12 refused, unknown read error, no usable type: everything back to 0, activations == shutdowns). Also: mode choice, fit (1080p60 capture card asked for 640×480 → 640×360), NV12 fallback, pacing 60 → 30 fps, stall once then resume, a silent camera, `NotFound` / `AlreadyStarted` / second consumer `Busy`, `modes`, and watch debounce and replacement. - **The Windows code was type-checked and linted for `x86_64-pc-windows-msvc` on Linux.** This used `cargo clippy --target x86_64-pc-windows-msvc --all-targets -- -D warnings` in a scratch crate that mounts this `camera` module (and `screen.rs`) against `windows` 0.62.2. It is clean, both with and without `fake`. A deliberate type error in `win.rs` was reported, which shows the block is really checked. The Windows runner checks it in the workspace after merge. - Local gate: `FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci`, all steps passed. It was run with `TMPDIR` set to a directory under the worktree's target dir, because `/tmp` was full of leaked `/tmp/appimage_extracted_*` directories (#983). No test needing a real display or device was run, and nothing was run with `FLUXER_DESKTOP_TESTS` or `FLUXER_CAMERA_TESTS`. ## Deviations from the plan - The real-device test is `tests/camera_real_win.rs` (Windows only, `#[ignore]`, `FLUXER_CAMERA_TESTS=1`) instead of a Windows case in `tests/camera_real.rs`. Task 10 creates that file in the same wave, and the change avoids an add/add conflict. The two can be merged later. - `media_foundation.rs` has two child files, `media_foundation/win.rs` (the real calls) and `media_foundation/memory.rs` (the in-memory I/O). - One added feature of the existing `windows` crate: `Win32_Devices_DeviceAndDriverInstallation`, for `CM_Register_Notification`. No new crate. - `camera/mod.rs`: `default_camera()` now dispatches to a `platform_camera()` per OS. Task 10 adds its Linux arm there. This is a registration file: keep both sides. ## What Charles must check on Windows (this code has never run) With a camera, `set FLUXER_CAMERA_TESTS=1` then `cargo test -p fluxer-capture --test camera_real_win -- --ignored --nocapture`, and in the app once Task 12 lands: 1. `list` does not turn the light on. Names are right and capture cards are listed. 2. A picture arrives within 5 s, the right way up and in true colours: an MJPEG-only webcam (does NV12 conversion work, or does the fallback decode run?) and a YUY2 one. 3. The light goes off within a second of `stop`. A covered or silent camera still stops promptly (the `Shutdown` from the stopping thread must unblock `ReadSample`). 4. The Windows Camera app holding the camera gives "in use" (`Busy`), at start or mid-stream. 5. Settings → Privacy → Camera off gives "not allowed" (`NotAllowed`), at activation or at the first read. 6. Unplugging during capture gives `Ended(DeviceLost)` and the light goes off. Plugging in or unplugging a camera updates the list (`watch`). 7. A capture card: its modes are all listed, and a 1080p60 mode can be selected. 8. On an N edition without the Media Feature Pack, the camera button is disabled and its tooltip gives the reason. Closes #963 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_0161qMQMsCyJhjYw4gqa6e4K
Refs #963

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0161qMQMsCyJhjYw4gqa6e4K
MfCamera, the Media Foundation camera back end (source reader, NV12 with a
native-decode fallback, first-read start errors, pacing by sample time,
stall watchdog, device loss through the mf_errors table, stop that shuts the
source down from the stopping thread and returns after it is released,
CM_Register_Notification hot-plug), generic over MfIo so its logic is tested
on every OS with the in-memory MemoryIo. default_camera() returns it on
Windows where Media Foundation starts.

Refs #963

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0161qMQMsCyJhjYw4gqa6e4K
Author
Owner

Merged into camera by hand (the merge API returns 405); the commits are in the branch.

Merged into `camera` by hand (the merge API returns 405); the commits are in the branch.
charles closed this pull request 2026-11-20 03:12:40 +00:00

Pull request closed

Sign in to join this conversation.
No description provided.