Camera 11: fluxer-capture camera on Windows (Media Foundation) (#963) #984
Closed
charles
wants to merge 0 commits from
camera-11-capture-camera-on-windows into camera
pull from: camera-11-capture-camera-on-windows
merge into: charles:camera
charles:main
charles:camera
charles:dailyuse
charles:textchat
charles:fix-1008-idle-in-voice
charles:camera-21-acceptance
charles:feat-1000-app-sounds-output
charles:fix-1001-local-time
charles:camera-20-parity-list-for-the
charles:fix-931-silent-voices
charles:camera-15-ui-settings-voice-video
charles:camera-14-ui-camera-buttons-camera
charles:camera-16-ui-layout-with-cameras
charles:camera-17-core-toggle-camera-shortcut
charles:camera-13-ui-mirrored-video-and
charles:camera-12-appsfluxer-camera-runtime-and
charles:camera-10-capture-camera-on-linux
charles:camera-9-voice-the-camera-track
charles:camera-7-core-the-own-camera
charles:camera-6-capture-camera-trait-mode
charles:camera-8-core-remote-cameras-tile
charles:camera-3-spike-a-camera-track
charles:camera-4-video-camera-formats-and
charles:camera-18-devserver-camera-scenario
charles:camera-1-spike-capture-linux
charles:camera-5-core-camera-ports-commands
charles:camera-2-spike-capture-windows
charles:camera-plan
charles:shell
charles:fix-925-keybinds-persist
charles:feat-934-935-stream-view
charles:feat-933-stream-output-device
charles:shell-44-acceptance
charles:feat-946-select-message-text
charles:shell-43-parity-list-for-the
charles:fix-589-black-band-linux-sender
charles:shell-37-ui-system-wide-shortcuts
charles:shell-ship
charles:fix-931-voices-silent-after-long-uptime
charles:shell-39-ui-settings-notifications
charles:shell-36-ui-settings-keybinds
charles:shell-16-core-autostart
charles:fix-923-late-publication-next-share
charles:fix-921-share-teardown-publish-race
charles:shell-1-spike-iced-daemon-hide
charles:shell-10-core-instance-messages-tasks
charles:shell-11-core-keybinds-actions-combos
charles:shell-13-core-badge-attention-sound
charles:shell-14-core-notification-settings-inactive
charles:shell-18-platform-desktop-entry-and
charles:shell-2-spike-tray-on-linux
charles:shell-3-spike-wayland-global-shortcuts
charles:shell-41-devserver-shell-scenario
charles:shell-5-spike-overlay-technique-for
charles:shell-6-core-shell-ports-settings
charles:shell-7-platform-settings-shell-keybinds
charles:shell-8-core-tray-model-and
charles:shell-9-core-window-rules-close
charles:fix-913-keep-awake-retry
charles:shell-35-ui-in-window-voice
charles:shell-sync-textchat
charles:shell-34-app-shortcuts-runtime
charles:fix-913-keep-screen-awake
charles:fix-910-deepfilter-windows-timing
charles:shell-33-app-shell-runtime
charles:shell-28-platform-shortcuts-on-wayland
charles:fix-910-deepfilter-late-frame
charles:shell-29-platform-shortcuts-by-direct
charles:shell-27-platform-shortcuts-on-x11
charles:shell-17-platform-single-instance
charles:shell-24-platform-badge-displays-and-24
charles:shell-26-platform-shortcuts-on-windows
charles:shell-21-platform-tray-on-linux
charles:shell-22-platform-tray-on-windows
charles:fix-908-flaky-stream-end-test
charles:shell-30-platform-wayland-activation
charles:shell-38-ui-settings-desktop
charles:shell-19-platform-autostart
charles:shell-25-platform-shortcut-back-end
charles:shell-23-platform-badge-displays-and
charles:shell-40-ui-companion-apps-control
charles:shell-31-statusapi-default-avatars
charles:shell-12-core-system-wide-shortcut
charles:shell-20-platform-tray-trait-icons
charles:shell-32-ui-daemon-window-lifecycle
charles:shell-4-spike-raising-a-wayland
charles:shell-15-core-status-payload-gaps
charles:fix-876-share-resolution-and-bars
charles:shell-42-packaging-windows-shortcuts-and
charles:fix-874-tests-must-not-notify
charles:fix-868-deepfilternet-start
charles:gamenight-plan
charles:fix-867-869-stream-end-soundboard-limits
charles:feat-229-windows-hardware-encoding
charles:fix-857-vaapi-hardware-encoding
charles:fix-849-inert-cards
charles:fix-848-pinned-messages-clickable
charles:shell-plan
charles:fix-764-nvenc-above-720p30
charles:serveradmin
charles:feat-780-forum-archived-filter
charles:serveradmin-35-acceptance
charles:feat-781-no-join-chime-with-entrance-sound
charles:feat-784-rejoin-voice-after-update
charles:fix-779-782-search-dropdown-paste-preview
charles:fix-783-781-dm-rail-entrance-sound
charles:serveradmin-34-parity-screenshots
charles:serveradmin-31-ui-channel-permissions
charles:fix-773-cargo-deny-advisories
charles:serveradmin-33-ui-sudo-delete-transfer
charles:serveradmin-28-ui-members-page
charles:serveradmin-26-ui-role-permissions
charles:serveradmin-30-ui-voice-overview-channel-invites
charles:serveradmin-fix-27-29
charles:serveradmin-25-ui-roles-tab
charles:serveradmin-27-ui-menus-moderation-modals
charles:serveradmin-29-ui-channel-settings
charles:fix-589-stream-top-strip-encode
charles:serveradmin-24-ui-moderation-invites-bans
charles:feat-760-version-in-title
charles:fix-751-stream-audio-pipewire
charles:serveradmin-20-core-menu-rows
charles:fix-752-linux-notification-panic
charles:serveradmin-17-core-overrides
charles:fix-740-updater-always-latest
charles:serveradmin-23-ui-community-settings-overview
charles:serveradmin-12-core-roles
charles:serveradmin-19-core-sudo-delete-transfer
charles:serveradmin-32-ui-sidebar-drag
charles:serveradmin-15-core-members-page
charles:serveradmin-16-core-channel-settings
charles:serveradmin-14-core-bans-invites
charles:fix-615-restore-on-stored-session-start
charles:serveradmin-22-runtime
charles:serveradmin-13-core-member-moderation
charles:serveradmin-18-core-channel-reorder
charles:fix-732-status-api-speaking
charles:serveradmin-10-core-permission-lists
charles:serveradmin-11-core-forms-community
charles:serveradmin-21b-devserver-channels-members-sudo
charles:serveradmin-4-api-community-roles-sudo
charles:serveradmin-9b-core-hierarchy-bulk-events
charles:serveradmin-21a-devserver-community-roles
charles:serveradmin-9a-core-ports
charles:serveradmin-8-form-widgets
charles:serveradmin-5-api-channels-members-bans
charles:serveradmin-7-settings-shell
charles:serveradmin-1-proto-community-roles
charles:serveradmin-3-proto-members-bans-invites
charles:serveradmin-6-reorder-widget
charles:serveradmin-2-proto-channel-admin
charles:fix-706-presence-after-reconnect
charles:fix-615-start-on-last-view
charles:feat-663-create-channel
charles:serveradmin-plan
charles:fix-662-long-text-wrap
charles:fix-660-661-menu-toolbar-paste-focus
charles:fix-620-forum-list-reactions
charles:fix-657-windows-console
charles:fix-622-emoji-in-names
charles:fix-smoke-645-648
charles:fix-614-install-location
charles:dailyuse-41-acceptance
charles:fix/597-copy-id
charles:dailyuse-40-parity-screenshots
charles:dailyuse-35-ui-quick-switcher-keybinds
charles:dailyuse-38-ui-profile-modal
charles:fix/voice-view
charles:fix/side-panel-composer
charles:dailyuse-37b-ui-composer-barriers-blocked
charles:fix/image-viewer
charles:fix/619-picker-flicker
charles:fix/editor-forum-edit
charles:dailyuse-32-ui-status-picker
charles:dailyuse-34b-ui-add-guild-invite-modal
charles:dailyuse-28-app-dailyuse-runtime
charles:dailyuse-36-ui-voice-badge-refusals
charles:fix/594-menu-placement-textchat
charles:fix/589-stream-top-strip
charles:fix/594-menu-placement
charles:dailyuse-39-ui-report-modal
charles:dailyuse-37a-ui-attachment-audio-group-icon
charles:dailyuse-24-core-notification-icon-passive
charles:fix/587-quiet-voice
charles:dailyuse-30-ui-notification-settings-modal
charles:dailyuse-31-ui-user-dm-menus
charles:dailyuse-22b-core-composer-barriers-blocked
charles:dailyuse-22a-core-mark-unread-attachments
charles:dailyuse-29-ui-channel-guild-menus
charles:dailyuse-18-core-user-dm-menus
charles:dailyuse-33-ui-confirm-friends-page
charles:dailyuse-22c-core-trusted-domains-hide-muted
charles:dailyuse-20-core-keyboard-navigation
charles:dailyuse-15-core-friends-blocking
charles:dailyuse-23-core-profile-report-flow
charles:dailyuse-19-core-quick-switcher
charles:dailyuse-14-core-status-idle-presence
charles:dailyuse-17-core-channel-guild-menus
charles:ci/27-linux-cache
charles:dailyuse-37c-ui-trust-links-hide-muted
charles:dailyuse-13-core-honour-mutes
charles:dailyuse-34a-ui-invite-cards-accept
charles:dailyuse-16b-core-create-invites-guilds
charles:dailyuse-12-core-notification-settings-mutes
charles:dailyuse-16a-core-invite-links
charles:dailyuse-25-audio-attachments
charles:dailyuse-21-core-voice-refusals
charles:dailyuse-11b-core-state-skeleton
charles:dailyuse-11a-core-dailyuse-ports
charles:dailyuse-26b-devserver-dailyuse-b
charles:dailyuse-26a-devserver-dailyuse-a
charles:dailyuse-4-api-settings-acks-reports
charles:dailyuse-5-api-relationships-invites-guilds
charles:dailyuse-27-platform-notification-idle
charles:dailyuse-10-highlight-languages
charles:fix-search-focus-hover-emoji
charles:dailyuse-8-context-menu-widget
charles:dailyuse-9-image-load-retry
charles:dailyuse-7-chat-navigation-persistence
charles:dailyuse-2-proto-relationships-invites-reports
charles:dailyuse-1-proto-notification-presence
charles:dailyuse-6-gateway-op12-presence
charles:textchat-next
charles:fix-stream-black
charles:fix-reactions-hover
charles:textchat-68-app-video-playback-runtime
charles:textchat-35-acceptance
charles:textchat-34-parity-screenshots
charles:textchat-74-app-updater-runtime
charles:textchat-67-ui-video-player
charles:textchat-stage-46-47
charles:textchat-47-ui-media-channel
charles:textchat-75-promote-stable-workflow
charles:textchat-46-ui-forum-composer
charles:textchat-73-ui-update-button
charles:textchat-64-media-playback-crate
charles:textchat-stage-43-48
charles:textchat-72-platform-updater
charles:textchat-44-ui-thread-view
charles:textchat-70-release-update-feed
charles:textchat-48-ui-announcements
charles:textchat-45-ui-forum-view
charles:textchat-71-core-updater
charles:textchat-43-ui-sidebar-links-notes
charles:textchat-stage-40
charles:textchat-42-app-channel-types-runtime
charles:textchat-39-core-forums
charles:textchat-40-core-announcements-thread-admin
charles:textchat-60-ui-entrance-sounds
charles:textchat-63-ffmpeg-lgpl-packaging
charles:textchat-stage-batch6
charles:textchat-69-proto-update-manifest
charles:textchat-61-app-soundboard-entrance-runtime
charles:textchat-52-app-audio-devices-runtime
charles:textchat-38-core-channel-routing
charles:soundboard-effect-emoji
charles:status-api-rebind
charles:textchat-28-ui-gif-picker
charles:textchat-41-devserver-channel-types
charles:voice-panel-soundboard
charles:textchat-29-ui-message-actions
charles:textchat-37-api-channel-types
charles:textchat-66-core-video-players
charles:autoupdate-plan
charles:textchat-33-ui-message-list-settings
charles:textchat-32-ui-dms
charles:textchat-62-rename-flaxative
charles:fix-chatcolumn-tree
charles:textchat-65-audio-media-source
charles:textchat-31-ui-threads
charles:textchat-stage-27
charles:textchat-30-ui-search-pins-inbox
charles:textchat-53-ui-audio-sources-menu
charles:textchat-27-ui-expression-picker
charles:textchat-57-core-entrance-library
charles:textchat-36-proto-channel-types
charles:textchat-26-ui-upload-tray
charles:textchat-59-ui-soundboard-popout-parity
charles:textchat-58-audio-sound-edit
charles:textchat-55-proto-api-entrance-library
charles:textchat-25-ui-composer
charles:textchat-56-core-soundboard-parity
charles:rename-video-plan
charles:textchat-stage-25
charles:textchat-stage-23-24
charles:soundboard-plan
charles:textchat-24-ui-message-row
charles:textchat-22-ui-attachments-media
charles:textchat-23-ui-embeds
charles:textchat-14-core-expressions
charles:textchat-21-app-textchat-runtime
charles:textchat-16-core-navigation-read
charles:textchat-17-core-search
charles:textchat-19-core-dms-settings
charles:textchat-stage-13-15-18
charles:textchat-54-livekit-stereo-audio
charles:textchat-50-core-stream-audio-sources
charles:textchat-12-core-uploads
charles:textchat-15-core-message-actions
charles:textchat-11-core-composer
charles:textchat-18-core-threads
charles:textchat-13-core-reactions
charles:textchat-51-audio-pipewire-devices
charles:textchat-49-capture-audio-targets
charles:audio-plan
charles:textchat-plan2
charles:textchat-20-devserver-textchat
charles:textchat-6-api-chat-endpoints
charles:textchat-10-core-chat-ports
charles:textchat-9-ui-code-highlighting
charles:textchat-3-proto-expressions-threads-prefs
charles:textchat-5-api-messages-uploads
charles:textchat-2-proto-chat-requests-events
charles:textchat-8-ui-inline-flow-emoji
charles:screenshare
charles:textchat-7-platform-files-clipboard
charles:textchat-1-proto-message-content
charles:textchat-4-emoji-crate
charles:screenshare-31-acceptance
charles:screenshare-29-parity-screenshots
charles:textchat-plan
charles:screenshare-24-ui-focus-layout
charles:screenshare-27-ui-stream-menu
charles:screenshare-30-packaging-ci
charles:screenshare-21-app-screenshare-runtime
charles:screenshare-22-ui-sidebar-live
charles:screenshare-23-ui-stream-tile
charles:screenshare-26-ui-source-picker
charles:screenshare-28-ui-screenshare-settings
charles:screenshare-13-core-previews
charles:screenshare-12-core-codec-negotiation
charles:screenshare-25-ui-share-controls
charles:screenshare-18-hw-encode-linux
charles:screenshare-19-spike-hw-encode-windows
charles:screenshare-11-core-stream-params
charles:screenshare-20-devserver-screenshare
charles:screenshare-10-core-go-live
charles:screenshare-17-voice-livekit-video
charles:screenshare-14-audio-capture-linux
charles:wip/screenshare-17
charles:screenshare-integration
charles:feat-user-profile
charles:fix-voice-join
charles:feat-entity-images
charles:screenshare-16-voice-video-session
charles:screenshare-9-core-watching
charles:screenshare-5-capture-windows
charles:fix-wire-livekit
charles:screenshare-4-capture-linux
charles:screenshare-15-audio-capture-windows
charles:fix-member-list
charles:screenshare-8-core-stream-ports
charles:fix-live-protocol
charles:screenshare-6-ui-video-widget
charles:screenshare-3-capture-crate
charles:fix-altcha-captcha
charles:screenshare-1-video-frames
charles:screenshare-7-proto-api-streams
charles:screenshare-2-spike-livekit-video
charles:voice
charles:voice-26-acceptance
charles:voice-23-voice-e2ee
charles:voice-25-packaging-ci
charles:voice-13-voice-livekit-interop
charles:voice-24-parity-screenshots
charles:voice-19-ui-soundboard
charles:voice-20-ui-participant-menu
charles:voice-22-ui-dm-calls
charles:voice-21-ui-voice-settings
charles:voice-18-ui-call-view
charles:voice-14-app-voice-runtime
charles:voice-10-core-soundboard
charles:voice-17-ui-voice-panel
charles:voice-9-core-voice-settings-advert
charles:voice-5-audio-clips-chimes
charles:voice-15-devserver-voice
charles:voice-8-core-live-room-volumes
charles:voice-11-core-calls-status
charles:voice-12-voice-session
charles:voice-4-audio-devices-mixer
charles:voice-16-ui-voice-sidebar
charles:voice-7-core-voice-connection
charles:foundation
charles:voice-2-proto-api-soundboard-rtc
charles:voice-6-core-voice-states
charles:voice-3-audio-dsp
charles:voice-1-proto-voice-gateway
charles:task/26-acceptance
charles:task/25-packaging-release
charles:task/24-screenshot-tests
charles:task/20-ui-message-list-composer
charles:task/22-ui-settings-parity
charles:task/18-ui-main-layout
charles:task/23-notifications
charles:task/17-ui-shell-login
charles:task/14-app-runtime
charles:ci/51-windows-disk-space
charles:task/12-platform-persistence
charles:task/13-devserver
charles:task/21-ui-avatar-image-cache
charles:task/10-core-messages-readstate
charles:task/9-core-guilds-channels
charles:task/7-gateway-machine-transport
charles:task/11-core-presence-ui
charles:task/19-ui-rich-text
charles:task/8-core-store-session
charles:task/16-ui-theme-fonts-icons
charles:task/15-status-api
charles:task/4-proto-gateway-events
charles:task/5-openapi-vendor-contract
charles:task/6-api-client
charles:task/3-proto-rest-models
charles:task/2-proto-primitives
charles:task/1-workspace-scaffold
No reviewers
Labels
Clear labels
bug
A bug found while working on something else; steps to reproduce and the commit seen on
ci
Build, CI runners and workflows
follow-up
Deferred reviewer finding, linked from the PR it came from
good-first-task
Small, well-scoped, no cross-crate knowledge needed
parity
Settings/behaviour parity with the Electron client
question
Design question that needs Charles; blocks its task until answered
ready-to-merge
Reviewer approved, foundation merged in, CI green on both runners; waiting for the orchestrator to merge
task
A plan task (one per Foundation plan task)
No labels
bug
ci
follow-up
good-first-task
parity
question
ready-to-merge
task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
charles/fluxer-native!984
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "camera-11-capture-camera-on-windows"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What this builds
fluxer_capture::camera::media_foundation::MfCamera, the WindowsCameraCapturerover the Media Foundation source reader (Camera 2's choice, spikes §2), anddefault_camera()returning it on Windows where Media Foundation starts (an N edition without the Media Feature Pack getsUnsupportedwith the reason).MfCamera<Io: MfIo>holds all the logic;MfIo/MfSourceare the calls it needs (enumerate, COM on a thread, activate + reader, native types, select, NV12, output type, read a sample with its locked buffer, interrupt, register notifications).WindowsIo(media_foundation/win.rs, Windows only) makes the real calls;memory::MemoryIo(tests and featurefake) replays scripted devices, failures at anyMfStepand reads, and counts COM threads, sources, locks and registrations alive.MFEnumDeviceSources(VIDCAP: capture cards list too), names throughnames::display, id = symbolic link. Activates nothing (list_activates_nothing).CM_Register_NotificationforKSCATEGORY_VIDEO_CAMERAandKSCATEGORY_CAPTURE. This is the windowless alternative that spikes §2 allows, and it is simpler than a message-only window. A burst of notifications callson_changeonce, 250 ms after the last one.forbidden()first. Then, on the session thread: COM MTA +MFStartup; the device by link (case-insensitive;Default= first). The device is refused withBusywhen anotherMfCameraof this process has it open. ThenActivateObject, a reader with advanced video processing, only the first video stream selected, the native type fromformat::choose, and NV12 asked for. If the reader refuses NV12, it re-selects the native type and decodes it withcamera::decode(the spike's fallback).startreturns after the first read, soE_ACCESSDENIEDat the first read is stillNotAllowed(the docs leave open which call returns it). A camera silent for 3 s starts anyway and the stall watchdog takes over. Samples are paced by their time stamps. They are converted byframe_from_buffer: the stride comes fromIMF2DBuffer2::Lock2DSize(a negative pitch means bottom-up and is flipped), else fromMF_MT_DEFAULT_STRIDE. Then they are fitted and never mirrored.MF_SOURCE_READERF_ERRORand end of stream all giveEnded(DeviceLost). Any other read error (preempted: "in use") givesError(text)and thenEnded(DeviceLost). The source is released beforeEndedis sent.IMFMediaSource::Shutdownfrom the stopping thread, which unblocks aReadSample. It then joins the threads: the session thread drops the interrupter, the reader and the source (Shutdown,ShutdownObject), releases the claim, and callsMFShutdownandCoUninitialize. After that it drops the sink and returns.Dropstops.id_hash.How it was tested
tests/camera_mf.rs(every OS, opens nothing): the plan'slist_activates_nothing,nv12_with_stride_and_negative_height,access_denied_is_not_allowed(at activation and at the first read),preempted_is_busy(at start and mid-stream),removed_is_device_lost(also the error flag and end of stream;Endedarrives with 0 sources alive),stop_shuts_source_down_before_returning(a blocked read, the sink dropped beforestopreturns,Drop),forbidden_env_refuses_before_activation,com_wrappers_release_on_every_path(a failure at each step, NV12 refused, unknown read error, no usable type: everything back to 0, activations == shutdowns). Also: mode choice, fit (1080p60 capture card asked for 640×480 → 640×360), NV12 fallback, pacing 60 → 30 fps, stall once then resume, a silent camera,NotFound/AlreadyStarted/ second consumerBusy,modes, and watch debounce and replacement.x86_64-pc-windows-msvcon Linux. This usedcargo clippy --target x86_64-pc-windows-msvc --all-targets -- -D warningsin a scratch crate that mounts thiscameramodule (andscreen.rs) againstwindows0.62.2. It is clean, both with and withoutfake. A deliberate type error inwin.rswas reported, which shows the block is really checked. The Windows runner checks it in the workspace after merge.FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci, all steps passed. It was run withTMPDIRset to a directory under the worktree's target dir, because/tmpwas full of leaked/tmp/appimage_extracted_*directories (#983). No test needing a real display or device was run, and nothing was run withFLUXER_DESKTOP_TESTSorFLUXER_CAMERA_TESTS.Deviations from the plan
tests/camera_real_win.rs(Windows only,#[ignore],FLUXER_CAMERA_TESTS=1) instead of a Windows case intests/camera_real.rs. Task 10 creates that file in the same wave, and the change avoids an add/add conflict. The two can be merged later.media_foundation.rshas two child files,media_foundation/win.rs(the real calls) andmedia_foundation/memory.rs(the in-memory I/O).windowscrate:Win32_Devices_DeviceAndDriverInstallation, forCM_Register_Notification. No new crate.camera/mod.rs:default_camera()now dispatches to aplatform_camera()per OS. Task 10 adds its Linux arm there. This is a registration file: keep both sides.What Charles must check on Windows (this code has never run)
With a camera,
set FLUXER_CAMERA_TESTS=1thencargo test -p fluxer-capture --test camera_real_win -- --ignored --nocapture, and in the app once Task 12 lands:listdoes not turn the light on. Names are right and capture cards are listed.stop. A covered or silent camera still stops promptly (theShutdownfrom the stopping thread must unblockReadSample).Busy), at start or mid-stream.NotAllowed), at activation or at the first read.Ended(DeviceLost)and the light goes off. Plugging in or unplugging a camera updates the list (watch).Closes #963
🤖 Generated with Claude Code
https://claude.ai/code/session_0161qMQMsCyJhjYw4gqa6e4K
Merged into
cameraby hand (the merge API returns 405); the commits are in the branch.Pull request closed