Windows: release exe is a GUI program; CLI output attaches to the parent console (#657) #659

Closed
charles wants to merge 0 commits from fix-657-windows-console into textchat
Owner

Closes #657

Cause

apps/fluxer/src/main.rs had no windows_subsystem attribute, so flaxative.exe was linked for the console subsystem and Windows gave it a console window when started from the shortcut or Explorer.

Fix

  • #![cfg_attr(all(windows, not(debug_assertions)), windows_subsystem = "windows")]: release builds are GUI programs; debug builds stay console programs.
  • output_to_terminal() in main.rs, called just before the process writes its command-line output (usage, --help, --version, --debug-info, a config error, and the two later startup errors: the async runtime or the window cannot start). If GetStdHandle has no stdout or stderr handle, it calls AttachConsole(ATTACH_PARENT_PROCESS). Microsoft's GetStdHandle documentation says that when a process attaches to a console, its standard handles are replaced with console handles unless STARTF_USESTDHANDLES was used at creation. So redirected or piped output (> file, pipes, Command::output, the smoke test) keeps the parent's handles. When both handles exist, it does not attach. It never attaches while the window runs, because an attached process would get the terminal's Ctrl-C and close with the terminal. The Velopack startup hook still runs first and is unchanged; its hook paths exit before any of this.
  • Workspace windows crate: added feature Win32_System_Console. The Windows-only imports are inside the #[cfg(windows)] block.
  • xtask package (Windows) reads the PE Subsystem field (libs::pe_subsystem), prints ==> xtask package: subsystem windows, and fails on a console exe (check_windows_subsystem).
  • release.yaml installer smoke test: --debug-info used to run through cmd /c "… > file". It now runs through .NET Process with stdout and stderr piped and a 2-minute deadline, so the step waits for the GUI program no matter how the shell treats GUI programs. It fails if the report has no flaxative line.

Child processes on Windows

  • Link opener rundll32 url.dll,FileProtocolHandler and the "open/reveal install folder" explorer: both are GUI-subsystem programs and never get a console, so they are unchanged. CREATE_NO_WINDOW is ignored for GUI programs.
  • Velopack: wait_exit_then_apply_updates starts Update.exe through run_process(..., show_window=false), which uses CREATE_NO_WINDOW (velopack 1.2.161 process_win.rs).
  • No ffmpeg or other helper processes: FFmpeg is linked as DLLs. Nothing else in fluxer-platform starts processes on Windows (update/appimage.rs is Linux-only).

Verification

  • New unit tests in xtask/src/package/libs.rs: pe_subsystem_reads_the_optional_header and the_windows_exe_must_be_a_gui_program. Both failed against a stub, then passed after the fix.
  • xtask/tests/package.rs (Windows packaging test, runs on the Windows CI runner) now asserts subsystem windows, and its existing --debug-info check runs the packaged GUI exe with piped output.
  • FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci on Linux: all steps passed.

Untested on Windows

I have no Windows machine or Windows Rust target here, so none of the cfg(windows) code has been compiled or run. I checked the API signatures against the windows 0.62.2 source only. Windows CI (clippy -D warnings, the packaging test) is the first real check. Not tested by hand: --help, --version and --debug-info from an interactive cmd or PowerShell, and the edited release smoke test, which only runs in release.yaml. Known GUI-program behaviour: an interactive shell does not wait for the program, so the attached output can appear after the prompt and %ERRORLEVEL% is not the program's exit code. Redirected or piped use is not affected.

Noticed, not changed

Velopack's delta path (UpdateManager::download_updates → download_and_apply_delta_updates) runs Update.exe with a plain std::process::Command and no CREATE_NO_WINDOW. Releases are packed with --delta None today, so this path is unused. If deltas are turned on and Update.exe is a console program, a window could flash. I did not verify Update.exe's subsystem.

🤖 Generated with Claude Code

Closes #657 ## Cause `apps/fluxer/src/main.rs` had no `windows_subsystem` attribute, so `flaxative.exe` was linked for the console subsystem and Windows gave it a console window when started from the shortcut or Explorer. ## Fix - `#![cfg_attr(all(windows, not(debug_assertions)), windows_subsystem = "windows")]`: release builds are GUI programs; debug builds stay console programs. - `output_to_terminal()` in `main.rs`, called just before the process writes its command-line output (usage, `--help`, `--version`, `--debug-info`, a config error, and the two later startup errors: the async runtime or the window cannot start). If `GetStdHandle` has no stdout or stderr handle, it calls `AttachConsole(ATTACH_PARENT_PROCESS)`. Microsoft's `GetStdHandle` documentation says that when a process attaches to a console, its standard handles are replaced with console handles unless `STARTF_USESTDHANDLES` was used at creation. So redirected or piped output (`> file`, pipes, `Command::output`, the smoke test) keeps the parent's handles. When both handles exist, it does not attach. It never attaches while the window runs, because an attached process would get the terminal's Ctrl-C and close with the terminal. The Velopack startup hook still runs first and is unchanged; its hook paths exit before any of this. - Workspace `windows` crate: added feature `Win32_System_Console`. The Windows-only imports are inside the `#[cfg(windows)]` block. - `xtask package` (Windows) reads the PE `Subsystem` field (`libs::pe_subsystem`), prints `==> xtask package: subsystem windows`, and fails on a console exe (`check_windows_subsystem`). - `release.yaml` installer smoke test: `--debug-info` used to run through `cmd /c "… > file"`. It now runs through .NET `Process` with stdout and stderr piped and a 2-minute deadline, so the step waits for the GUI program no matter how the shell treats GUI programs. It fails if the report has no `flaxative ` line. ## Child processes on Windows - Link opener `rundll32 url.dll,FileProtocolHandler` and the "open/reveal install folder" `explorer`: both are GUI-subsystem programs and never get a console, so they are unchanged. `CREATE_NO_WINDOW` is ignored for GUI programs. - Velopack: `wait_exit_then_apply_updates` starts `Update.exe` through `run_process(..., show_window=false)`, which uses `CREATE_NO_WINDOW` (velopack 1.2.161 `process_win.rs`). - No ffmpeg or other helper processes: FFmpeg is linked as DLLs. Nothing else in `fluxer-platform` starts processes on Windows (`update/appimage.rs` is Linux-only). ## Verification - New unit tests in `xtask/src/package/libs.rs`: `pe_subsystem_reads_the_optional_header` and `the_windows_exe_must_be_a_gui_program`. Both failed against a stub, then passed after the fix. - `xtask/tests/package.rs` (Windows packaging test, runs on the Windows CI runner) now asserts `subsystem windows`, and its existing `--debug-info` check runs the packaged GUI exe with piped output. - `FLUXER_SCREENSHOT_TESTS=1 cargo xtask ci` on Linux: all steps passed. ## Untested on Windows I have no Windows machine or Windows Rust target here, so none of the `cfg(windows)` code has been compiled or run. I checked the API signatures against the windows 0.62.2 source only. Windows CI (clippy `-D warnings`, the packaging test) is the first real check. Not tested by hand: `--help`, `--version` and `--debug-info` from an interactive cmd or PowerShell, and the edited release smoke test, which only runs in release.yaml. Known GUI-program behaviour: an interactive shell does not wait for the program, so the attached output can appear after the prompt and `%ERRORLEVEL%` is not the program's exit code. Redirected or piped use is not affected. ## Noticed, not changed Velopack's delta path (`UpdateManager::download_updates` → `download_and_apply_delta_updates`) runs `Update.exe` with a plain `std::process::Command` and no `CREATE_NO_WINDOW`. Releases are packed with `--delta None` today, so this path is unused. If deltas are turned on and `Update.exe` is a console program, a window could flash. I did not verify `Update.exe`'s subsystem. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
flaxative.exe had no windows_subsystem attribute, so it was a console
program and Windows opened a console window with the app. Release builds
are now GUI-subsystem programs. Usage, --help, --version, --debug-info and
startup errors attach to the parent's console (AttachConsole) when stdout
or stderr has no handle; redirected and piped output keeps its handles.

xtask package reads the exe's PE subsystem and refuses a console exe; the
Windows packaging test asserts it. The release smoke test starts
--debug-info through .NET Process with pipes and a deadline, so it waits
for the GUI program and checks the report arrived.

Closes #657

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
Owner

Merged into textchat by hand (the merge API returns 405); the commits are in the branch.

Merged into `textchat` by hand (the merge API returns 405); the commits are in the branch.
charles closed this pull request 2026-11-15 18:55:20 +00:00

Pull request closed

Sign in to join this conversation.
No description provided.