Task 24: Screenshot test harness and golden images (#24) #94
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "task/24-screenshot-tests"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Stacked on: task/20-ui-message-list-composer, task/22-ui-settings-parity
What this builds
fluxer_ui::testing::render(view: Element<UiMessage>, size: (u32, u32)) -> RgbaImage(src/testing/offscreen.rs): one frame of a view, drawn by iced's wgpu renderer (iced::Rendererthroughrenderer::Headless, backend"wgpu") into an offscreen texture. Dark theme, bundled fonts, scale factor 1, no cursor. It does what iced does before a draw: layout, aRedrawRequestedpass so widgets settle their status (a button that never saw one draws as disabled), another layout if a widget asked for it, then draw and read back.fluxer_ui::testing::assert_matches_golden(name, &image, tolerance)(src/testing/mod.rs): compares withfixtures/golden/<name>.png;DEFAULT_TOLERANCEis 0.5 (% of differing pixels).UPDATE_GOLDEN=1rewrites the golden. On failure it writes<name>.actual.pngand<name>.diff.png(differences in red) toFLUXER_SCREENSHOT_OUTand names them in the message.diff()is the comparison alone.crates/fluxer-ui/tests/screenshots.rs:login_screen_matches_golden,main_screen_with_basic_scenario_matches_golden,settings_appearance_matches_golden, 1280x720. State is built from the fixtures by applying events toApp(READY, GUILD_CREATE, history, MESSAGE_CREATE, PRESENCE_UPDATE, asfixtures/scenarios/basic.tomlplays them); no devserver. The clock is fixed.fixtures/golden/{login,main-general,settings-appearance}.png, drawn by lavapipe (Mesa 23.2.1) in anubuntu:22.04container with the CI job's package list..forgejo/workflows/ci.yaml: the Linux job setsWGPU_BACKEND=vulkanandVK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.x86_64.json, and fails in the package step if that file is missing.mesa-vulkan-driversandFLUXER_SCREENSHOT_TESTS=1were already there.docs/screenshots/README.md: the tests, the reference renderer, updating goldens, adding a screen, and how to capture the Electron side-by-side.How it was tested
testingmodule), then failed on missing goldens, then passed afterUPDATE_GOLDEN=1.ubuntu:22.04(same apt packages and env as the Linux job). I looked at all three images.FLUXER_SCREENSHOT_TESTS=1 cargo xtask cigreen locally. Not run on the Forgejo runners by me.Deviations from the plan
xtask/src/main.rs: only a doc comment changed. The step from Task 1 (cargo test --workspace -- --ignoredwhenFLUXER_SCREENSHOT_TESTS=1) already runs these tests, which are#[ignore]d.CHANNEL_TOLERANCE); the plan only gives the 0.5 %. Without it, rounding differences in anti-aliasing would count.tests/screenshots.rsis#![cfg(not(windows))], likemessage_view.rs(#93): a test binary that buildsiced::Renderercannot load on the Windows runner. Windows does not setFLUXER_SCREENSHOT_TESTSanyway.testingis an ordinary public module of the library (integration tests cannot seecfg(test)items). It reads and writes golden PNGs, which is file I/O in a crate that otherwise does none; the README says so. Nothing the application runs calls it.fluxer-devserver.crates/fluxer-ui/src/app.rsandcrates/fluxer-ui/README.md(both sides kept: Task 20'smain::updateand Task 22's settings routing).For the reviewer
error: XDG_RUNTIME_DIR not set in the environment.a few times in the container; the tests pass regardless. Expect that noise in the Linux log.docs/screenshots/README.mdare standard Chromium DevTools; I did not run the Electron app to try them.Closes #24
🤖 Generated with Claude Code
Independent review — Task 24 (screenshot tests), commit
480fde6Scope:
git diff e93471f..origin/task/24-screenshot-tests(11 files).Checks
login_screen_matches_golden,main_screen_with_basic_scenario_matches_golden,settings_appearance_matches_goldenall exist incrates/fluxer-ui/tests/screenshots.rs, build their state fromfixtures/(READY, guild_create, message_create, presence_update; no devserver), render at 1280x720 and compare withDEFAULT_TOLERANCE= 0.5 %. The three goldens are 1280x720 RGBA and show the expected screens (I looked at each).testing::render(view: Element<'_, UiMessage>, size: (u32, u32)) -> RgbaImage(iced wgpuHeadlessrenderer, offscreen),testing::assert_matches_golden(name, &image, tolerance),UPDATE_GOLDEN=1rewrites. Present as the plan names them (the image is taken by reference).fluxer-core; no secrets or tokens in code, fixtures or docs; nothing here parses server data.fluxer-uiI/O: see finding 1.FLUXER_SCREENSHOT_TESTS=1 cargo xtask ciin a fresh review worktree with its own target dir: green (exit 0; the 3 screenshot tests passed, working tree left clean). This ran on the host's hardware GPU, not on lavapipe; I did not reproduce the lavapipe container run, and both Forgejo jobs for480fde6were still pending when I looked once.Findings
crates/fluxer-ui/src/testing/mod.rs: golden PNG reads/writes are a second file-I/O path inside thefluxer-uilibrary, compiled unconditionally aspub mod testing(notcfg(test), no cargo feature). The plan prescribes this module and its behaviour, the application never calls it, and the crate README documents it, so I do not count it as an invariant break. ButCLAUDE.mdstill says the ImageLoader is the single exception and was not updated. Suggested follow-up: puttestingbehind atestingcargo feature (enabled by the crate's own dev-dependency on itself orrequired-featureson the test) and add one sentence toCLAUDE.md.crates/fluxer-ui/src/lib.rs/testing/offscreen.rs:render()buildsiced::Rendererand is now part of the lib that the Windows unit-test binary is built from. #93 showed that a binary that keeps such code fails to load on the Windows runner (opengl32.dll). It should be dropped by the linker likerun()is, but that is unverified: the Windows job on this commit has to be seen green before merge. Atestingfeature (finding 1) would also remove this risk.testing/mod.rs:CHANNEL_TOLERANCE = 8is on top of the plan's 0.5 %. At 1280x720, 0.5 % is 4608 pixels, so a missing line of message text or a wrongly coloured presence dot/badge stays under the threshold; the per-channel slack widens that further. Within the plan's numbers, so not blocking; worth knowing when relying on these tests for regressions (smaller per-widget frames would be sharper).xtask/src/main.rs: only a doc comment changed; the existing step runs every#[ignore]d test in the workspace under the env var, which covers these. Acceptable deviation from the plan's "Modify".docs/screenshots/README.md: the Electron capture procedure was not tried against the Electron app (implementer's own note); Task 26 will exercise it.Verdict: APPROVE — zero blocking findings, local ci green. Merge condition for the integrator: both runners green on
480fde6, the Linux job being the first real lavapipe run of the fullxtask ci, the Windows job covering finding 2.