fix(dnsdist): health-check flap tolerance + log container dropdown #1

Merged
charles merged 2 commits from fix/dnsdist-flap-and-log-dropdown into main 2026-06-28 02:12:37 +00:00
Owner

Two fixes from the dnsdist timeout + missing-logs investigation on the live grid.

fix(dnsdist): tolerate slow CHAOS health probes (maxCheckFailures=3, rise=2)

A single missed 3s CHAOS TXT version.bind probe marked a backend down. With the Internal-auth pool served by a single PowerDNS member, one slow probe black-holed all internal-zone queries (intermittent client SERVFAIL/timeouts), especially while that backend absorbed bursts of dynamic-DNS UPDATEs. Now requires 3 consecutive failures to mark down, 2 successes to mark up.

fix(logs): list containers via stats by so low-volume members are not truncated

listContainerStreams pulled query=* limit=100000 and derived the dropdown from returned rows. With ~170k lines/24h the cap truncated and dropped bursty/idle containers (e.g. a dnsdist member that only logs on backend flaps) from the UI picker. Now aggregates with * | stats by (mesh_host_id, container_name) count() — one row per container, no row limit.

Tests updated; bun test for both suites green (39 pass). Pre-commit biome + typecheck passed.

🤖 Generated with Claude Code

Two fixes from the dnsdist timeout + missing-logs investigation on the live grid. ## fix(dnsdist): tolerate slow CHAOS health probes (`maxCheckFailures=3, rise=2`) A single missed 3s `CHAOS TXT version.bind` probe marked a backend down. With the Internal-auth pool served by a single PowerDNS member, one slow probe black-holed all internal-zone queries (intermittent client SERVFAIL/timeouts), especially while that backend absorbed bursts of dynamic-DNS UPDATEs. Now requires 3 consecutive failures to mark down, 2 successes to mark up. ## fix(logs): list containers via `stats by` so low-volume members are not truncated `listContainerStreams` pulled `query=* limit=100000` and derived the dropdown from returned rows. With ~170k lines/24h the cap truncated and dropped bursty/idle containers (e.g. a dnsdist member that only logs on backend flaps) from the UI picker. Now aggregates with `* | stats by (mesh_host_id, container_name) count()` — one row per container, no row limit. Tests updated; `bun test` for both suites green (39 pass). Pre-commit biome + typecheck passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
A single missed 3s CHAOS TXT version.bind probe marked a backend down. With
the Internal-auth pool served by a single PowerDNS auth member, one slow probe
black-holed all internal-zone queries (intermittent client SERVFAIL/timeouts),
especially while that backend absorbed bursts of dynamic-DNS UPDATEs. Require
3 consecutive failures to mark down and 2 successes to mark back up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fix(logs): list containers via stats by so low-volume members aren't truncated
Some checks failed
test / bun-test (pull_request) Failing after 32s
test / typecheck (pull_request) Successful in 57s
test / lint (pull_request) Successful in 26s
test / docker-build (pull_request) Failing after 30s
100d746099
listContainerStreams pulled `query=* limit=100000` and derived the container
dropdown from the returned rows. With ~170k lines/24h the cap truncated and
silently dropped bursty/idle containers (e.g. a dnsdist member that only logs
on backend up/down flaps) from the UI picker. Aggregate with
`* | stats by (mesh_host_id, container_name) count()` — one row per distinct
container regardless of volume, no row limit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
charles deleted branch fix/dnsdist-flap-and-log-dropdown 2026-06-28 02:12:37 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
charles/projectoctet!1
No description provided.