Fix slow voice joins and the missing connect chime; add a voice log #215
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "fix-voice-join"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Stacked on: #203 (
fix-wire-livekit). Merge #203 first; this branch contains it.Two voice bugs reported from the first real joins (slow connect, no connect sound), plus a voice log, which did not exist before. Refs #126.
Slow connect
The cause is proven by the new end-to-end test on loopback. The LiveKit SDK builds its WebRTC runtime inside the first
Room::connect: the peer connection factory, APM/AEC3, the codec factories and their VAAPI probes. It then drops that runtime with the last room. So every join rebuilt it before the first signalling byte: 0.55 to 1.2 s here, and it blocked a tokio worker while doing so (livekit-0.10.1/src/rtc_engine/lk_runtime.rs:76-95keeps it behind aWeak).LiveKitRoomnow keeps the runtime for its whole life.LiveKitRoom::warmed_up(), which the app uses, builds it on its own thread at start.Join timeline on loopback (
since_join_ms, from the log):LkRuntime::new)No connect sound
The cause is proven by code:
fluxer-corenever emitted a chime for this client's own join.live.rschime_forexplicitly skips the client's own connection, andconnection.rson_room(Connected)played nothing. The original playsuser-joinon LiveKitRoomEvent.Connected(VoiceRoomEventBinder.ts:329,VoiceSelfJoinChime.ts).What the client does now:
user-joinwhen it becomes Connected, also while deafened.user-moveat the click (not while deafened), with no join sound after it. Theuser-move.mp3asset is copied from the fork.voice-disconnectalso plays when a join is abandoned while still connecting, as the original'srunDisconnectFromVoiceChanneldoes.Checked and fine: the assets are
include_bytes!, so they are present in release builds. No setting gates them: the sound settings are not built yet (sub-project 6), and the original leaves the voice sounds on by default. Output is opened on demand forCpalHostandNullHostalike.Voice log
info: onevoice:line per step, withsince_join_ms. Steps: join requested, op 4 sent, server update (host + connection id), room connect requested / connecting / joined (with duration), mic published, UI state Connected, capture started/stopped, audio input/output opened (device name, rate, channels), chime played (name), disconnected/failed (with why).debug: room connection state, subscriptions, participants, tracks.logrecords are bridged totracingaslivekit_sdk(at mostdebug). URL queries, JWTs and token/Bearer values are struck out first (logging::redact, unit-tested). The e2e test logs atdebugand asserts the token and its signature are absent from the log file.End-to-end test
apps/fluxer/tests/voice_livekit.rsruns incargo xtask ciunderFLUXER_LIVEKIT_TESTS=1, in thelivekit-interopstep. Setup:LiveKitRoom::warmed_up()andNullHostVoiceScript::set_media_server. It sends VOICE_SERVER_UPDATE before VOICE_STATE_UPDATE, as the real server does./livekitprefix proxy in front of the locallivekit-serverIt joins and leaves twice. Each time it checks: Connected in under 1 s; the
user-joinchime found in the virtual output (by correlation) within 1.5 s; thevoice-disconnectchime on leave. It also checks that the runtime was built only once (LkRuntime::new()appears once in the log).On the old code it failed on the chime. Without keeping the runtime it measures 0.71–1.21 s per join.
CI
FLUXER_SCREENSHOT_TESTS=1 FLUXER_LIVEKIT_TESTS=1 cargo xtask ci: all steps pass locally.For the real server
What the loopback test cannot show is WAN signalling/ICE time and the gateway's op 4 → VOICE_SERVER_UPDATE time. Join once with the default
infolog: thevoice:lines give each step'ssince_join_ms, so any remaining delay shows which step it is in.🤖 Generated with Claude Code
The server deduplicates users in READY: guilds[].members[].user is {"id"} only and the full users are in the top-level `users` list (session_ready_collect.erl). fluxer-proto required a full user on every member, so a real READY failed to parse as a whole and the client showed no guilds, no DMs and "Unknown User". - fluxer-proto: `UserRef` (full user or `{"id"}` reference) for GuildMember::user, Presence::user (the offline PRESENCE_UPDATE sent when a friend/group-DM subscription is dropped) and GUILD_MEMBER_REMOVE's user (always a reference). `ReadyPayload::resolve_users()` fills the references from `users` and `user`. - Every list of ReadyPayload / GuildReadyData is read leniently: a malformed entry is dropped with a warning instead of losing the session; a malformed guild object leaves `properties: None`. Adds `tracing` to fluxer-proto for that warning. - fluxer-core: members and presences that are references add nothing to the user cache; names come from READY.users, unknown users show as Unknown User. - fixtures/gateway/ready.json now has the real structure (member user references, `users` with every member, relationships without `user`, empty guild presences, sessions/version/...), with the same fake identities. The devserver resolves it before indexing message authors. - tests/replay.rs in fluxer-core: with FLUXER_REPLAY_DIR set, replays a --record recording through the parsers and the store and reports mismatches by key path and redacted error only; skipped otherwise. - Regression tests: references resolved via `users`, an unknown member user id kept, malformed members/channels/users/guilds dropped without failing READY, references in GUILD_MEMBER_REMOVE and PRESENCE_UPDATE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>The real server sends only the user's own member in READY and no guild presences; the other members arrive in GUILD_MEMBER_LIST_UPDATE after an op 14 (LAZY_REQUEST) subscription, which the client never sent. - fluxer-proto: LazyRequest/GuildSubscription/MemberListRange, the GUILD_MEMBER_LIST_UPDATE and GUILD_SYNC dispatches, with a fixture. - fluxer-core: MemberLists sends {active, sync} on guild select and the shown channel's member list ranges, again after every reconnect; keeps the server list and feeds member_list, users and presences from it; falls back to READY's members when no answer comes. GUILD_CREATE and GUILD_SYNC add their partial members instead of replacing them. - fluxer-ui: group headers show the server's count. - fluxer-devserver: READY and GUILD_CREATE carry only the signed-in member; op 14 is answered with a server-shaped member list, and presence changes resend it. New end-to-end test. Follow-ups: #195 (scroll ranges past 1,000 rows), #196 (PASSIVE_UPDATES). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>The runtime of Voice task 14 used a placeholder room that refused every join ("this build cannot join voice rooms yet") until the LiveKit room of task 13 existed. Task 13 merged later and nothing replaced the placeholder, so voice only worked with --devserver. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>