Voice 13: fluxer-voice LiveKit room and interop test (#74) #121
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "voice-13-voice-livekit-interop"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What
crates/fluxer-voice/src/rtc.rs:LiveKitRoom: Roomon the LiveKit Rust SDK (livekit0.10.1) andVoiceSession::livekit(events) -> VoiceHandle.set_subscribedresolves identity + track sid to the publication and tells the server.signalling_base: the SDK pushesrtconto the path; a trailing slash, query or fragment is dropped first, so/livekit/does not become/livekit//rtc).max_bitrateclamped to 8-510 kbit/s, WebRTC's own processing off.RoomEvents.fluxer_voice::apm(Apm,ApmConfig,ApmError): the SDK does expose WebRTC's audio processing module, so it is wrapped with a purei16interface (10 ms, 48 kHz, 1 or 2 channels; other shapes are an error, never a panic). No follow-up issue for spec §12 Q3 was needed.crates/fluxer-voice/tests/livekit_interop.rs:two_sessions_hear_a_tone,mute_and_unsubscribe_stop_audio,data_round_trip_on_negotiation_topic,connects_through_path_prefix, plusroom_receives_only_subscribed_tracks(below the session, which gates audio by itself: the room must not receive an unsubscribed track). An in-test reverse proxy serves the server under/livekitonly and answers 404 elsewhere; tokens are minted with the dev key (livekit-api, dev-dependency,access-tokenfeature only).tools/livekit-test-server.sh: downloadslivekit-serverv1.12.0 (SHA-256 checked) and runs it in dev mode on loopback.xtask: new last steplivekit-interopwhenFLUXER_LIVEKIT_TESTS=1(starts the script unless a server is already listening, runs the tests single-threaded, stops it). Thecargo test --workspacesteps no longer pass that variable on, because-- --ignored(screenshot step) also selects the interop tests, which then skip themselves.FLUXER_LIVEKIT_TESTS=1, a "Fetch livekit-server" step,libglib2.0-dev, and clang 21 from apt.llvm.org withCXX=clang++-21.Cargo.toml:livekit(featurerustls-tls-native-roots),livekit-api.deny.tomlneeded no change (cargo deny checkis green as it is).Stacked on: voice-12-voice-session, voice-2-proto-api-soundboard-rtc
How it was tested
FLUXER_SCREENSHOT_TESTS=1 cargo xtask cigreen locally (base98726e0was green before any change).rtc.rsthat need no server: path prefix, publish options and bitrate clamp, stereo mix-down, token never in errors, reason/source mapping, a room that is not joined, refused E2EE rooms,Apmon real libwebrtc.tools/livekit-test-server.shhave been run since this description was first written:FLUXER_SCREENSHOT_TESTS=1 FLUXER_LIVEKIT_TESTS=1 cargo xtask ciis green locally at10e1104, with xtask starting livekit-server v1.12.0 through the script. (The first version of this text said they had never been run; that held for commit6af1f49only.)2a9d563keeps data packets that arrive before their sender is announced (EarlyData, see the module docs);10e1104makes sure packets whose sender the SDK named are never dropped with them, and moves the round-trip probe to its own task.Deviations from the plan
livekit-signalinghas a TODO for it). The room reports the nominated ICE candidate pair's RTT instead, measured every second; the session still reports every 2 s.Connectwith ane2ee_keyis refused with an error instead of joining unencrypted; Voice 23 wires the key.webrtc-sysrefuses GCC and needs clang >= 21 (the libc++ inside the prebuilt libwebrtc says so), hence the apt.llvm.org step in CI; Ubuntu 22.04's own clang is 14. Voice 25 (packaging) inherits this.ParticipantJoinedis sent on the SDK'sParticipantConnected, which (per the SDK's docs) fires before that participant can receive data messages. The protocol covers it (every client answers the first advertisement of an identity), but an advertisement sent onParticipantJoinedalone may miss a newcomer.fluxer-protois a dev-dependency offluxer-voice(the interop test usesCODEC_NEGOTIATION_TOPICand a realSelectProtocol).logcrate; nothing bridgeslogtotracingtoday, so none of it reaches the log file. If Voice 14 adds a bridge, the SDK's records need the same redaction as ours (Review Focus 7).Windows / sizes
Not built on Windows here; the non-interop tests have no platform-specific code, but the Windows runner is the check. On Linux the prebuilt libwebrtc is 692 MB unpacked inside the target directory (
debug/build/scratch-*/out/livekit_webrtc); the test binaries link it statically and need only libc/libm/libgcc_s at run time. The Windows target-dir size after a clean build still has to be read off the runner: tracked in #127.Closes #74
🤖 Generated with Claude Code
Independent review, Voice 13 (commit
6af1f49, diff from98726e0)Verdict: changes requested. Two blocking findings; both leave the Linux CI job red. Everything else checked out.
Blocking
data_round_trip_on_negotiation_topicfails deterministically against livekit-server v1.12.0 (10 of 10 runs here).crates/fluxer-voice/tests/livekit_interop.rs:246:user_1001_quiet-otter: the expected event did not come within 3s. The other four interop tests pass every time.VoiceEvent::Connectednever arrives. Sleeping 3 s before Bob'sPublishDatamakes the test pass (2 of 2). Sleeping 3 s between Alice's join and Bob's join does not (0 of 2). So the sender is not ready, not the receiver, andpublish_datareports no error.Connectedis dropped silently. EitherLiveKitRoom::connectmust return only once the room can deliver data, orpublish_datamust wait or retry until it can.tools/livekit-test-server.shis committed with mode 100644. The new CI step "Fetch livekit-server" runs it directly (tools/livekit-test-server.sh --fetch), which ends inpermission denied.tools/fj.shis 100755.cargo xtask ciis not affected because it starts the script throughbash. Fix:git update-index --chmod=+x tools/livekit-test-server.sh, or call it withbashin the workflow.Minor
room_receives_only_subscribed_trackscannot tell "the server stopped sending" from "the pump stopped reading": the reader task is aborted onTrackUnsubscribed, so zeroAudioevents follow either way.publication.set_subscribed(false)is called, so the behaviour is probably right; the test just does not prove it (Review Focus 1).ParticipantJoinedmay miss a newcomer; finding 1 shows the same loss is real on the newcomer's own side.Verified
two_sessions_hear_a_tone,mute_and_unsubscribe_stop_audio,data_round_trip_on_negotiation_topic,connects_through_path_prefixall exist, are#[ignore]d and gated onFLUXER_LIVEKIT_TESTS=1. They assert the 440 Hz tone by energy share, mute and unsubscribe stopping it, the round trip onCODEC_NEGOTIATION_TOPICwithin 3000 ms, and requests under/livekit/rtcthrough a proxy that answers 404 elsewhere.LiveKitRoom: Room,VoiceSession::livekit(events) -> VoiceHandle,fluxer_voice::apm(Apm,ApmConfig,ApmError, plaini16). Auto-subscribe off, 48 kHz native source, RED on, DTX off, bitrate clamped to 8 000–510 000 bit/s. No LiveKit type in a public signature.livekit-signaling0.1.9 pushesrtc(andv1) onto the path segments, so the prefix survives;signalling_basestrips the trailing slash first. Confirmed live byconnects_through_path_prefix.fluxer-coreuntouched (core-isolation step green);livekit-apiis a dev-dependency only; the only credentials are the publicdevkey/secretpair; server-driven paths inrtc.rshave no panics (frame shapes are checked, including before the APM call).--dev --bind 127.0.0.1 --node-ip 127.0.0.1work with v1.12.0 on Linux x86_64.cargo xtask ci(Linux, clang 23.1.4 viaCXX=clang++,FLUXER_SCREENSHOT_TESTS=1 FLUXER_LIVEKIT_TESTS=1)Not checked: the Windows build, and the apt.llvm.org clang 21 step on Ubuntu 22.04. Target dir after this run: 22 GB (debug, whole workspace, with both test passes).
Review round 1: what changed (commit
2a9d563)FLUXER_SCREENSHOT_TESTS=1 FLUXER_LIVEKIT_TESTS=1 cargo xtask ciis green locally, with the server started by xtask through the script. The interop suite (now 6 tests) passed 10 of 10 further runs against livekit-server v1.12.0.Blocking
data_round_trip_on_negotiation_topic: fixed inrtc.rs, on the receiving side.DataReceived { participant: None }, and the pump only forwarded packets with a sender. A probe sending one packet every 100 ms from Bob'sConnectedshowed it: packet 0 arrived at Alice without a sender, about 45 ms before herParticipantConnected(bob), and packets 1 onward arrived normally. That is why a sleep before the send helped and a gap between the joins did not.connectreturning later,publish_datawaiting or retrying) can work: the sender cannot see when the others have learned of it. The server announces Bob when his track is published, which is the moment hisconnectreturns. The original client has the same race and drops the packet (if (topic !== PROTOCOL_TOPIC || !participant) return).EarlyDatainrtc.rs. A packet without a sender is kept (at most 2 s, 32 packets, 256 KiB) and delivered as the participant announced next, 150 ms after that announcement and in order with what that participant sends meanwhile. If a second participant is announced within the 150 ms, the sender is uncertain and the kept packets are dropped, as before.EarlyData(early_data_*), and the interop test unchanged apart from a comment.tools/livekit-test-server.shis now 100755.tools/livekit-test-server.sh --fetchrun directly works.Minor
webrtc-sys-build0.3.21 pinsWEBRTC_TAG = webrtc-a545de4; the archive iswebrtc-linux-x64-release.zipfrom thelivekit/rust-sdksrelease of that tag. The archive carries no WebRTC milestone number, so the tag is what is recorded.room_receives_only_subscribed_tracks: fixed. New diagnosticLiveKitRoom::bytes_received()(transport statistics). The test measures what the connection receives in one second while subscribed and again after unsubscribing: 17,484 bytes/s against 34 bytes/s here. It asserts more than 3,000 bytes/s while subscribed and less than a quarter of the subscribed rate afterwards.data_sent_on_participant_joined_reaches_the_newcomer: Alice sends the moment she getsParticipantJoined(bob)and Bob receives it within the 3 s grace period (10 of 10). MappingParticipantJoinedto the SDK'sParticipantActiveis not done, because it changes the order of events the session relies on: #125.Independent review, Voice 13, round 2 (commit
2a9d563, diff from98726e0)Verdict: changes requested. One blocking finding, in the new
EarlyData. Local CI is green and both round-1 blockers are fixed.Blocking
EarlyDatadiscards packets whose sender is known (crates/fluxer-voice/src/rtc.rs,EarlyData::announced, withEarlyData::named).named()queues a newcomer's properly attributed packets in the sameClaim::packetsas the kept unnamed ones. A second announcement within the 150 ms then doesself.claim.take()and throws the whole claim away, named packets included.unnamed(1),announced("user_1_a"),named("user_1_a", 2)returnsNone,announced("user_2_b")50 ms later, andsettlenever hands out packet 2.Decision asked of the reviewer: attribution by inference
Accepted, once finding 1 is fixed. I confirmed in
livekit0.10.1 (Room::handle_data) that the SDK resolves the sender against known participants and dispatchesparticipant: Noneotherwise, so no identity is available. The bounds (2 s, 32 packets, 256 KiB, 150 ms settle) are reasonable, the only topic this client interprets is codec negotiation, and a wrong attribution is overwritten by the sender's next advertisement. Dropping as the original does would fail Review Focus 2 for the newcomer's first advertisement.Minor
Pump::runawaitsmeasure_rtt(up to 1 s, every 1 s) inside the select arm. Ifget_statsstalls, for example while reconnecting, room events and data wait behind it.ParticipantJoinedonParticipantConnected(#125) and the SDK's token-bearing log records (#126) are deferred with linked follow-ups; both exist and are labelled.Verified
two_sessions_hear_a_tone(energy share at 440 Hz, nothing before subscribing),mute_and_unsubscribe_stop_audio,data_round_trip_on_negotiation_topic(realSelectProtocolonCODEC_NEGOTIATION_TOPIC, within 3000 ms, both directions),connects_through_path_prefix(proxy answers 404 outside/livekit; requests seen under/livekit/rtc). All#[ignore]d and gated onFLUXER_LIVEKIT_TESTS=1. Plusroom_receives_only_subscribed_tracks(now with the byte-rate check) anddata_sent_on_participant_joined_reaches_the_newcomer.LiveKitRoom: Room,VoiceSession::livekit(events) -> VoiceHandle,fluxer_voice::apm(Apm,ApmConfig,ApmError, plaini16). Auto-subscribe off, native source at 48 kHz, RED on, DTX off, bitrate clamped to 8 000-510 000 bit/s. No LiveKit type in a public signature.fluxer-coreuntouched;livekit-apiis a dev-dependency only; the only credentials are the public dev pair; server-driven paths inrtc.rsdo not panic;EarlyDatabyte accounting cannot underflow.tools/livekit-test-server.shis 100755; the README recordswebrtc-a545de4; the CI log of my run contains no token.cargo xtask ci(Linux, clang 23.1.4)cargo xtask ciFLUXER_SCREENSHOT_TESTS=1 FLUXER_LIVEKIT_TESTS=1 cargo xtask ciNot checked: the Windows build and the remote runners.
Review round 2: what changed (commit
10e1104)FLUXER_SCREENSHOT_TESTS=1 FLUXER_LIVEKIT_TESTS=1 cargo xtask ciis green locally (interop suite included, server started by xtask through the script).Blocking
EarlyDatadiscards packets whose sender is known: fixed.Claimnow keeps the two kinds apart:kept(the packets without a sender, taken for the newcomer's) andsent(what the newcomer sent under its own name while they waited).kept(and the unnamed packets that arrived since, as before).EarlyData::announcedreturns the first newcomer'ssentpackets and the pump delivers them at once, in order, before the secondParticipantJoined. Nothing waits after that, so the first newcomer's later packets pass straight through behind them.EarlyData::leftreturns itssentpackets and they are delivered beforeParticipantLeft(they were dropped too).settleis unchanged for the reader:keptthensent.packets_with_a_known_sender_survive_a_second_arrival(the finding's sequence:unnamed(1),announced(a),named(a, 2),named(a, 3),unnamed(4),announced(b)at +50 ms yields(a, [2, 3]), nothing later, and both pass straight through afterwards);early_data_of_a_participant_that_left_is_droppedextended for the leaving case;early_data_is_dropped_when_two_participants_arrive_at_oncenow asserts that nothing is handed out when the first one sent nothing.Minor
EarlyDatanow states that packets with a named sender are never dropped.Pump::runawaitedmeasure_rttinside the select arm: fixed. The probe is a task of its own (probe_rtt, in the pump'sJoinSet, so it is aborted with the pump) that sendsRoomEvent::Rttitself; statistics that stall no longer hold up room events or data packets. Interval and timeout are unchanged.ParticipantConnectedvsParticipantActive(#125), SDK log records with the token (#126): no change, deferred to those follow-ups.Independent review, Voice 13, round 3 (commit
10e1104, diff from98726e0)Verdict: approved. No blocking findings. Local CI is green, interop step included. The Windows build and the remote runners are not checked here.
Blocking
None.
Minor
follow-up); the Windows runner is the only check thatwebrtc-sysbuilds there.probe_rtt(now a task in the pump'sJoinSet) is covered only by the interop suite running with it. By reading: it is aborted with the pump, ends when the session stops listening, and no longer sits in the select arm.RoomEvent::Rttis documented inroom.rsas the signalling connection's round trip, whileLiveKitRoomreports the nominated ICE pair's. The deviation is stated in the PR, the README andrtc.rs; the enum's doc comment is the one place that still says otherwise.cargo xtask ciand pass. For Charles to close.ParticipantJoinedonParticipantConnected(#125), the SDK's token-bearing log records (#126).Round 2 finding: fixed
Claimseparateskept(packets without a sender) fromsent(packets the SDK attributed to the newcomer). A second announcement inside the 150 ms drops onlykeptand the unnamed queue;EarlyData::announcedreturnssent, and the pump delivers it before the secondParticipantJoined.EarlyData::leftreturnssent, delivered beforeParticipantLeft.packets_with_a_known_sender_survive_a_second_arrivalruns the sequence from the finding and asserts(user_1_a, [2, 3]), nothing handed out later, and straight pass-through afterwards. It passes.bytesis reset wherever the queue is cleared or drained).Verified
two_sessions_hear_a_tone(energy share at 440 Hz in 100 ms windows; no audio before subscribing),mute_and_unsubscribe_stop_audio,data_round_trip_on_negotiation_topic(realSelectProtocolonCODEC_NEGOTIATION_TOPIC, both directions, each within 3000 ms),connects_through_path_prefix(proxy answers 404 outside/livekit; request seen at/livekit/rtc; trailing slash does not produce//). All#[ignore]d and gated onFLUXER_LIVEKIT_TESTS=1. Extra:room_receives_only_subscribed_tracks,data_sent_on_participant_joined_reaches_the_newcomer.LiveKitRoom: Room,VoiceSession::livekit(events) -> VoiceHandle,fluxer_voice::apm(Apm,ApmConfig,ApmError, plaini16). Auto-subscribe off; native source at 48 kHz; RED on, DTX off;max_bitratefromRoomOptions, clamped to 8 000-510 000 bit/s; remote audio from native streams at 48 kHz; active speakers, track and participant events, data, RTT mapped. No LiveKit type in a public signature.signalling_basekeeps the prefix and strips trailing slash, query and fragment; proven live byconnects_through_path_prefix. Review Focus 2 (this task's half): advertisement on connect and onParticipantJoinedboth arrive within 3000 ms.fluxer-coreuntouched (core-isolation green);fluxer-audiountouched;livekit-apiis a dev-dependency withaccess-tokenonly; the only credentials are the publicdevkey/secretpair; server-driven paths inrtc.rsdo not panic (frame shapes, track sids, RTT values and APM shapes are checked); the SDK error text has the token struck out;deny.tomlunchanged andcargo denygreen.Refs #74and the attribution line; the PR description now says the interop tests are run.FLUXER_SCREENSHOT_TESTS=1 FLUXER_LIVEKIT_TESTS=1 cargo xtask ci(Linux, clean target dir)The CI log of this run contains no token.