TheLanc3/rustdesk
1
0
Fork 0
mirror of https://github.com/rustdesk/rustdesk.git synced 2026-09-19 19:01:05 +03:00
Code Issues Packages Projects Releases Wiki Activity

Webrtc (#15684)

Browse Source 
* feat: add rendezvous WebRTC signaling fields

* feat: route WebRTC ICE on controlled side

* feat: race WebRTC as a direct transport enhancement

* fix: route WebRTC ICE through rendezvous paths

* feat: WebRTC transport racing, DTLS identity binding, and pc-leak fixes

- prefer-P2P racing (race_transports_prefer_webrtc) across punch and RelayResponse; ICE bridge with 400ms candidate resend
- controlled-side answerer and ICE routing; sign local DTLS fingerprint into SignedId, controller verifies the binding fail-closed
- fix pc leaks: close_webrtc() on insecure-decline paths (io_loop, port_forward); compute direct before disarming the offerer guard
- point hbb_common to the WebRTC data-plane commit 9f5a296

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: preserve WebRTC transport preference

* feat: decouple WebRTC from UDP punch, route controlled signaling over TCP

- the WebRTC offer now rides any punch request; only an offer-less request
  may close and reuse the rendezvous socket for TCP punching
  (request_allows_tcp_punch replaces the udp_port-based invariant), with a
  separate offer-less request racing as the TCP fallback
- WebSocket mode no longer disables WebRTC — ws only tunnels the
  signaling/relay legs while ICE stays the only P2P path there; SOCKS proxy
  still disables it (ICE would bypass the proxy and leak the real IP)
- controlled side: WebRTC-only punch replies and trickled ICE candidates go
  over dedicated TCP connections to the rendezvous server instead of the UDP
  mediator channel, for ws/TCP-only hbbs deployments; drop the now-redundant
  rz_sender plumbing and the 400ms candidate re-send on that leg
- guard is_udp handling against responses to requests that advertised no
  udp_port; skip the IPv6 socket bind under force-relay
- test_udp_uat: drop the STUN port race — the punch port must come from the
  rendezvous server's TestNatResponse observing this socket's mapping, a
  STUN probe from another socket can advertise an unreachable port
- bump hbb_common (webrtc 0.13 MSRV pin rationale + upgrade checklist docs)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: KCP/UDP resilience to ICMP resets; optional KCP congestion control

- treat ICMP-driven UDP socket errors (WSAECONNRESET 10054 on Windows,
  ECONNREFUSED on Linux) as packet loss in punch_udp and the KCP pump
  instead of tearing the session down; KCP retransmits through them and a
  truly dead link is still reaped by the pong/app-level timeouts
- resolve STUN hostnames via tokio::net::lookup_host so DNS never blocks a
  runtime worker; fix the inverted non-IPv4 error message
- add enable-kcp-congestion-control option (default on): switch the turbo
  profile to nc=0 so brief loss on constrained links no longer spirals into
  stalls; sender-side only, no wire negotiation
- pin kcp-sys to the rustdesk-patches branch: upstream main lost the
  RustDesk patches on the EasyTier sync, and this branch also wires
  set_kcp_config_factory into connection setup, making the option effective

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: carry switch_code through WebRTC relay fallbacks after rebase

The rebase onto master (switch-code feature) added an 8th request_relay
parameter; pass the interface's switch code from both WebRTC->relay
fallback paths so a role-swap session survives the fallback. Also drop
a duplicate bindgen 0.72.1 entry the Cargo.lock merge produced.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* fix: don't let the preferred branch's own relay preempt a direct fallback

race_transports_prefer_webrtc committed any success from its first argument
outright, on the assumption that it is the WebRTC connect. It is not: the call
site passes a whole punch attempt, which internally falls back to request_relay
when its direct transports fail. That relay was therefore committed instantly
while the offer-less fallback's TCP punch was still in flight — inverting the
preference this function exists to enforce, since the is_p2p predicate the
caller already supplies was applied only to the `others` branch.

Apply it to both branches: a direct result from either side still commits
immediately, and a relayed result from either side is held for the window so
the other side can land something direct. Also commit a held connection when
the surviving branch errors, which the previous code only did on the first
branch's failure path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* fix: evict the oldest pending ICE candidate, not the newest

Candidates arrive in gathering order — host, then srflx, then relay — so a
full buffer was discarding exactly the ones that traverse NAT while keeping
host ones that only work on a shared LAN. Evict from the front instead.

Also document why the controller's ICE bridge must not reconnect on error, in
contrast to the controlled side's per-candidate retry: its socket address is
the return route itself (mangled into PunchHole.socket_addr, echoed back in
IceCandidate.socket_addr, resolved through tcp_punch), so a reconnect would
arrive from an address no route points at, and the server drops the old entry
when the connection closes. Once it dies both directions are dead, and
abandoning WebRTC is the correct response rather than retrying.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* fix: bound log volume on sites whose rate a peer or retry loop controls

Debug output goes to the log file, so a site that fires per received message
or per retry lets someone else decide how much a machine writes to disk. The
WebRTC work added the first such sites.

- KCP io loop: absorbing ICMP errors as packet loss made a broken socket write
  ~100 lines a second for the 60s until the pong timeout reaps it. Log by run
  instead: one line when a run starts, one per ~5s while it persists so a stuck
  socket stays visible, and one on recovery with the total.
- punch_udp: the recv error retries every 10ms for up to MAX_TIME, so one line
  per occurrence wrote thousands per punch. Log the first, report the count in
  the timeout message.
- ICE candidate paths (client, mediator): the peer sets the candidate rate and
  the rendezvous route carrying them needs no prior punch, so throttle to one
  line a minute each with the suppressed count.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* fix: the KCP io throttle reset itself every cycle, so it never throttled

The send and recv arms shared one counter, and an ICMP error on a connected
socket is reported once and then cleared — so the steady state is an
alternation: the send succeeds and clears the counter, the next recv reports
the error and finds the counter at 1, and logs. Every error still wrote a
line, at the ~100/s the previous commit set out to stop, while the
persistent-failure and recovery branches were unreachable.

Use one LogThrottle per direction instead of a hand-rolled counter. That
removes the shared state the bug lived in, drops a third throttling mechanism
in favour of the one already added, and leaves the surrounding `if let Err`
untouched rather than reshaping it into a match.

Also fix test_udp_uat's socket-error arm, the untreated twin of the punch_udp
site: it had no backoff at all, so a persistent error re-armed recv
immediately and spun the loop at CPU speed, one warn line per iteration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* bump kcp-sys: 14 review fixes on rustdesk-patches (6e44b93 -> fa51c15)

Picks up the handshake-recovery work plus the review round on top of it:
ABBA deadlock between the endpoint's two DashMaps, graceful-close tail
truncation, mid-stream hole on ikcp_send failure, FIN retransmission for
lost-FIN half-open hangs, SYN-ACK budget burned on dropped packets,
spurious ConnectTimeout after a completed handshake, accept-backlog
overflow stranding conns, aliasing UB in the output callback, and the
log-facade/throttling cleanup (per-packet sites no longer reach the
debug-level file logger, peer-rate warns throttled).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* ws: decouple ICE policy from force_relay — full-ICE WebRTC over WebSocket

WebSocket support folds into force_relay because a ws tunnel kills
classic TCP/UDP punching — but that conflated transport necessity with
relay policy, and the WebRTC decisions keyed off the merged flag: a ws
client built no offerer at all without TURN, and only a Relay-only-ICE
one with it. ws deployments could never reach a direct WebRTC
connection, which is exactly the path they are supposed to live on.

Split the flag. LoginConfigHandler now tracks policy_relay (the
force-always-relay option, an explicit relay request — /r ids and
retry-via-relay included — and proxy) separately; force_relay stays
policy_relay || use_ws() and keeps governing the classic paths, so
non-ws behavior is unchanged everywhere:

- the offerer's existence and ICE policy follow policy_relay: under
  pure ws the offer gathers every candidate type and may go direct;
  under relay-by-policy it stays Relay-only ICE, TURN-gated, exactly
  as before;
- the RelayResponse race applies the prefer-P2P window under ws (a
  direct ICE path is worth delaying an already-ready relay for) while
  policy relay keeps first-success semantics;
- the request carries webrtc_all_ice (hbb_common 64b54ab) so the
  controlled side knows the offer is full-ICE: it answers with full ICE
  and no TURN requirement, while offers without the bit keep today's
  relay-only answer path on every version-skew combination.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* bump kcp-sys: 7 review fixes on rustdesk-patches (fa51c15 -> 023a006)

Reverts the connect/accept/add_conn changes that regressed concurrent
connects (the state_map guard held across add_conn is load-bearing), states
the single-conn contract on KcpEndpoint so shared-endpoint behaviour stops
consuming review effort, pins the two invariants that keep truncated input
from aborting under panic='abort', and fixes three findings from external
review: sendwnd() echoing raw config instead of KCP's effective window (a
non-positive factory value stalled sending forever), the passive closer's
lost final FIN delaying EOF by up to ~20s, and the doubled window
overflowing for extreme factory values.

Lock-only change: cargo update -p kcp-sys also re-picked libloading's
windows-targets between two versions already present in the lock; that was
reverted to keep this commit to the one line it is about. cargo metadata
--locked passes on the result.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ws: read the all-ICE declaration from the offer envelope, drop the proto field

Companion to hbb_common 68d2729: the full-ICE declaration now lives as
an `ice_policy: "all"` key inside the webrtc:// envelope, so the request
assembly no longer sets webrtc_all_ice and the controlled side asks the
envelope (endpoint_declares_all_ice) instead of a PunchHole field. The
rendezvous server carries the offer opaquely — no forwarding to keep in
sync. Skew behavior is unchanged: an unmarked or unparseable envelope
reads as the old Relay-only semantics.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* add enable-webrtc option; gate test_ipv6 under forced relay

OPTION_ENABLE_WEBRTC (hbb_common 48c2d4d) follows the udp/ipv6 punch
options end to end: default on against the public server, off against
private ones, same settings UI placement on desktop and mobile, and the
same bool2option local-option handling. Gates:

- controller: should_create_webrtc_offerer checks it first — no pc, no
  STUN/TURN gathering, no offer in the request;
- controlled: unlike the udp/ipv6 legs, which deliberately follow the
  request, answering builds a pc that gathers ICE from this host, so
  the answerer honors this machine's own switch too.

Translations for "Enable WebRTC P2P connection" added to all 50 lang
files next to the IPv6 entry (IPv6 and WebRTC are invariant terms in
the same grammatical slot in every one of them).

Also stop probing v6 reachability (test_ipv6) under any forced relay:
the v6 punch socket is never bound there, so the probe was wasted work
on every ws/proxy/relay connection.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* kcp: client-side integration tests over real loopback sockets

kcp-sys has been through two review rounds of behavioral fixes; the
client wrapper (kcp_io pumps, connect/accept deadlines, framed-stream
adaptation, guard lifetimes) had no tests pinning what rustdesk actually
relies on. Four now do, each through real 127.0.0.1 UDP sockets and the
BytesCodec framing sessions use:

- handshake + bidirectional framed roundtrip + graceful close: the peer
  observes end-of-stream instead of hanging (guard outlives the framed
  stream so the FIN goes out);
- a writer that queues 50 frames and closes immediately loses none of
  them - the client-side pin for the close-tail-drain semantics;
- socket errors after the peer vanishes are treated as loss: writes keep
  succeeding, nothing tears down (ICMP is advisory on connected UDP);
- the connect deadline holds when nothing answers.

Mutation-checked: dropping inbound forwarding in kcp_io reddens exactly
the three tests that need the pump, and the timeout test alone stays
green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* ipc/auth: replace the local throttle with the shared throttled_log!

auth.rs predated hbb_common's LogThrottle and grew its own equivalent:
same shape (last_log_at + suppressed), same 5s interval, plus a helper
and three OnceLock<Mutex<..>> statics. It also counted the other way -
excluding the event being reported - so each of the three sites carried
two near-identical log::warn! arms to avoid printing "suppressed 0".

The shared macro covers all of it: one static per call site declared by
the expansion, and the multiplicity suffix appears only when there is
one, which is what those duplicated arms were for. 102 lines out, 27 in.

Behavior difference, deliberate: a burst now reads "(x47)" - the total
including this line - instead of "(suppressed 46 similar events)". One
number, no arithmetic, and one convention across the codebase.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* kcp: make the congestion-control profile opt-in, not the default

The branch had flipped KCP to nc=0 (built-in congestion window) for
every session. That is a transport-behavior change for all users made on
reasoning alone, and the reasoning does not decide it: which profile wins
depends on why packets are being lost.

nc=1 - what RustDesk has always shipped - never shrinks the send window,
so on a genuinely congested uplink it deepens the loss it is reacting to.
But nc=0's backoff is blunt: a fast retransmit halves the window while an
RTO sets cwnd = 1 outright (ikcp.c) and recovery slow-starts from one
packet, so on a link with random loss and no congestion - Wi-Fi
interference, a long-haul path - it reads loss as congestion and can
stall an interactive stream for seconds. That failure mode is also the
more visible one to a remote-desktop user.

No benchmark settles this either: a loopback A/B has no bottleneck queue,
hence no congestion to control, and would flatter nc=1 by construction.
Deciding it needs a shaped link or field data.

So keep the profile users already run and let the other one be asked for
("enable-kcp-congestion-control" = "Y"). Flipping the default later is a
one-line change once there is evidence. kcp-sys keeps its own test
covering the nc=0 path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* android: define getifaddrs/freeifaddrs for the api-21 sysroot

Turning on hbb_common's "webrtc" feature pulls webrtc-util into the android
link, and its ifaces() -- reached from vnet::Net::new() on every ICE gather --
calls getifaddrs(). bionic exports getifaddrs/freeifaddrs only from API 24,
while flutter/ndk_*.sh builds against --platform 21, so every abi failed to
link on the undefined symbols.

Raising the platform to 24 would have to drag minSdkVersion 22 with it and
turn the link error into a load-time one on Android 5.1/6.0, so define the
two symbols instead, using the RTM_GETLINK + RTM_GETADDR netlink dump bionic
itself uses. The definition also shadows bionic's on API >= 24 rather than
delegating to it, so the path that ships is the path every test device runs.

Checked against synthesised netlink dumps on the host -- link/address parsing,
prefix masks, point-to-point, ipv6 scope ids, malformed and truncated messages
-- under UBSan and byte-exact guard malloc, with a deliberately unsigned
remainder as the negative control.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix three ways ws + WebRTC could not work in practice

Review of #15684 and hbb_common#579. Each of these left the code reading
correct while the feature did not function.

- The RelayResponse race classified P2P with `result.2 == "IPv6"`, but
  that site's futures are only ever the relay ("Relay"/"WebSocket") and
  the WebRTC branch's own "WebRTC" — so the predicate was constantly
  false. When the relay landed first the result was still right (the
  webrtc arm's `others_fut.is_none()` fallback), but when WebRTC
  connected FIRST it was parked as if it were a relay and the relay was
  committed on arrival, discarding a live direct connection. That is the
  LAN case: the better the network, the worse the outcome. Classify by
  what the label means, via is_direct_transport, and test both orderings
  — only the relay-first one was covered.

- handle_peer_info wrote "force-always-relay=Y" into the peer's saved
  config whenever force_relay was set, which now includes the WebSocket
  transport. One ws session therefore turned the peer into a permanent
  relay-by-policy peer, and relay-by-policy means Relay-only ICE, so
  WebRTC could never go direct to it again — the flagship path worked
  exactly once. Persist policy_relay, which is the user's choice; the
  transport is a property of this client, not of the peer.

- The answerer gated on this machine's enable-webrtc option, but that is
  LocalConfig: the UI process writes it and never syncs it over IPC,
  while handle_punch_hole runs in the server process, which on Windows
  resolves LocalConfig under a different profile and reads the
  private-server default of "N". The gate refused to answer in exactly
  the self-hosted deployments the transport exists for. Drop it: the
  answerer follows the request, like the udp/ipv6 legs, and the option
  still gates the feature where it can — an offer only exists because
  some controller had it enabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* webrtc: close without an await point; do not report an unknown path as direct

- close_webrtc is no longer async (hbb_common 88f965f), so the ten call
  sites in port_forward and io_loop - all inside select! arms or futures
  the UI can abandon - can no longer be cancelled mid-teardown, which
  left the pc unclosable and its session entry stranded. Client's own
  spawn_close_webrtc went with it: the runtime-teardown guard it existed
  for now lives in close_detached, so both Drop paths share one
  implementation.

- webrtc_relayed() returns None when no candidate pair is selected or
  the pc closed under a concurrent teardown, and both call sites read
  that as "not relayed", i.e. direct. A TURN-relayed session could
  therefore be shown to the user as peer-to-peer. Claiming a direct path
  needs evidence of one, so an unknown answer now counts as relayed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* scrap/benchmark: give the Duration divisor an explicit u32

The webrtc feature pulls time 0.3 into scrap's graph (hbb_common ->
webrtc -> webrtc-dtls -> der-parser -> asn1-rs), and that crate carries
an `impl Div<time::Duration> for std::time::Duration`. Orphan rules
allow it because the RHS is its own type, and trait impls are visible
across the whole dependency graph without a use, so std::time::Duration
now has two Div candidates. `yuv_count as _` casts to a plain inference
variable, which both candidates fit, so it stops resolving:

  error[E0282]: type annotations needed
    --> libs/scrap/examples/benchmark.rs:146:33

Only two of the four sites are reported - rustc emits one E0282 per
function body - so all four are annotated. The already-explicit
`as u32` at the hwcodec site and `start.elapsed() / cnt` are unaffected,
the latter because an integer literal's variable can only unify with an
integral type and rules the time impl out on its own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* webrtc: judge the race by the resolved path, not the label; bound the ICE queue

Third review round. Two of these are regressions from the previous one.

- The RelayResponse race predicate was `is_direct_transport(result.2)`,
  which answers true for the label "WebRTC" - but WebRTC is only a
  direct path when ICE nominated a non-TURN pair. A TURN-relayed WebRTC
  result therefore committed instantly and cancelled the IPv6 attempt
  racing beside it, which is the same inversion the previous fix removed
  in the other direction. (That fix was also argued from a wrong premise:
  the site does carry an IPv6 future, pushed ~50 lines earlier than the
  relay one.) Each future now resolves whether its path is direct and
  the predicate reads that bool, matching the outer race, and the
  downstream recomputation goes away.

- policy_relay still folded in Config::is_proxy(), and that is what gets
  persisted into the peer's config as force-always-relay - so one
  session through a proxy pinned the peer to relay forever and disabled
  WebRTC for it, exactly the latch the previous round fixed for
  WebSocket. Split out peer_relay: the saved option or an explicit
  request for THIS peer, and the only part written back.

- The controlled side buffered remote ICE candidates in an unbounded
  channel while the controller caps the same buffer at 64, and draining
  one costs a JSON parse plus the ICE agent's lock. Whoever can reach a
  session's route could grow it without limit inside the long-lived
  service process. Bounded, with the overflow logged through the
  existing throttle.

- That route was also removed by key alone when an answerer finished, so
  a punch retry that built a fresh answerer under the same fingerprint
  had its live sender deleted by the previous one's cleanup - after
  which it received no candidates at all. Evict only our own sender, the
  way the session cache already guards the analogous case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* webrtc: trim the comments to AGENTS.md length; drop is_direct_transport

386 added comment lines down to 287 across client, mediator, kcp_stream
and common. Same rule as hbb_common 3d64e43: out go past-bug narration,
rejected alternatives, measurements and restatements of the code; the
non-derivable why stays.

is_direct_transport goes with them. Judging the race by a transport
label was replaced by the resolved direct flag, leaving it used only by
its own test — and, having been inserted between the doc comment and
race_transports_prefer_webrtc, it had also taken that function's
contract with it. Removing it reattaches the doc where it belongs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* webrtc: fix race edge cases that discard or mislabel a direct connection

Three correctness fixes in the transport race, plus three convention
cleanups.

- race_transports_prefer_webrtc committed a relayed result while a direct
  attempt was still in flight: the others arm returned on
  webrtc_fut.is_none() even with an unfinished direct future, and the
  WebRTC-error arm returned a held relay without checking others_fut. A
  relay is now committed only when nothing direct can still arrive (or
  the window expires); a parked relay is also preferred over composing
  an error when both sides fail. Three regression tests, mutation-checked.

- connect()'s plain select_ok let a TURN-relayed WebRTC win as "first
  success", dropping still-racing UDP/IPv6 direct attempts and reporting
  the relayed pair as direct. It now runs through the same prefer-P2P
  race with each attempt carrying whether its path is direct, and the
  WebRTC future resolves is_relayed() so a TURN win is held behind
  direct attempts, not committed as one.

- The RelayResponse path kept direct == true when a WebRTC win's DTLS
  handshake failed and it fell back to relay, so the relay was reported
  P2P. Clear the flag with the transport switch.

- Trim the OffererGuard doc to the three-line max; move the new
  enable-webrtc localization key to the end of every lang list; the KCP
  option constant moved to hbb_common config::keys (0f663aa).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExUfAkYbq8UC9pQCiLy8TQ

* bump hbb_common: WebRTC peer connections own their I/O runtime

Closing the controlling window left the controlled side waiting out
ICE decay — ~25-30s in the peer's log, its disconnected/failed ladder
running to completion — where TCP delivers a FIN at once. The session
end closed the pc by spawning onto io_loop's own
`#[tokio::main(flavor = "current_thread")]` runtime, which is dropped
the moment io_loop returns, and nothing after that call yields: the
task was never polled even once, so no DTLS close_notify ever left.

Every attempt to fix that on the caller's side failed the same way,
because the mismatch was never about where the close ran: a pc's UDP
sockets register with the reactor, and its ICE/DTLS/SCTP pumps spawn
on the runtime, that is current while it is built — so a pc created
by a session outlives the only runtime that can drive its I/O, and a
close driven anywhere else completes without reaching the wire.

The bump homes them where they can outlive any caller: WebRTCStream
builds on a process-lifetime runtime and every detached close runs
there as its own never-cancelled task. io_loop keeps its plain
close_webrtc() calls and only documents why nothing here may spawn or
await the teardown on the dying session runtime.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HV43uh1ztv6Wm5qi3Y1ne

* fix: give the UDP NAT test a real window when the TCP clock is faked

The punch request carries udp_port only if the rendezvous server's
TestNatResponse has arrived, and the wait for it was bounded by
rtt / 2 — half the TCP connect time, on the assumption that TCP and
UDP round trips are comparable and the test, started earlier, has
already answered.

A transparent TCP proxy breaks that assumption: a TUN-mode VPN on the
host, or a redirect-mode proxy on the LAN gateway serving every device
behind it, completes the handshake locally in ~3ms while the real UDP
round trip is hundreds of ms. Log-confirmed against 5.161.65.208: ping
341ms, TCP connect 3.7ms, connect to a dead port there "succeeds" just
as fast. The window collapsed to ~1.5ms, udp_port stayed 0 on every
attempt, and UDP punch was never even requested — although UDP itself
passes such gateways untouched.

So use the TCP clock only when it is believable: below a plausible WAN
round trip it says nothing about the UDP path, and a flat ceiling
applies instead. The loop still exits the moment the port arrives, so
a genuinely nearby server pays nothing and only a UDP-dead network
waits out the ceiling — on the udp-carrying round alone, while the
parallel pure-TCP round is unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HV43uh1ztv6Wm5qi3Y1ne

* feat: make the TCP punch a user option, with TCP as the backstop

TCP punching was the one direct transport without a switch, while UDP,
IPv6 and WebRTC each had one. Add "Enable TCP hole punching" above the
UDP toggle on both desktop and mobile, default on — including on
self-hosted servers, since unlike the other three (whose default-off
there guards against an hbbs that cannot forward their fields) TCP
punching has always been supported by every server.

Turning all four off would leave no way to punch at all, so TCP runs
regardless in that case. That backstop keys off the switches alone: a
transport that is enabled but fails to materialize — no public v6
address, no NAT port, a failed offerer — is already covered by the
relay fallback for a round that ends up with no usable direct
transport. With the TCP punch off, the fallback request is skipped
too: it exists only to carry that punch, and would otherwise reach
connect() with nothing to try and merely open a second relay.

Known cost, unchanged behavior for the peer: the request carries no
field for this choice, so a peer that receives one with no udp_port and
no offer still punches a TCP hole and listens for a connection the
controller will not make. Representing the transport choice on the
wire needs a proto field and the server forwarding it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HV43uh1ztv6Wm5qi3Y1ne

* bump hbb_common: name the punch by every transport it carries

`get_local_endpoint_trickle` became `local_endpoint() -> &str`, which
cannot fail, so both call sites lose an unreachable error arm — the
mediator's closed a pc against a failure that no longer exists.

`punch_type` named one transport, and picked it off `allow_tcp_punch`.
A round carries several at once — a NAT port and a v6 address and an
offer — and since the TCP punch became a switch it can carry none, so
one name had to misreport both: the logs of the round that broke WebRTC
read "#1 UDP punch attempt" while the request also carried the v6
address and the offer that was actually failing, and a round with
nothing to punch with was labelled "WebRTC". List them instead —
"UDP+IPv6+WebRTC" — and call the empty round "Relay", which is what it
can still end as and what `typ` prints for it.

The offer is moved into the request rather than cloned into it; that
was its last use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019UzcMTdYTEv2QbMHcTSUy3

* bump hbb_common: drop link-local IPv6 from ICE gathering

Also pin webrtc-util to a fork of 0.11.0 carrying a Windows IPv6 enumeration fix.
`ifaces` reads the adapter list's on-wire IPv6 bytes as host-order `[u16; 8]`, so on a
little-endian host every group comes out byte-swapped and unbindable: a peer's real
240e:369:9606:4600:f52a:7a8d:2530:4de0 is enumerated as e24:6903:696:46:2af5:8d7a:3025:e04d,
::1 as ::100 and fe80:: as 80fe::. Each fails to bind with WSAEADDRNOTAVAIL, so ICE gathers
no IPv6 host candidate at all on Windows - where a globally routable address is the one
NAT-free path a CGNAT'd peer has.

Never reported upstream; the unix twin of the same bug was fixed in webrtc-rs#475 (2023).
Fork: rustdesk-org/webrtc, branch rustdesk-patches, tag webrtc-util-0.11.0-win-ipv6.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019UzcMTdYTEv2QbMHcTSUy3

* bump hbb_common: name the family a WebRTC session runs over

`stream_type` reaches the UI as the transport that won the race, and every other transport
already carries the family in that label - the v6 punch reports `IPv6`. WebRTC does not: one
label covers both families, and it is the one path whose real remote address can differ from
the rendezvous-observed one the session is identified by.

Refine it at the hand-off to the UI rather than at the source: five sites in client.rs
compare `typ == "WebRTC"`, so widening the label there would silently move control flow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019UzcMTdYTEv2QbMHcTSUy3

* bump hbb_common: one STUN list, and drop the dead IPv4 half

`test_ipv6` kept its own hand-written copy of the STUN servers. It now reads
`WebRTCStream::stun_servers()`, so an operator who points OPTION_ICE_SERVERS at their own
server gets it on both paths instead of one.

`test_bind_ipv6` sends nothing - `connect` only makes the kernel pick a route and a source
address - so the whole cost is DNS. It races the lookups rather than betting this host's
IPv6 support on whether the first entry happens to publish a AAAA where the user resolves
from; google's does not, from a Chinese resolver, and it was the entry being bet on.

`stun_ipv4_test`, `STUNS_V4` and `test_nat_ipv4` have had no callers since the punch stopped
taking its port from a second socket, and go.

`get_kcp_cc_enabled` reads the renamed option through `option2bool`, like every other one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019UzcMTdYTEv2QbMHcTSUy3

* webrtc: take dcsctp's retransmission timings and IPv6-safe MTU

webrtc-sctp ships RFC 4960's RTO.Initial/RTO.Min (3000/1000), TCP's values for
arbitrary public paths. On this workload they set the recovery time outright:
a request/response exchange keeps one chunk in flight, so no later SACK ever
raises miss_indicator to the 3 that arms fast retransmit, and the T3 floor is
the only way back. A single loss during a handshake or a first keyframe
therefore costs whole seconds.

The fork now carries dcsctp's numbers instead - the SCTP implementation Google
wrote to replace usrsctp for Chrome's WebRTC data channels, the same realtime
workload: rto_initial 500, rto_min 400, a 220ms floor under the RTT variance,
and mtu 1191. INITIAL_MTU 1228 plus DTLS/UDP/IPv6 overhead is 1313, past the
1280 minimum, so every full-size chunk fragmented on an IPv6 path.

Both patch entries move to the new branch, which also carries the Windows IPv6
byte-swap fix, so one rev matches the whole webrtc 0.13 stack.

* udp: make the punch prove itself, and keep the listener answering

punch_udp sent a zero-length datagram and called the hole open on whatever
arrived next. The rendezvous NAT test's own leftover replies satisfy that
immediately - connect() does not flush the receive queue - so the retry loop
never ran and success meant nothing. The dead socket then cost KCP its full
timeout to rediscover, which is how a failed punch came to take 18 seconds.

Probes now carry a magic and a 64-bit transaction id, and both ends answer
each other's probes, so returning is a fact: a reply echoing our own id is the
one thing that proves the pair carries traffic both ways. With failure now
distinguishable from 'not yet', the window drops from 20s to 3s.

Two asymmetries fall out of that:

Only the connector stops on its own acknowledgement, because only it has
something to send next. An acknowledgement proves our probe came back, not
that the peer's probe was answered - and after punch_udp returns nothing
answers probes any more, since KCP's io loop drops anything shorter than its
header. A listener that stopped there would go mute while a peer whose own
probe or answer was lost - the normal state of a hole still opening - kept
probing an endpoint that works, until it timed out.

So the listener stops on the peer's first real packet instead, and hands that
packet to KcpStream::accept as its init_packet: its arrival proves the pair as
well as an acknowledgement would, and KCP never retransmits its SYN.

* webrtc: correct the RTT variance floor to dcsctp's scaling

The earlier commit took dcsctp's min_rtt_variance = 220 as a raw floor under
rttvar. dcsctp divides the option by kHeuristicVarianceAdjustment = 8.0 first,
a historical accident it kept because downstream users had measured good
values with it, so the intended floor is 27.5ms of variance contributing 110ms
to RTO. Flooring at 220 contributed 880ms instead, which on a 50ms path left
RTO within 7% of the 1000ms default this change exists to escape.

The fork also now records why T1/T2 share T3's RTO manager here, unlike
dcsctp's separate control timers: RTO_INITIAL is the T3 value for the first
DATA chunk, since no RTT sample exists before the first SACK.

* webrtc: skip the controller's ICE re-send instead of queueing it twice

The controller sends every candidate twice, because the server's hop to a
peer registered over UDP can lose one. The ICE agent that dedups repeats
sits downstream of the answerer's queue, so the answerer paid for both
copies: a slot, a JSON parse, and the ICE agent's lock, once per repeat.

Remember a digest of what was queued and skip the repeat. Recorded only
once queued, so a candidate a full queue refused stays repairable by the
re-send.

The queue's depth is unchanged. A real peer gathers well under it - four
STUN servers, link-local IPv6 filtered, one component - and the drain
empties it as candidates trickle in, so what this removes is the redundant
work, not an overflow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* tcp: repeat the punch across the controller's dial window

The single punch leaves before hbbs has told the controller where to dial, so
it is never in flight at the same time as the controller's SYN: it opens our
NAT, meets nothing, and a gateway that answers it with RST takes the mapping
down with it, leaving the listener waiting on a hole that no longer exists.

Punch again while the controller may still be dialing, and race those punches
against the accept. That is two ways in where there was one: the mapping is
rebuilt if a RST took it, and once the controller sits in SYN_SENT one of the
punches meets its SYN and completes as a simultaneous open - which a punch sent
before the controller had been told anything never could. The crossing reaches
the punch rather than the listener because the two sockets share the address
but only the punch matches the four-tuple, which the tests now pin down.

There is no instant to aim at, and no window either. `Client::connect` sizes
the controller's dial only after our PunchHoleSent, from its own rendezvous
time and the direct failures it has recorded for us: CONNECT_TIMEOUT between
two known-asymmetric NATs that never failed, punch_time_used times three or
six otherwise, floored at a second - so a peer that failed once dials for a
second or two from then on, and none of that reaches this side. The repeats
therefore cover our own ceiling instead, CONNECT_TIMEOUT, which is exactly as
long as the accept has always been willing to take a connection through the
hole, and back off across it: dense at the start, where every window begins
and the short ones end, sparse afterwards, which is `punch_udp`'s shape for
the same reason. A window past that ceiling was lost before this change too,
and mostly to the controller's own kernel - Windows gives a SYN up at 21s,
Linux's next re-send after 15s is at 31s; a window short of it costs a few
SYNs to a port already closed.

No punch is cut on a per-attempt timeout; one in flight is bounded only by
the shared deadline plus PUNCH_GRACE. A punch is cancel-safe only while it is
still in SYN_SENT; once the controller's SYN has crossed it the socket is half
way through a handshake, and cutting it there cuts the connection the
controller is opening - whose `connect` has already returned, so that attempt
fails outright, there being no relay fallback after a failed TCP handshake. A
timer cannot tell the two states apart, and none is needed: a gateway that
answers with RST fails the connect at once and the loop punches again, while
one that drops the SYN in silence leaves the socket in SYN_SENT, holding the
mapping open while the kernel re-sends, which any SYN of the controller's then
crosses - a second punch has nothing to add. The deadline decides whether
another punch starts; one in flight runs a grace past it, enough for a
crossing begun just before it to complete. The last sleep is cut at the
deadline rather than run out past it, so the window ends on a punch given
that grace and not on a gap of up to the backoff ceiling: the controller's
window opened after ours, on the PunchHoleSent hbbs relayed, so one as long
as ours is still open through our tail.

Only the accept races the punch, never `accept_connection`: that one does not
return until the session it goes on to run has ended, so racing it would tear a
live session down.

Whichever arrives first is the one connection the request produces. `meta`
carries the control permissions hbbs granted for this one controller, so
serving the loser as well would hand them to a second peer - and nothing about
a connection tells the two apart before `create_tcp_connection` has spoken to
it, least of all its address: a carrier NAT shares one between subscribers,
and a NAT that pools its external addresses may dial us from a different one
than hbbs saw the controller through. So the address is not checked, as
`accept_connection` never checked it; the handshake says who arrived, and what
holds the invariant is that there is no second serve. Those
permissions are a ceiling and not a grant either way: `Connection` gates every
message on `authorized`, and latches the login scope of the first request it
accepts, so a peer that reached the hole still arrives with nothing.

The accept loops rather than taking a single connection, so that a transient
accept error does not spend the window the controller still has to arrive in.

libp2p's DCUtR reaches the same place by having both peers dial at one instant
agreed over the relay. Nothing we send reaches the controller directly, so we
cover its dial window rather than name an instant inside it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* hbb_common: bump to the webrtc branch rebased on main

Picks up upstream's session-cache eviction by pc identity (#589, adopted without its
unused insert-path helper), the 90-day log retention, and the wlroots output fixes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* webrtc: send over SCTP without a congestion window, as KCP does

The same link that streams over KCP crawls over WebRTC. webrtc-sctp runs
RFC 4960's AIMD: a fast retransmit halves cwnd, a T3 drops it to one MTU, and
slow start only rebuilds it while data is queued behind it. Where the loss is
random rather than congestion - a lossy long-haul link - the rate settles at
the Mathis ceiling MSS/(RTT*sqrt(p)) however idle the link is: about 1.3 Mbps
at 70ms RTT and 1% loss, 0.6 Mbps at 5%, while 1080p wants 2-5 Mbps. KCP's
turbo profile (nc=1) has no congestion window at all.

The fork now carries a switch that bypasses the two places gating sends on
cwnd, and hbb_common turns it on for every peer connection unless
`allow-webrtc-congestion-control` is set - the same opt-in KCP has in
`allow-kcp-congestion-control`, for the reason at `get_kcp_cc_enabled`.
Sender-side only; a browser or an older build on the other end interoperates.

Measured over a simulated link (35ms one-way, random loss both ways, 12 KB
frames at 30fps, 300 frames): at 1% loss the window stretches 9.9s of video to
20.7s with a mean latency of 5.5s; without it the stream stays realtime at a
mean of 113ms. At 3%: 47s and 15s against 10.2s and 290ms.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* webrtc: take the fork's loss recovery for sending without a congestion window

rustdesk-org/webrtc 825a0a48: without a congestion window a chunk is lost
once three chunks sent after its latest transmission are acked, counted in
send order so retransmitted chunks are covered too, and the fast retransmit
sends every lost chunk at once, as KCP nc=1 does; before, a lost
retransmission waited for T3-rtx. Also fixes the delayed SACK timer never
re-arming, the switch applying to established associations, T3-rtx
resending one chunk when the peer's window is full, and bounds new data to
1 MiB / 1024 chunks in flight like KCP's snd_wnd.

Simulated 35ms one-way, random loss both ways, 30 fps, frames later than
200ms out of 1200: 12 KB at 5% loss 996 -> 55 (KCP 61); 40 KB at 2% loss
1183 -> 20 (KCP 39).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* bump hbb_common: decode TURN userinfo, add the webrtc_echo example

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXJJGEGdgu26wgCppvUXdZ

* web: show the WebRTC toggle and transport in the web UI

The web client now speaks WebRTC, but the desktop settings page hides
the punch options on web and the remote page opens without the session
tab that carries the transport name. Let the existing "Enable WebRTC P2P
connection" checkbox through on web (the other punch options stay
native-only), and add a Transport row to the quality monitor for WebRTC
sessions only (with "(TURN)" when ICE relayed), on every platform.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXJJGEGdgu26wgCppvUXdZ

* bump hbb_common: end the ICE forwarder at gathering complete, drop the closes Drop covers

hbb_common now closes the local-candidate channel when gathering
completes, so the controlled side's forwarder in spawn_webrtc_answerer
ends there, and its signaling connection to hbbs with it, instead of
sitting on a socket hbbs closed at 90s idle for the rest of the session.
It also keeps the reassembly buffer across fragmented frames.

Stream closes the WebRTC peer connection on drop (hbb_common b0b624d),
so the close_webrtc() calls in port_forward and io_loop that sat
immediately before a return or the end of scope did nothing Drop was
not about to do, while the comments beside them still said a bare drop
leaked the pc. Remove both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* bump hbb_common: quiet the webrtc-rs warnings that describe the race's normal outcome

Cancelling the transport that lost the race, and trickle checking before it
holds a pair, are what the design does on every session that connects - and
webrtc-rs reports both at warn, 90 lines of a 386-line controlled-side log,
beside connections that succeeded. agent_internal and peer_connection drop to
error; agent_gather keeps warn, since an unreachable STUN server is the one
upstream signal that explains a session which never connected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M54JAqUK4RynudFou89hod

* port_forward: restore the `?` the close removal left as a match

Dropping the explicit close_webrtc() from the parse-error arm left a match
that only re-spells `?`; master just reworked this function, so the branch
now leaves port_forward.rs untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* l10n: the two WebRTC keys were missing from Urdu

Every other lang file on the branch carries them; ur.rs was skipped when
they were added.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* udp: make the punch deadline absolute, so a talking peer cannot defer it

`select!` rebuilds every arm each iteration, so the relative retry sleep was
restarted by each datagram that arrived before it fired. The peer sets that
rate, and an old-build peer's empty datagrams match no arm and loop without
even the recv-error pause, so MAX_TIME went unchecked and the retransmit was
starved with it. `udp_nat_connect` awaits the punch ahead of the KCP timeout
and nothing above it bounds the phase, so the punch held the direct race open
and the relay fallback out of reach for as long as the peer kept sending.

Absolute instants for both clocks. The new test floods empty datagrams for
four times the deadline: the punch now ends at 3s where it ran the full 12s.

Also note at the symmetric-NAT branch that WebRTC not following the legacy
relay decision there is deliberate, so it is not later "fixed" into agreement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* bump webrtc fork: MTU-safe bundles, a reordering window, tail loss within the RTT

rustdesk-org/webrtc cc6633bc, three commits on 825a0a48, all on the path
that sends without a congestion window:

Both bundlers counted a DATA chunk by its payload alone; with the header and
padding counted, bundles of small chunks stay within the MTU, and the fragment
payload rounds down to 1160 so a full chunk does too. A chunk is fast
retransmitted at most five times, KCP's IKCP_FASTACK_LIMIT.

A frame's chunks go out within microseconds of each other, so on a path that
jitters the send-order rule resent every chunk that landed behind three of
its siblings: 2.7x the payload on the wire at 10ms of jitter, and on a link
without the room for that, a queue that fed on itself. A reordering window,
RACK's, makes evidence count only from what was sent a quarter of an srtt
after the chunk once the path is seen to reorder, widening on the duplicate
TSNs the receiver reports. 5 Mbps, 1% loss, 20ms jitter: 600 of 600 frames
at a 98ms mean where 290 arrived at 6.2s.

A chunk lost at the tail of a burst has only T3-rtx, which ran from floors
sized for a 200ms delayed ack and restarted only on the tail's predecessor's
ack: 600ms and more. Every DATA chunk now carries the I bit, the floors are
KCP's shape, and a fast retransmission restarts the timer. One 200-byte
message per frame at 5% loss: 9 of 600 later than 200ms, from 42.

Random loss without jitter is unchanged at every rate and frame size.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* bump webrtc fork: T3-rtx restarts only for the earliest chunk's fast retransmission

rustdesk-org/webrtc 2b8e55bc. Sending without a congestion window, a fast
retransmission of any chunk restarted T3-rtx, so a chunk past the fast
retransmission cap - left to that timer - never reached it while later
chunks kept being resent, which a lossy stream does every couple of frames.
The timer is the earliest in-flight chunk's, and only its resend restarts
it now. Nothing else changes; the benchmark is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

* bump webrtc fork: T3-rtx restart on fast retransmission while shutting down too

rustdesk-org/webrtc 48100bf1. The restart for the earliest chunk's fast
retransmission reached only the Established branch of the write loop; the
shutdown states still carry data in flight and recover it the same way, so a
closing association could still resend everything on a loss its fast
retransmit had already recovered. Both branches share one helper now.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019aokqJuhjvB3kijXtAg5Ns

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
RustDesk
2026-09-06 00:21:28 +08:00
committed by GitHub
parent 3fc11c0f81
commit ae6af2de43
71 changed files with 3111 additions and 541 deletions
Download Patch File Download Diff File Expand all files Collapse all files

759
src/rendezvous_mediator.rs
View File

@@ -1,4 +1,6 @@
use std::{
collections::{hash_map::RandomState, HashMap, VecDeque},
hash::BuildHasher,
net::SocketAddr,
sync::{
atomic::{AtomicBool, Ordering},
@@ -21,8 +23,13 @@ use hbb_common::{
rendezvous_proto::*,
sleep,
socket_client::{self, connect_tcp, is_ipv4, new_direct_udp_for, new_udp_for},
tokio::{self, select, sync::Mutex, time::interval},
tokio::{
self, select,
sync::{mpsc, Mutex},
time::interval,
},
udp::FramedSocket,
webrtc::WebRTCStream,
AddrMangle, IntoTargetAddr, ResultType, Stream, TargetAddr,
};
@@ -47,7 +54,66 @@ lazy_static::lazy_static! {
static ref SOLVING_PK_MISMATCH: Mutex<String> = Default::default();
static ref LAST_MSG: Mutex<(SocketAddr, Instant)> = Mutex::new((SocketAddr::new([0; 4].into(), 0), Instant::now()));
static ref LAST_RELAY_MSG: Mutex<(SocketAddr, Instant)> = Mutex::new((SocketAddr::new([0; 4].into(), 0), Instant::now()));
static ref WEBRTC_ICE_TXS: Mutex<HashMap<String, IceRoute>> = Default::default();
static ref ICE_DIGEST_STATE: RandomState = Default::default();
}
/// Remote ICE candidates buffered per session while the answerer applies them. Same depth as the
/// controller's own buffer (`Client::MAX_PENDING_WEBRTC_ICE`), though that one evicts its oldest
/// where a full channel here refuses the newest.
const MAX_PENDING_REMOTE_ICE: usize = 64;
/// Queued candidates remembered so the controller's re-send is skipped instead of taking a slot
/// of its own. Far more than an honest peer gathers, at eight bytes each.
const ICE_DEDUP_WINDOW: usize = 256;
// The rendezvous ICE route is reachable without a prior punch and the peer decides how many
// candidates it sends, so these sites would let someone else set how much this machine writes to
// its log file. One line a minute each, carrying the suppressed count.
const ICE_LOG_INTERVAL: std::time::Duration = std::time::Duration::from_secs(60);
static UNKNOWN_ICE_SESSION_LOG: hbb_common::log_throttle::LogThrottle =
hbb_common::log_throttle::LogThrottle::new(ICE_LOG_INTERVAL);
static REJECTED_REMOTE_ICE_LOG: hbb_common::log_throttle::LogThrottle =
hbb_common::log_throttle::LogThrottle::new(ICE_LOG_INTERVAL);
static FULL_ICE_QUEUE_LOG: hbb_common::log_throttle::LogThrottle =
hbb_common::log_throttle::LogThrottle::new(ICE_LOG_INTERVAL);
struct IceRoute {
tx: mpsc::Sender<String>,
recent: VecDeque<u64>,
}
impl IceRoute {
fn new(tx: mpsc::Sender<String>) -> Self {
Self {
tx,
recent: VecDeque::new(),
}
}
/// Keeps `queue` the only way onto the channel, so nothing reaches it unrecorded.
fn is_same_channel(&self, other: &mpsc::Sender<String>) -> bool {
self.tx.same_channel(other)
}
/// Skip the controller's re-send of a candidate already queued: the ICE agent that dedups
/// repeats is downstream of this queue, so the copy would spend a slot of its own.
/// False means the candidate was dropped.
fn queue(&mut self, candidate: String) -> bool {
let digest = ICE_DIGEST_STATE.hash_one(candidate.as_str());
if self.recent.contains(&digest) {
// Only honest about the drop if the route is still alive to have taken it.
return !self.tx.is_closed();
}
// Recorded once queued, never before: a refused candidate stays repairable by the re-send.
if self.tx.try_send(candidate).is_err() {
return false;
}
if self.recent.len() >= ICE_DEDUP_WINDOW {
self.recent.pop_front();
}
self.recent.push_back(digest);
true
}
}
static SHOULD_EXIT: AtomicBool = AtomicBool::new(false);
static MANUAL_RESTARTED: AtomicBool = AtomicBool::new(false);
static SENT_REGISTER_PK: AtomicBool = AtomicBool::new(false);
@@ -399,6 +465,30 @@ impl RendezvousMediator {
allow_err!(rz.handle_intranet(fla, server).await);
});
}
Some(rendezvous_message::Union::IceCandidate(ice)) => {
let queued = {
let mut txs = WEBRTC_ICE_TXS.lock().await;
txs.get_mut(&ice.session_key)
.map(|route| route.queue(ice.candidate))
};
match queued {
Some(false) => {
if let Some(n) = FULL_ICE_QUEUE_LOG.due() {
log::debug!("dropped {} ICE candidate(s): queue full or closed", n);
}
}
None => {
if let Some(n) = UNKNOWN_ICE_SESSION_LOG.due() {
log::debug!(
"dropped {} ICE candidate(s) for unknown WebRTC session key, last: {}",
n,
ice.session_key
);
}
}
_ => {}
}
}
Some(rendezvous_message::Union::ConfigureUpdate(cu)) => {
let v0 = Config::get_rendezvous_servers();
Config::set_option(
@@ -508,6 +598,7 @@ impl RendezvousMediator {
rr.secure,
false,
Default::default(),
String::new(),
meta,
)
.await
@@ -522,6 +613,7 @@ impl RendezvousMediator {
secure: bool,
initiate: bool,
socket_addr_v6: bytes::Bytes,
webrtc_sdp_answer: String,
meta: ConnectionMeta,
) -> ResultType<()> {
let peer_addr = AddrMangle::decode(&socket_addr);
@@ -540,6 +632,7 @@ impl RendezvousMediator {
socket_addr: socket_addr.into(),
version: crate::VERSION.to_owned(),
socket_addr_v6,
webrtc_sdp_answer,
..Default::default()
};
if initiate {
@@ -606,6 +699,7 @@ impl RendezvousMediator {
true,
true,
socket_addr_v6,
String::new(),
meta,
)
.await
@@ -642,6 +736,163 @@ impl RendezvousMediator {
Ok(())
}
/// Build the WebRTC answerer for a punch-hole offer and return the SDP answer that rides in
/// the punch reply (PunchHoleSent / RelayResponse).
///
/// Awaited inline on the punch-reply path, which only holds because everything here is local
/// (pc + keygen + SDP; trickle means the answer carries no candidates). Keep network I/O out
/// — connection setup belongs in the detached task below.
async fn spawn_webrtc_answerer(
&self,
ph: &PunchHole,
relay_only_ice: bool,
server: ServerPtr,
peer_addr: SocketAddr,
meta: ConnectionMeta,
) -> ResultType<String> {
let mut stream =
WebRTCStream::new(&ph.webrtc_sdp_offer, relay_only_ice, CONNECT_TIMEOUT).await?;
let answer = stream.local_endpoint().to_owned();
let session_key = stream.session_key().to_owned();
let return_route = ph.socket_addr.clone();
// A duplicate PunchHole (the offerer re-sends the same request across punch attempts)
// resolves to the SESSIONS-cached stream. `take_local_ice_rx` yields the receiver
// exactly once per stream instance, so `None` here means an answerer was already
// spawned for this offer: return the (identical) cached answer without spawning a
// second connect task. Otherwise two `create_tcp_connection` tasks would detach and
// read the same data channel, interleaving the handshake and corrupting the session.
let Some(mut local_ice_rx) = stream.take_local_ice_rx() else {
return Ok(answer);
};
// Bounded: how many candidates arrive is the sender's choice, while draining one costs a
// JSON parse and the ICE agent's lock, so an unbounded queue lets whoever can reach this
// session's route grow it without limit inside a long-lived service process. A full queue
// drops the newest candidate, and the controller re-sends it once — the digests beside the
// sender are what keep that re-send from spending a slot of its own.
let (remote_ice_tx, mut remote_ice_rx) = mpsc::channel::<String>(MAX_PENDING_REMOTE_ICE);
let own_ice_tx = remote_ice_tx.clone();
WEBRTC_ICE_TXS
.lock()
.await
.insert(session_key.clone(), IceRoute::new(remote_ice_tx));
let stream_for_remote_ice = stream.clone();
tokio::spawn(async move {
while let Some(candidate) = remote_ice_rx.recv().await {
if let Err(err) = stream_for_remote_ice.add_remote_ice_candidate(&candidate).await
{
if let Some(n) = REJECTED_REMOTE_ICE_LOG.due() {
log::warn!(
"failed to add {} remote WebRTC ICE candidate(s), last: {}",
n,
err
);
}
}
}
});
{
let host = self.host.clone();
let socket_addr = return_route.clone();
let session_key_for_ice = session_key.clone();
tokio::spawn(async move {
// Candidates ride a dedicated TCP connection to the rendezvous server, like
// the answer, NOT the mediator channel: that channel is UDP in the default
// setup, and target deployments front hbbs with websocket/TCP only, where
// its UDP port is unreachable. The server keeps candidate-carrying TCP
// connections open, so one lazily-opened connection serves the whole
// trickle, and TCP reliability replaces the old 400ms duplicate re-send
// (the controller keeps its own re-send for the server->peer UDP downlink).
let mut conn = None;
while let Some(candidate) = local_ice_rx.recv().await {
let mut msg = Message::new();
msg.set_ice_candidate(IceCandidate {
socket_addr: socket_addr.clone(),
session_key: session_key_for_ice.clone(),
candidate,
..Default::default()
});
// One reconnect attempt per candidate: the first send after an hbbs
// restart or an idle-killed connection fails on the stale stream.
for _ in 0..2 {
if conn.is_none() {
match connect_tcp(&*host, CONNECT_TIMEOUT).await {
Ok(s) => conn = Some(s),
Err(err) => {
log::warn!(
"failed to connect for WebRTC ICE candidate: {}",
err
);
break;
}
}
}
if let Some(s) = conn.as_mut() {
match s.send(&msg).await {
Ok(()) => break,
Err(err) => {
log::debug!(
"WebRTC ICE candidate send failed, reconnecting: {}",
err
);
conn = None;
}
}
}
}
}
});
}
let session_key_for_cleanup = session_key.clone();
tokio::spawn(async move {
let result = stream.wait_connected(CONNECT_TIMEOUT).await;
// Only evict our own route. The key is the offer's DTLS fingerprint, identical across
// the controller's punch retries, so a retry that built a fresh answerer has already
// replaced this entry — removing it blindly would delete the live session's sender and
// leave it receiving no candidates at all.
{
let mut txs = WEBRTC_ICE_TXS.lock().await;
if txs
.get(&session_key_for_cleanup)
.is_some_and(|route| route.is_same_channel(&own_ice_tx))
{
txs.remove(&session_key_for_cleanup);
}
}
if let Err(err) = result {
log::warn!("webrtc wait_connected failed: {}", err);
// Release the pc now rather than waiting for the ICE agent to time out into a
// terminal state (~30s); this also drops the SESSIONS entry promptly.
stream.close().await;
return;
}
// create_tcp_connection takes ownership of the stream; keep a handle to close the pc
// once the session returns. It runs the whole session and returns Ok on normal end,
// Err on setup failure — either way the pc must be closed, else it lingers forever in
// SESSIONS (its state handler only fires on a terminal ICE state, which a cleanly
// closed session may never reach) leaking the pc, channels, and socket fds.
let stream_for_cleanup = stream.clone();
if let Err(err) = crate::server::create_tcp_connection(
server,
Stream::WebRTC(stream),
peer_addr,
true,
meta,
)
.await
{
log::warn!("failed to create WebRTC server connection: {}", err);
}
stream_for_cleanup.close().await;
});
Ok(answer)
}
async fn handle_punch_hole(&self, ph: PunchHole, server: ServerPtr) -> ResultType<()> {
let mut peer_addr = AddrMangle::decode(&ph.socket_addr);
let last = *LAST_MSG.lock().await;
@@ -651,18 +902,52 @@ impl RendezvousMediator {
return Ok(());
}
let peer_addr_v6 = hbb_common::AddrMangle::decode(&ph.socket_addr_v6);
let relay = use_ws() || Config::is_proxy() || ph.force_relay;
let local_proxy = use_ws() || Config::is_proxy();
let relay = local_proxy || ph.force_relay;
let mut socket_addr_v6 = Default::default();
let meta = connection_meta(
ph.control_permissions.into_option(),
ph.controlled_context.into_option(),
ph.control_permissions.clone().into_option(),
ph.controlled_context.clone().into_option(),
);
// The controller's force_relay alone does not say whether ICE must be Relay-only; its
// offer envelope does. `ice_policy: "all"` means the relay was forced by the transport
// (ws), so answer with full ICE and let a direct pair form.
let webrtc_relay_only =
ph.force_relay && !WebRTCStream::endpoint_declares_all_ice(&ph.webrtc_sdp_offer);
// No enable-webrtc check here: it is LocalConfig, which the UI process writes and never
// syncs over IPC, so this (server) process would read the private-server default of "N"
// and refuse to answer in exactly the self-hosted deployments the transport is for.
// A proxy still rules it out — ICE would bypass it and leak the real IP.
let webrtc_viable = !ph.webrtc_sdp_offer.is_empty()
&& !Config::is_proxy()
&& (!webrtc_relay_only || WebRTCStream::has_turn_server());
let webrtc_sdp_answer = if webrtc_viable {
self.spawn_webrtc_answerer(
&ph,
webrtc_relay_only,
server.clone(),
peer_addr,
meta.clone(),
)
.await
.unwrap_or_else(|err| {
log::warn!("failed to create WebRTC answer: {}", err);
String::new()
})
} else {
String::new()
};
if peer_addr_v6.port() > 0 && !relay {
socket_addr_v6 =
start_ipv6(peer_addr_v6, peer_addr, server.clone(), meta.clone()).await;
}
let relay_server = self.get_relay_server(ph.relay_server);
// for ensure, websocket go relay directly
// A symmetric NAT relays the legacy transports but deliberately not WebRTC: the answer
// built above rides along on the relay request, and ICE probes the candidate pairs rather
// than trusting this classification, so a direct WebRTC pair can still form on a
// connection this branch has already called relay-only. Do not gate the answerer on
// nat_type to make the two agree.
if ph.nat_type.enum_value() == Ok(NatType::SYMMETRIC)
|| Config::get_nat_type() == NatType::SYMMETRIC as i32
|| relay
@@ -678,6 +963,7 @@ impl RendezvousMediator {
true,
true,
socket_addr_v6.clone(),
webrtc_sdp_answer.clone(),
meta,
)
.await;
@@ -691,6 +977,7 @@ impl RendezvousMediator {
nat_type: nat_type.into(),
version: crate::VERSION.to_owned(),
socket_addr_v6,
webrtc_sdp_answer,
..Default::default()
};
if ph.udp_port > 0 {
@@ -699,12 +986,25 @@ impl RendezvousMediator {
.await?;
return Ok(());
}
if !ph.webrtc_sdp_offer.is_empty() {
// Return the answer over its own short-lived TCP connection rather than the mediator
// channel: that channel is UDP by default, and hbbs applies UDP-punch semantics
// (source-address observation) to a PunchHoleSent that arrives on it. No TCP punch
// is made — the controller keeps its request socket for trickled ICE.
let mut msg_out = Message::new();
msg_out.set_punch_hole_sent(msg_punch);
let mut socket = connect_tcp(&*self.host, CONNECT_TIMEOUT).await?;
socket.send(&msg_out).await?;
return Ok(());
}
log::debug!("Punch tcp hole to {:?}", peer_addr);
let mut socket = {
let socket = connect_tcp(&*self.host, CONNECT_TIMEOUT).await?;
let local_addr = socket.local_addr();
// key important here for punch hole to tell my gateway incoming peer is safe.
// it can not be async here, because local_addr can not be reused, we must close the connection before use it again.
// Awaited rather than spawned so the mapping exists before `PunchHoleSent` goes out;
// `local_addr` itself is shared, not exclusive - every socket here binds it with the
// reuse flags `new_socket` sets.
allow_err!(socket_client::connect_tcp_local(peer_addr, Some(local_addr), 30).await);
socket
};
@@ -712,7 +1012,10 @@ impl RendezvousMediator {
msg_out.set_punch_hole_sent(msg_punch);
let bytes = msg_out.write_to_bytes()?;
socket.send_raw(bytes).await?;
crate::accept_connection(server.clone(), socket, peer_addr, true, meta).await;
let local_addr = socket.local_addr();
// The listener inside takes this address over, so the mediator's socket goes first.
drop(socket);
punch_tcp_until_connected(server, peer_addr, local_addr, meta).await;
Ok(())
}
@@ -951,11 +1254,11 @@ async fn udp_nat_listen(
let socket_cloned = socket.clone();
let func = async {
socket.connect(peer_addr).await?;
let res = crate::punch_udp(socket.clone(), true).await?;
let init_packet = crate::punch_udp(socket.clone(), true).await?;
let stream = crate::kcp_stream::KcpStream::accept(
socket,
Duration::from_millis(CONNECT_TIMEOUT as _),
res,
init_packet,
)
.await?;
crate::server::create_tcp_connection(server, stream.1, peer_addr_v4, true, meta).await?;
@@ -971,6 +1274,194 @@ async fn udp_nat_listen(
Ok(())
}
/// Where the repeats start, and the factor they slow by. The controller's SYN arrives once, at an
/// instant we are never told, inside a window we are not told either: `Client::connect` sizes its
/// dial only after our PunchHoleSent, from its own rendezvous time and the direct failures it has
/// recorded for us - `CONNECT_TIMEOUT` between two known-asymmetric NATs that never failed, as
/// little as a second once one has. So the repeats cover our own ceiling instead, `CONNECT_TIMEOUT`,
/// which is as long as the accept below has always been willing to take a connection, and back
/// off across it: dense at the start, where every window begins and the short ones end, sparse
/// afterwards, which is `punch_udp`'s shape for the same reason.
const PUNCH_INTERVAL: f32 = 0.15;
const PUNCH_BACKOFF: f32 = 1.5;
const PUNCH_MAX_INTERVAL: f32 = 2.0;
/// How long a punch in flight may run past the deadline, and the only timer it runs on. A punch
/// is cancel-safe while it is still in SYN_SENT and not once the controller's SYN has crossed it:
/// the socket is then half way through a handshake, and dropping it there cuts the connection the
/// controller is opening - which its `connect` has already returned, so that attempt fails
/// outright rather than falling back to relay. A timer cannot tell the two states apart, so no
/// punch is cut on a schedule of its own, and none needs to be. A gateway that answers with RST
/// fails the connect at once, and the loop punches again. One that drops the SYN in silence
/// leaves the socket in SYN_SENT, where it holds the mapping open and the kernel re-sends the
/// SYN, and any SYN of the controller's that arrives crosses it - a second punch has nothing to
/// add. That leaves the deadline, and this much past it lets a crossing begun just before it
/// complete; Windows gives a SYN up at about 21s anyway.
const PUNCH_GRACE: u64 = 3000;
/// The punch above leaves before hbbs has told the controller where to dial, so it is never in
/// flight at the same time as the controller's SYN: it opens our NAT, meets nothing, and a gateway
/// that answers it with RST takes the mapping down with it - leaving the listener below waiting on
/// a hole that no longer exists. Punching again across the window in which the controller dials
/// rebuilds it, and once the controller sits in SYN_SENT one of those punches meets its SYN and
/// completes as a simultaneous open: a second way in, which a single punch never had.
async fn punch_tcp_until_connected(
server: ServerPtr,
peer_addr: SocketAddr,
local_addr: SocketAddr,
meta: ConnectionMeta,
) {
use hbb_common::tcp::new_listener;
// Shadows the module's `std::time::Instant`: the deadline is held against tokio's sleeps and
// timeouts, so it runs on their clock.
use hbb_common::tokio::time::Instant;
// Not fatal on its own - the punch below can still meet the controller's SYN without it, and
// that half is the one a listener the OS refused to bind could not have covered anyway.
let listener = match new_listener(local_addr, true).await {
Ok(listener) => {
log::info!("Server listening on: {local_addr}");
Some(listener)
}
Err(err) => {
log::warn!("Failed to listen on {local_addr} after punching: {err}");
None
}
};
// Bounds both halves: the punch keeps the mapping open only while the accept is still
// willing to take a connection through it.
let until = Instant::now() + Duration::from_millis(CONNECT_TIMEOUT);
let punch = punch_until(until, peer_addr, |ms| {
socket_client::connect_tcp_local(peer_addr, Some(local_addr), ms)
});
let Some(listener) = listener else {
if let Some(stream) = punch.await {
serve_punched(server, stream, peer_addr, meta).await;
}
return;
};
// Accepting in a loop, not once: a transient `accept` error must not spend the whole window
// the controller still has to arrive in.
let accept = async {
loop {
let left = until.saturating_duration_since(Instant::now()).as_millis() as u64;
if left == 0 {
break;
}
match hbb_common::timeout(left, listener.accept()).await {
// Not filtered by address, as `accept_connection` never did: hbbs saw the
// controller through one mapping and a NAT that pools its external addresses may
// dial us from another, and what keeps `meta`'s control permissions from a second
// peer is the handshake, plus that exactly one connection is ever served.
Ok(Ok(accepted)) => return Some(accepted),
Ok(Err(err)) => {
log::warn!("Failed to accept from {peer_addr}: {err}");
// One that persists - EMFILE, say - would otherwise spin here for the window.
sleep(1.).await;
}
Err(_) => break,
}
}
log::info!("Nothing connected to the hole punched to {peer_addr}");
None
};
// Only the accept races the punch. Racing `accept_connection` instead would race the whole
// session it goes on to run, so a punch landing mid-session would tear that session down.
//
// Whichever arrives first is the one connection this request produces. Serving the loser too
// would give a second peer the control permissions hbbs granted for this one controller, and
// no test on the connection itself can tell the two apart before `create_tcp_connection` has
// spoken to it - so the invariant is kept here, by there being no second serve.
let punched = select! {
// Both ready at once is two connections, not one seen twice - a crossing carries the
// punch's four-tuple, which the listener never matches - and the punch is the one kept:
// it is known to have met something at the address hbbs gave, where the accept takes
// any address, and dropping it would reset the connection the controller is opening.
biased;
Some(stream) = punch => stream,
Some((stream, addr)) = accept => {
return accept_punched_connection(server, stream, addr, meta).await;
}
else => return,
};
serve_punched(server, punched, peer_addr, meta).await;
}
/// The repeats of `punch_tcp_until_connected`, over any punch rather than `connect_tcp_local`
/// alone, so that a test can run the schedule against a paused clock - which no socket can be.
async fn punch_until<T, F, Fut>(
until: tokio::time::Instant,
peer_addr: SocketAddr,
mut punch: F,
) -> Option<T>
where
F: FnMut(u64) -> Fut,
Fut: std::future::Future<Output = ResultType<T>>,
{
use hbb_common::tokio::time::Instant;
let mut interval = PUNCH_INTERVAL;
let mut round = 0;
loop {
// The deadline decides whether another punch starts, never how long one already in
// flight may take: that one runs to PUNCH_GRACE past it.
let left = until.saturating_duration_since(Instant::now());
if left.is_zero() {
log::debug!("None of {round} punches to {peer_addr} was met");
return None;
}
// Cut at the deadline rather than slept out past it, so the window ends on a punch and
// not on a gap of up to PUNCH_MAX_INTERVAL: the controller's window opened after ours,
// on the PunchHoleSent hbbs relayed, so one as long as ours is still open through our tail.
tokio::time::sleep(Duration::from_secs_f32(interval).min(left)).await;
interval = (interval * PUNCH_BACKOFF).min(PUNCH_MAX_INTERVAL);
let ms = until.saturating_duration_since(Instant::now()).as_millis() as u64 + PUNCH_GRACE;
match punch(ms).await {
// The controller's SYN crossed this punch, so the stream is the connection it
// dialed, not a spare one: dropping it would reset that connection.
Ok(stream) => return Some(stream),
// Not logged one by one, but the count says which gateway it was: RST fails a
// punch at once and fits a dozen into the window, a silent drop holds the one
// punch for the whole of it. `connect_tcp_local` keeps no errno anyway.
Err(_) => round += 1,
}
}
}
async fn serve_punched(
server: ServerPtr,
stream: Stream,
peer_addr: SocketAddr,
meta: ConnectionMeta,
) {
log::info!("Punched tcp hole to {peer_addr}, connected on the punch itself");
if let Err(err) =
crate::server::create_tcp_connection(server, stream, peer_addr, true, meta).await
{
log::warn!("Failed to serve the connection punched to {peer_addr}: {err}");
}
}
/// The accept half of `accept_connection`, kept here because only the accept may race the punch.
async fn accept_punched_connection(
server: ServerPtr,
stream: tokio::net::TcpStream,
addr: SocketAddr,
meta: ConnectionMeta,
) {
use crate::server::create_tcp_connection;
stream.set_nodelay(true).ok();
match stream.local_addr() {
Ok(stream_addr) => {
let stream = Stream::from(stream, stream_addr);
if let Err(err) = create_tcp_connection(server, stream, addr, true, meta).await {
log::warn!("Failed to serve the connection from {addr}: {err}");
}
}
Err(err) => log::warn!("Failed to read the address accepted from {addr}: {err}"),
}
}
// When config is not yet synced from root, register_pk may have already been sent with a new generated pk.
// After config sync completes, the pk may change. This struct detects pk changes and triggers
// a re-registration by setting key_confirmed to false.
@@ -995,3 +1486,255 @@ impl Drop for CheckIfResendPk {
}
}
}
#[cfg(test)]
mod tests {
use super::{mpsc, socket_client, tokio, IceRoute, ICE_DEDUP_WINDOW, MAX_PENDING_REMOTE_ICE};
use hbb_common::tcp::new_listener;
use std::net::SocketAddr;
// A SOCKS proxy makes `connect_tcp_local` dial the proxy and ignore the local address, so
// nothing these two assert can hold. Read once, from the same global config production reads.
fn proxied() -> bool {
hbb_common::config::Config::get_socks().is_some()
}
/// Both held while their addresses are read, so the pair cannot be the same port - which
/// `SO_REUSEPORT` would let bind twice rather than refuse, leaving the tests degenerate.
async fn free_loopback_pair() -> (SocketAddr, SocketAddr) {
let (a, b) = (
tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(),
tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(),
);
(a.local_addr().unwrap(), b.local_addr().unwrap())
}
fn queue(route: &mut IceRoute, candidate: &str) -> bool {
route.queue(candidate.to_owned())
}
#[test]
fn the_re_sent_copy_does_not_spend_a_queue_slot() {
// Two slots, three sends: without the dedup the re-send takes the second and "relay",
// the one that traverses NAT, is the one refused.
let (tx, mut rx) = mpsc::channel::<String>(2);
let mut route = IceRoute::new(tx);
for _ in 0..2 {
assert!(queue(&mut route, "host"));
}
assert!(queue(&mut route, "relay"));
let mut queued = Vec::new();
while let Ok(candidate) = rx.try_recv() {
queued.push(candidate);
}
assert_eq!(queued, vec!["host".to_owned(), "relay".to_owned()]);
}
#[test]
fn a_candidate_the_full_queue_refused_is_not_remembered() {
let (tx, mut rx) = mpsc::channel::<String>(1);
let mut route = IceRoute::new(tx);
assert!(queue(&mut route, "host"));
assert!(!queue(&mut route, "relay"));
// The re-send is the only repair for a refused candidate; remembering it would swallow it.
assert_eq!(rx.try_recv().ok(), Some("host".to_owned()));
assert!(queue(&mut route, "relay"));
assert_eq!(rx.try_recv().ok(), Some("relay".to_owned()));
}
#[test]
fn a_re_send_is_skipped_while_the_original_is_still_queued() {
let (tx, mut rx) = mpsc::channel::<String>(MAX_PENDING_REMOTE_ICE);
let mut route = IceRoute::new(tx);
for i in 0..MAX_PENDING_REMOTE_ICE {
assert!(queue(&mut route, &format!("candidate-{}", i)));
}
assert!(queue(&mut route, "candidate-0"));
let mut queued = 0;
while rx.try_recv().is_ok() {
queued += 1;
}
assert_eq!(queued, MAX_PENDING_REMOTE_ICE);
}
#[test]
fn the_window_forgets_in_arrival_order() {
let (tx, mut rx) = mpsc::channel::<String>(MAX_PENDING_REMOTE_ICE);
let mut route = IceRoute::new(tx);
for i in 0..=ICE_DEDUP_WINDOW {
assert!(queue(&mut route, &format!("candidate-{}", i)));
assert!(rx.try_recv().is_ok());
}
// The oldest digest made room for the newest, so its re-send is admitted again.
assert!(queue(&mut route, "candidate-0"));
assert!(rx.try_recv().is_ok());
// A recent one is still skipped.
let recent = format!("candidate-{}", ICE_DEDUP_WINDOW);
assert!(queue(&mut route, &recent));
assert!(rx.try_recv().is_err());
}
// The second way in that the repeat punch opens: a punch reaching a peer already in SYN_SENT
// is answered by that socket rather than reset, and the two ends come up on one connection.
// A punch that misses the crossing is reset outright here, loopback having no NAT to absorb
// it and no round trip to hide behind - so a single punch lands only by luck, and repeating
// is what makes it land at all. That is the premise of the repeat, asserted directly. A round
// that misses costs one loopback RST, so rounds are cheap and there are many.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn a_punch_that_meets_the_peers_syn_connects_both_ends() {
// The crossing needs both connects genuinely in flight at once. Loopback answers a SYN to
// a port nobody is listening on with an instant RST, so on one CPU the first connect runs
// to completion before the second is scheduled and no round can ever cross - a property of
// the box, which this test cannot tell apart from a broken punch.
if proxied() || std::thread::available_parallelism().map_or(true, |cpus| cpus.get() < 2) {
return;
}
for _ in 0..256 {
let (a, b) = free_loopback_pair().await;
// Held for the whole crossing, because production always has one here and the design
// rests on which of the two the kernel hands the connection to: the punch and the
// peer's SYN share a four-tuple exactly, the listener only matches the address, and
// the punch has to win that or every crossing would be swallowed as a plain accept.
let listener = new_listener(a, true).await.unwrap();
let to_b = tokio::spawn(socket_client::connect_tcp_local(b, Some(a), 3000));
let to_a = tokio::spawn(socket_client::connect_tcp_local(a, Some(b), 3000));
let (at_a, at_b) = tokio::join!(to_b, to_a);
let (Ok(Ok(mut at_a)), Ok(Ok(mut at_b))) = (at_a, at_b) else {
continue;
};
at_a.send_bytes(bytes::Bytes::from_static(b"punch"))
.await
.unwrap();
let got = at_b.next_timeout(3000).await.unwrap().unwrap();
assert_eq!(&got[..], b"punch", "both ends must share one connection");
assert!(
hbb_common::timeout(200, listener.accept()).await.is_err(),
"the crossing must reach the punch, not be accepted as an inbound connection"
);
return;
}
panic!("no punch met the peer's SYN in 256 rounds on a machine that can cross them");
}
// The punch binds the address the listener already holds, so it has to go through the same
// `connect_tcp_local` production uses - a punch built by hand here would still pass if
// `new_socket` ever stopped setting the reuse flags, while every real punch failed to bind.
// The peer's view of the source port is what proves the bind took: a fallback to an ephemeral
// one would connect just as happily.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn a_punch_binds_the_address_the_listener_holds() {
if proxied() {
return;
}
// `free_loopback_pair` hands back ports it no longer holds, so another process can take
// one in between; retry rather than fail for something the punch had no part in.
for _ in 0..8 {
let (local, peer_addr) = free_loopback_pair().await;
let (Ok(listener), Ok(peer)) = (
new_listener(local, true).await,
new_listener(peer_addr, true).await,
) else {
continue;
};
let punch = tokio::spawn(socket_client::connect_tcp_local(
peer_addr,
Some(local),
1500,
));
let (_peer_side, seen_as) = hbb_common::timeout(3000, peer.accept())
.await
.expect("the punch must reach the peer")
.unwrap();
assert_eq!(
seen_as.port(),
local.port(),
"the punch must leave from the address the listener holds, not an ephemeral one"
);
// Held, not asserted and dropped: the coexistence below is only exercised while this
// socket is still on the address, which is the state production spends its window in.
let _punched = punch.await.unwrap().expect("the punch must connect");
let dialed = tokio::spawn(tokio::net::TcpStream::connect(local));
let accepted = hbb_common::timeout(3000, listener.accept()).await;
assert!(
matches!(accepted, Ok(Ok(_))),
"the listener must still take connections while a punch shares its address: {accepted:?}"
);
assert!(dialed.await.unwrap().is_ok());
return;
}
panic!("could not hold two free loopback addresses in 8 tries");
}
// The schedule on its own, against a paused clock: the window is CONNECT_TIMEOUT long, and
// what these pin is where inside it the punches fall, which no socket could show.
#[tokio::test(start_paused = true)]
async fn the_punches_end_on_one_at_the_deadline() {
use super::{punch_until, PUNCH_GRACE, PUNCH_INTERVAL, PUNCH_MAX_INTERVAL};
use hbb_common::{anyhow::anyhow, config::CONNECT_TIMEOUT};
use std::time::Duration;
use tokio::time::Instant;
let peer: SocketAddr = "127.0.0.1:1".parse().unwrap();
let start = Instant::now();
let until = start + Duration::from_millis(CONNECT_TIMEOUT);
let mut punches = Vec::new();
// A gateway that answers with RST: every punch fails the moment it is made.
let met = punch_until::<(), _, _>(until, peer, |ms| {
punches.push((Instant::now(), ms));
async { Err(anyhow!("RST")) }
})
.await;
assert!(met.is_none());
assert_eq!(
Instant::now(),
until,
"must return the moment the window closes, not a backoff later"
);
// Tokio rounds every sleep up to the next millisecond.
let slack = Duration::from_millis(1);
assert!(punches[0].0 - start <= Duration::from_secs_f32(PUNCH_INTERVAL) + slack);
for pair in punches.windows(2) {
assert!(
pair[1].0 - pair[0].0 <= Duration::from_secs_f32(PUNCH_MAX_INTERVAL) + slack,
"no gap in the window may exceed the backoff ceiling: {pair:?}"
);
}
assert_eq!(
*punches.last().unwrap(),
(until, PUNCH_GRACE),
"the window must end on a punch, given the whole grace"
);
}
#[tokio::test(start_paused = true)]
async fn a_punch_in_flight_runs_the_grace_past_the_deadline_and_no_further() {
use super::{punch_until, PUNCH_GRACE};
use hbb_common::{anyhow::anyhow, config::CONNECT_TIMEOUT};
use std::time::Duration;
use tokio::time::Instant;
let peer: SocketAddr = "127.0.0.1:1".parse().unwrap();
let until = Instant::now() + Duration::from_millis(CONNECT_TIMEOUT);
let mut punches = 0;
// A gateway that drops the SYN in silence: the punch sits in SYN_SENT for all it is given.
let met = punch_until::<(), _, _>(until, peer, |ms| {
punches += 1;
async move {
tokio::time::sleep(Duration::from_millis(ms)).await;
Err(anyhow!("timed out"))
}
})
.await;
assert!(met.is_none());
assert_eq!(
punches, 1,
"a punch held in SYN_SENT is the only one the window needs"
);
assert_eq!(
Instant::now(),
until + Duration::from_millis(PUNCH_GRACE),
"must return when the grace runs out, not a backoff later"
);
}
}