Repository navigation
Port the link watchdog - #101
jamiecounsell wants to merge 2 commits into
Conversation
|
Thanks for this, and for the detailed write-up and the on-hardware testing against Python RNS. Porting the link watchdog fills a real gap, and running it after the job loop to avoid the Before this can go in, a few things need sorting out:
|
The link watchdog was still marked as TODO. Without it, a link request with no reply stayed pending forever, initiators never sent keepalives, and dead links were never found stale or closed. The watchdog is now one pass of the reference loop per link, run from Transport::jobs() for each pending and active link, after the job loop next to the resource watchdogs (it sends keepalives and link closes, and outbound() waits for the job loop). Where the reference sleeps, the pass compares times instead: - PENDING and HANDSHAKE links time out after the establishment timeout. - ACTIVE links: when nothing has come in for the keepalive interval, an initiator sends a keepalive (at most one per interval), and a link with nothing in for stale_time goes STALE. - STALE links send LINKCLOSE and close once the grace period has passed: rtt * keepalive_timeout_factor + STALE_GRACE, capped at WATCHDOG_MAX_SLEEP, as the reference sleeps. The watchdog reads three values that nothing was setting, so this sets them as the reference does: - the keepalive interval, derived from the RTT when a link comes up (adds KEEPALIVE_MIN/KEEPALIVE_MAX; KEEPALIVE keeps its value) - the time the last keepalive was sent - the time of the last proof received on the link: validate_link_proof() now records it on the link, as the reference does test_transport checks that an unanswered link request times out through Reticulum::loop(), and that an active link goes stale and then closes after the grace period. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KbF8zYqrDkMpUrin37AMCv
A C++ initiator opens a link to a Python destination and leaves it idle for 22 s, longer than the 10 s stale time on a local link. Python checks that it answered at least three keepalives and that the link is still active. Python then drops everything on the link except LINKCLOSE, and the C++ side has to go stale, close the link with TIMEOUT and send LINKCLOSE, which Python sees as INITIATOR_CLOSED. Without the link watchdog, the C++ side sends no keepalives and Python closes the idle link after 15 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KbF8zYqrDkMpUrin37AMCv
40e02c4 to
3f4a49f
Compare
|
Problem
The link watchdog was still marked as TODO. Without it, a link request with no reply stayed pending forever, initiators never sent keepalives, and dead links were never found stale or closed.
Solution
I ported the reference watchdog as one pass per link, run from
Transport::jobs()for each pending and active link:LINKCLOSEand close after the grace period (L760-L765, which calls__teardown_packet())The reference has sent
LINKCLOSEon a stale close since at least RNS 0.9.6, so this matches it. The commented-out Python inLink.cppcame from an older version. The extralast_outboundkeepalive trigger from the first version of this PR is gone.The watchdog reads three values that nothing was setting, so this sets them as the reference does:
keepaliveandstale_time, from the RTT when the link becomes active (__update_keepalive()). AddsKEEPALIVE_MIN/KEEPALIVE_MAX;KEEPALIVEkeeps its value (360).last_keepalive, inhad_outbound(true)(L652-L655).last_proof:PacketReceipt::validate_link_proof()now records it on the link, as the reference does (Packet.pyL447). This is a one-line behavior change outside the watchdog. Without it, a link whose only inbound traffic is proofs would go stale.The watchdog runs after the job loop because it sends packets (keepalives,
LINKCLOSE), andTransport::outbound()waits for the job loop to finish.jobs()now also collects pending links, so unanswered link requests time out.Compatibility
LINKCLOSEare sent when the reference sends them (links above)RNS_*preprocessor guard (off by default) (none)Link::update_keepalive(),Link::last_proof(double),Type::Link::KEEPALIVE_MIN/KEEPALIVE_MAX)Testing
New native tests in
test_transport:test_link_watchdog_pending_timeout: a link request to a destination that never answers staysPENDINGbefore the establishment timeout and closes withTIMEOUTafter it, throughReticulum::loop(). Fails onmaster(the link staysPENDING).test_link_watchdog_stale_then_close: an active link staysACTIVEwhile traffic arrives, goesSTALEafterstale_timewithout it, staysSTALEduring the grace period, then closes withTIMEOUT.A new interop scenario
test_interop/run_link_keepalive.sh(C++ initiator, Python RNS responder) was added in its own commit:ACTIVE.LINKCLOSE. The C++ side should go stale, close withTIMEOUT, and sendLINKCLOSE, which Python receives asINITIATOR_CLOSED.Run against Python RNS 1.3.8:
The last answered keepalive was at ~21s, so the link went stale 10s later (
stale_time) and closed 5s after that (the grace period). Onmasterthe same test fails: no keepalives go out, and Python closes the link after ~15s.Platforms tested (and how):
wiscore_rak4631.test_interop/):run_link_keepalive.shandrun_link_lifecycle.shboth pass against RNS 1.3.8 on Linux aarch64. The senders run with transport disabled, so they were built with-DRNS_PERSIST_PATHS=0 -DRNS_PERSIST_KNOWN_DESTINATIONS=0 -DRNS_PERSIST_HASHLIST=0.