Skip to content

Issue: hosts behind at finalize must pull the missing diffs #2079

Open @a-kuprin opened 2026-10-21 21:21 UTC 0 comments Updated 2026-10-21 21:21 UTC
enhancement

Issue: hosts behind at finalize must pull the missing diffs

Milestone: Devshard v7

Follows: #1785 (fix/devshard-finalize-resume-v6)

Summary

#1785 can merge. It makes Finalize resume from PhaseFinalizing and skip a host whose reply is rejected. It does not bring a lagging host up to the tip.

Commit af125be removed the only catch-up push that PR added. On resume, refreshPendingTxsFromHosts sends one request per host at the current nonce with an empty Diffs slice (TestFinalize_ResumeRefreshDoesNotPushCatchUpToLaggingHost). A host that is behind is left behind. No host in this tree pulls diffs or snapshots to close that gap.

v7 should add that pull. A host that is behind during finalization fetches the diffs it lacks and applies them before the remaining settlement diffs close.

What is left after #1785

Settlement is reached when LatestNonce >= FinalizeNonce + groupSize. That is groupSize + 1 nonces after the last active nonce, the same budget as FinalizeNonceReserve. The phase changes on that nonce whether or not every host has applied the journal.

Two older pushes are still in Finalize:

  • finalizeDiffRound calls sendDiffRound, which still runs catchUpForSend for the host selected for that round. The diff is composed first. The mempool in the reply is queued for a later diff.
  • After Settlement, CollectSignatures still falls back from GET /signatures to sendCatchUpWith.

A host the round-robin still contacts can receive the journal that way. A host that is only probed by the resume refresh receives nothing. A finish that the host produces only after it applies the gap misses the drain when no later diff is composed in time. settleLiveRecordLocked then closes a still-live inference at ReservedCost.

The PR discussion treats this as the host catching up on its own. That pull is the v7 work. It is not in the tree, and #1785 does not wait for it.

Work

A host that finds itself behind the finalize tip pulls the missing diffs (and a snapshot, when that is the cheaper prefix) and applies them on its own.

The pull has to land in time for the host's mempool — finishes and the other host-proposed txs — to be back on the gateway before the diff that crosses FinalizeNonce + groupSize is composed. Catch-up attached to a round after that diff is composed cannot carry those txs into settlement.

Until the pull exists, a resume with a lagging executor can settle that executor's live inferences at reserved cost.

🔄 Auto-synced from Issue #2079 every hour.