SIGN IN SIGN UP
FuelLabs / fuel-core UNCLAIMED

Rust full node implementation of the Fuel v2 protocol.

0 0 15 Rust

fix: group reconciliation votes by block_id to resolve same-block deadlock (#3269)

## Summary

- Fixes a PoA reconciliation deadlock observed on devnet 2026-04-17
where the same block ended up on all 6 Redis nodes with three different
epochs, causing permanent livelock
- `unreconciled_blocks` now groups votes by `block_id` only, tracking
max epoch as a tiebreaker. Identical blocks written during re-promotion
storms count toward quorum.
- Added a regression test that reproduces the exact production error
string

## The bug

During re-promotion storms (two pods racing for leadership), the same
block can be written to different Redis nodes with different epochs. The
old vote grouping `(epoch, block_id)` fragmented these identical blocks
into separate vote groups:

```
Node state (same block_id, different epoch stamps):
  1a-0, 1a-1, 1b-1: epoch 268 → vote group A, count=3
  1b-0:            epoch 269 → vote group B, count=1
  1c-0, 1c-1:      epoch 270 → vote group C, count=2  ← max-epoch winner

Required quorum: 4.  Winner count: 2 → repair attempted.
Repair writes the winner to all 6 nodes → HEIGHT_EXISTS on every node
(each has SOME entry at that height) → Written=0 → total=2 < quorum.
Permanent livelock.
```

## The fix

Group by `block_id` alone; track max epoch per block_id as the
tiebreaker when block_ids genuinely differ:

```rust
// Before
HashMap::<(u64, BlockId), (usize, SealedBlock)>
vote_key = (*epoch, block.entity.id())
winner = max_by_key(epoch)

// After
HashMap::<BlockId, (u64, usize, SealedBlock)>  // (max_epoch, count, block)
vote_key = block.entity.id()
winner = max_by_key(max_epoch)
```

**Behavior change:**
- Same block with multiple epochs → single vote group → counts as a
single block on N nodes → reconciles directly without repair (this fixes
the deadlock)
- Genuinely different blocks at same height → picks higher-epoch block →
same behavior as before

## Test plan

- [x] New test
`leader_state__when_same_block_has_different_epochs_across_nodes_then_reconciles_without_repair`
reproduces the exact production error without the fix (`"Backlog
unresolved at height 1: repair failed to reach quorum"`) and passes with
it
- [x] All 9 existing `leader_state__*` tests still pass
- [ ] Deploy to devnet and verify the stuck authority recovers
B
Brandon Kite committed
f7826d1c1b58cdfeabbf44bc7b671a7b14ea3039
Parent: c30e093
Committed by GitHub <noreply@github.com> on 4/17/2026, 5:46:06 PM