Engineering Notes

Short explanations of correctness problems that surfaced while building the larger systems.

Why ReadIndex Exists

A node can still believe it is leader after losing the quorum. What evidence does it need before returning a linearizable read?

Why Replicas Are Not Backups

A correct replica copies a destructive transaction too. Historical recovery needs a different source of truth.

What a Fencing Token Prevents

A lease can expire while its worker keeps running. The receiver still needs a way to reject the stale owner.

Why an Outbox Is Still At-Least-Once

Atomic local intent prevents event loss, but a crash after publication can still produce redelivery.