RELAY / OPERATIONS RUNBOOK 07
Relay Queue Recovery Runbook
Pause delivery before replaying events. Confirm the checkpoint, then resume in a controlled batch.
| Signal | Proceed | Stop |
|---|---|---|
| Acknowledgements | 50 of 50 matched | Any missing ID |
| Duplicates | Handled idempotently | Side effect repeated |
| Consumer health | Checks passing | Timeout or error |
01 Stabilize
Pause the worker and record the last acknowledged event ID. Confirm that the consumer is healthy before attempting a replay.
02 Recover
Replay the next 50 events from the checkpoint. Check duplicate handling and compare delivered IDs with acknowledgements.
03 Verify
Resume only when the batch reconciles and error rate remains below the agreed threshold. If it fails, pause again and retain the evidence.
This fictional runbook handles a stalled delivery queue. It demonstrates a safe sequence; adapt commands and thresholds to the actual system before operational use.
Synthetic operational example · Commands require environment-specific review.