RELAY / OPERATIONS RUNBOOK 07

Relay Queue Recovery Runbook

Pause delivery before replaying events. Confirm the checkpoint, then resume in a controlled batch.

Signal Proceed Stop
Acknowledgements 50 of 50 matched Any missing ID
Duplicates Handled idempotently Side effect repeated
Consumer health Checks passing Timeout or error

01 Stabilize

Pause the worker and record the last acknowledged event ID. Confirm that the consumer is healthy before attempting a replay.

02 Recover

Replay the next 50 events from the checkpoint. Check duplicate handling and compare delivered IDs with acknowledgements.

03 Verify

Resume only when the batch reconciles and error rate remains below the agreed threshold. If it fails, pause again and retain the evidence.

This fictional runbook handles a stalled delivery queue. It demonstrates a safe sequence; adapt commands and thresholds to the actual system before operational use.

Synthetic operational example · Commands require environment-specific review.