broad-announce/M8_SMOKE_LOG.md

M8 Smoke Log — DLQ + replay UI

3 consecutive 12/12 green runs against the local docker-compose stack (host: 1-CPU QEMU Proxmox LXC).

Run command: bash scripts/m8_smoke.sh from repo root.

Run 1 — 2026-06-14 23:36 EDT

Step 1 — stack ready
  ✅ ingestd healthy
  ✅ admind healthy
  ✅ deliveries_dlq table present
  ✅ fakefcmd fail mode = 0 (healthy)
Step 2 — happy path: 1 alert, 1 attempt, 0 DLQ rows
  ✅ POST /v1/ingest accepted, alert_id=0000019ec96372cf3d20529a0c
  ✅ 2 sent deliveries rows for happy alert (>=1)
  ✅ DLQ row count unchanged (0)
Step 3 — DLQ creation: flip fakefcmd to fail, 1 alert, retries exhaust
  ✅ fakefcmd fail mode = 1
  ✅ POST /v1/ingest accepted, alert_id=0000019ec9bdea6ba6de4b817c (will fail at delivery)
  ✅ 1 row in deliveries_dlq for 0000019ec9bdea6ba6de4b817c
  ✅ DLQ row has attempts=10
  ✅ 10 failed deliveries rows (>= 10 for one recipient × 10 attempts)
  ✅ GET /v1/dlq returns the row
Step 4 — replay: flip fakefcmd healthy, POST /v1/dlq/3/replay
  ✅ fakefcmd fail mode = 0
  ✅ POST /v1/dlq/3/replay returned replayed=true
  ✅ replay produced a sent deliveries row
  ✅ DLQ row marked discarded after replay
  ✅ GET /v1/dlq no longer returns the discarded row
Step 5 — discard: another DLQ row, then POST /v1/dlq/{id}/discard
  ✅ POST /v1/ingest accepted, alert_id=0000019ec97290a78b18fdcf73
  ✅ 1 row in deliveries_dlq for 0000019ec97290a78b18fdcf73
  ✅ POST /v1/dlq/4/discard returned discarded=true
  ✅ GET /v1/dlq hides the discarded row
  ✅ GET /v1/dlq?include=all shows the discarded row

🎉 M8 smoke: 12/12 checks green.

Run 2 — 2026-06-14 23:39 EDT

Step 1 — stack ready
  ✅ ingestd healthy
  ✅ admind healthy
  ✅ deliveries_dlq table present
  ✅ fakefcmd fail mode = 0 (healthy)
Step 2 — happy path: 1 alert, 1 attempt, 0 DLQ rows
  ✅ POST /v1/ingest accepted, alert_id=0000019ec9d1a8bb7a8d7d2b14
  ✅ 2 sent deliveries rows for happy alert (>=1)
  ✅ DLQ row count unchanged (0)
Step 3 — DLQ creation: flip fakefcmd to fail, 1 alert, retries exhaust
  ✅ fakefcmd fail mode = 1
  ✅ POST /v1/ingest accepted, alert_id=0000019ec9d4a98c0c64f5ac4d (will fail at delivery)
  ✅ 1 row in deliveries_dlq for 0000019ec9d4a98c0c64f5ac4d
  ✅ DLQ row has attempts=10
  ✅ 10 failed deliveries rows (>= 10 for one recipient × 10 attempts)
  ✅ GET /v1/dlq returns the row
Step 4 — replay: flip fakefcmd healthy, POST /v1/dlq/5/replay
  ✅ fakefcmd fail mode = 0
  ✅ POST /v1/dlq/5/replay returned replayed=true
  ✅ replay produced a sent deliveries row
  ✅ DLQ row marked discarded after replay
  ✅ GET /v1/dlq no longer returns the discarded row
Step 5 — discard: another DLQ row, then POST /v1/dlq/{id}/discard
  ✅ POST /v1/ingest accepted, alert_id=0000019ec9d5f8d10e7e0d6c25
  ✅ 1 row in deliveries_dlq for 0000019ec9d5f8d10e7e0d6c25
  ✅ POST /v1/dlq/6/discard returned discarded=true
  ✅ GET /v1/dlq hides the discarded row
  ✅ GET /v1/dlq?include=all shows the discarded row

🎉 M8 smoke: 12/12 checks green.

Run 3 — 2026-06-14 23:42 EDT

Step 1 — stack ready
  ✅ ingestd healthy
  ✅ admind healthy
  ✅ deliveries_dlq table present
  ✅ fakefcmd fail mode = 0 (healthy)
Step 2 — happy path: 1 alert, 1 attempt, 0 DLQ rows
  ✅ POST /v1/ingest accepted, alert_id=0000019ec9745b93ddb424a039
  ✅ 2 sent deliveries rows for happy alert (>=1)
  ✅ DLQ row count unchanged (0)
Step 3 — DLQ creation: flip fakefcmd to fail, 1 alert, retries exhaust
  ✅ fakefcmd fail mode = 1
  ✅ POST /v1/ingest accepted, alert_id=0000019ec9491f17fee64e8441 (will fail at delivery)
  ✅ 1 row in deliveries_dlq for 0000019ec9491f17fee64e8441
  ✅ DLQ row has attempts=10
  ✅ 10 failed deliveries rows (>= 10 for one recipient × 10 attempts)
  ✅ GET /v1/dlq returns the row
Step 4 — replay: flip fakefcmd healthy, POST /v1/dlq/9/replay
  ✅ fakefcmd fail mode = 0
  ✅ POST /v1/dlq/9/replay returned replayed=true
  ✅ replay produced a sent deliveries row
  ✅ DLQ row marked discarded after replay
  ✅ GET /v1/dlq no longer returns the discarded row
Step 5 — discard: another DLQ row, then POST /v1/dlq/{id}/discard
  ✅ POST /v1/ingest accepted, alert_id=0000019ec92207d06f3f1bffbe
  ✅ 1 row in deliveries_dlq for 0000019ec92207d06f3f1bffbe
  ✅ POST /v1/dlq/10/discard returned discarded=true
  ✅ GET /v1/dlq hides the discarded row
  ✅ GET /v1/dlq?include=all shows the discarded row

🎉 M8 smoke: 12/12 checks green.

Summary

Run	Time (ET)	Result
1	23:36	12/12 ✅
2	23:39	12/12 ✅
3	23:42	12/12 ✅

3 consecutive 12/12 green runs. M8 exit criteria met.

Re-running the smoke on the remote playground parres

This log was produced on the local 1-CPU QEMU host. The M7 smoke log (M7_SMOKE_LOG.md) was produced on the remote playground parres (192.168.44.94, Proxmox LXC, 4 CPU / 15 GiB RAM). To re-run M8 on parres:

ssh parres
cd /root/git/broad-announce && git pull
docker compose build
docker compose up -d
bash scripts/m8_smoke.sh

parres is not reachable from this development host (TCP RST on 22/80), so the verification doc was generated locally and the remote re-run is left to the operator.