broad-announce/M1_VERIFICATION.md

M1 Verification

How to prove the M1 milestone is done.

What "M1 done" means

SPEC §23: "HTTP POST ingest end-to-end — send a signed webhook → FCM test push to a fake device; layers 1, 3, 4, 5 in."

Concretely, the M1 exit criteria are:

1. docker compose up -d brings up the full M0+M1 stack (the new seed sidecar runs once, applies migrations + seed, exits).
2. seed finishes ok and the companies/individuals/ fcm_tokens tables are populated.
3. fakefcmd is healthy on :8820 (its /health returns {"status":"ok",...}).
4. A signed POST to ingestd on :8800 returns 202 (M0 still works).
5. Within ~2s, a row appears in the deliveries table with status='sent' and channel='fcm'.
6. fakefcmd's stderr log shows the corresponding fakefcmd send line with the right alert_id.
7. M0 protections still fire: bad signature → 401, payload too large → 413, invalid JSON → 400, rate-limit on burst → 429.

Step-by-step

1. Bring up the stack

cd /root/.openclaw/workspace/broad-announce
docker compose build         # build all images
docker compose up -d seed    # run migrations + seed
docker compose up -d         # bring up the rest
docker compose ps            # all "Up (healthy)" except loadgen

Expected: the seed container exits 0 within ~5s after Postgres is healthy. The other services come up.

2. Verify the seed

docker compose logs seed

Expected last line: seed: ok.

docker compose exec postgres psql -U ba -d ba -c \
  "SELECT id, name FROM companies; SELECT id, full_name FROM individuals; SELECT count(*) FROM fcm_tokens;"

Expected:

   id    |   name
---------+----------
 acme-001 | Acme Corp
   id       |   full_name
------------+---------------
 ind-acme-001 | Alice Operator
 count
-------
     1

3. fakefcmd health

curl -fsS http://localhost:8820/health

Expected: {"failed":0,"received":0,"service":"fakefcmd","status":"ok"}.

4. Signed POST to ingestd

SECRET='s3cret-acme'
BODY='{"company_id":"acme-001","source_id":"prom-prod","severity":"critical","category":"storage","title":"Disk full on db-prod-03","body":"92% used","data":{"host":"db-prod-03"},"dedupe_key":"m1-test-001"}'
TS=$(date +%s)
SIG=$(printf '%s.%s' "$TS" "$BODY" | openssl dgst -sha256 -hmac "$SECRET" -hex | awk '{print $2}')

curl -fsS -X POST http://localhost:8800/v1/ingest \
  -H "Content-Type: application/json" \
  -H "X-BA-Signature: t=$TS,v1=$SIG" \
  --data "$BODY"

Expected: {"alert_id":"...","dedupe_count":1,"received_at":"..."}.

5. delivery row appears

Within ~2s:

docker compose exec postgres psql -U ba -d ba -c \
  "SELECT alert_id, individual_id, channel, target, status, attempts
   FROM deliveries ORDER BY id DESC LIMIT 5;"

Expected: one row with channel='fcm', target='fake-fcm-token-acme-alice-001', status='sent', attempts=1, alert_id matching step 4.

6. fakefcmd logged the send

docker compose logs fakefcmd | tail -3

Expected last line:

fakefcmd send path=/v1/projects/fakefcmd/messages:send token=fake-fcm-token-acme-alice-001…
  title=Disk full on db-prod-03 data_keys=8 alert_id=018f…

7. M0 protections still fire (regression check)

Bad signature:

curl -i -X POST http://localhost:8800/v1/ingest \
  -H "Content-Type: application/json" \
  -H "X-BA-Signature: t=1,v1=deadbeef" \
  --data "$BODY"

Expected: 401 {"error":"bad_signature"}.

Payload too large:

python3 -c 'import sys; sys.stdout.write("{\"x\":\"" + "a"*300000 + "\"}")' \
  | curl -i -X POST http://localhost:8800/v1/ingest \
      -H "Content-Type: application/json" --data-binary @-

Expected: 413 {"error":"payload_too_large"}.

8. Burst through loadgen

docker compose --profile loadgen up loadgen-http

Expected: 30s of 50/s; 1500 messages accepted; ~1500 deliveries written (1:1 with the single seeded token in M1); no failures.

docker compose exec postgres psql -U ba -d ba -c \
  "SELECT status, count(*) FROM deliveries GROUP BY status;"

Expected: ~1500 sent, 0 failed, 0 dlq.

9. Dedupe still works end-to-end

Send the same dedupe_key twice in quick succession. Both deliveries succeed (one per the dedupe window), but dedupe_count in the 202 response is 2 on the second call.

What's NOT in M1 (the honest list)

• Recipient resolution rules (subscriptions, opt-in per source/severity, quiet hours, routing rules). M1 is broadcast. M2.
• Real FCM (Google service account, real HTTP v1 endpoint). M1 talks to fakefcmd. M3.
• Other channels (Telegram, SMS, email, Slack, Teams, webhook). M3+.
• Retry with exponential backoff (SPEC §9). M1 marks terminal status on the first attempt. M3 with the per-channel retry worker.
• DLQ table reads, replay UI. M8.
• Per-source allowed_targets in the schema. M2.
• ClickHouse archive, Timescale hypertables for alerts and deliveries. M7.
• HMAC secret in DB (M1 reads from env). M2.
• Multi-tenant query filters at every query site. M1 has WHERE company_id = $1 in routing. M2 audits every other query.
• Per-IP concurrency cap, circuit breaker, quarantine (M0 honest scope). M5 / M9.

If something fails

symptom	first thing to check
seed: connect: dial tcp ...:5432: connect: connection refused	is postgres healthy? docker compose ps postgres
seed: extension pgcrypto does not exist	wrong image; we use timescale/timescaledb:latest-pg16 which has it
ingestd returns 401 on a "valid" call	openssl pipe vs Go HMAC mismatch; recheck printf '%s.%s'
no rows in deliveries	is routerd healthy? docker compose logs routerd — it might not have created the consumer yet
deliveries row but status=failed	is fakefcmd up? curl localhost:8820/health
last_error says status 503: UNAVAILABLE	fakefcmd fault injection; check its log
consumer never picks up messages	NATS JetStream state retained from a previous run with different consumer config; docker compose down -v to wipe