|
|
@@ -0,0 +1,189 @@
|
|
|
+# M1 Verification
|
|
|
+
|
|
|
+How to prove the M1 milestone is done.
|
|
|
+
|
|
|
+## What "M1 done" means
|
|
|
+
|
|
|
+SPEC §23: "HTTP POST ingest end-to-end — send a signed webhook →
|
|
|
+FCM test push to a fake device; layers 1, 3, 4, 5 in."
|
|
|
+
|
|
|
+Concretely, the **M1 exit criteria** are:
|
|
|
+
|
|
|
+1. `docker compose up -d` brings up the full M0+M1 stack (the new
|
|
|
+ `seed` sidecar runs once, applies migrations + seed, exits).
|
|
|
+2. `seed` finishes `ok` and the `companies`/`individuals`/
|
|
|
+ `fcm_tokens` tables are populated.
|
|
|
+3. `fakefcmd` is healthy on `:8820` (its `/health` returns
|
|
|
+ `{"status":"ok",...}`).
|
|
|
+4. A signed POST to ingestd on `:8800` returns `202` (M0 still works).
|
|
|
+5. Within ~2s, a row appears in the `deliveries` table with
|
|
|
+ `status='sent'` and `channel='fcm'`.
|
|
|
+6. fakefcmd's stderr log shows the corresponding `fakefcmd send`
|
|
|
+ line with the right `alert_id`.
|
|
|
+7. M0 protections still fire: bad signature → 401, payload too large
|
|
|
+ → 413, invalid JSON → 400, rate-limit on burst → 429.
|
|
|
+
|
|
|
+## Step-by-step
|
|
|
+
|
|
|
+### 1. Bring up the stack
|
|
|
+
|
|
|
+```bash
|
|
|
+cd /root/.openclaw/workspace/broad-announce
|
|
|
+docker compose build # build all images
|
|
|
+docker compose up -d seed # run migrations + seed
|
|
|
+docker compose up -d # bring up the rest
|
|
|
+docker compose ps # all "Up (healthy)" except loadgen
|
|
|
+```
|
|
|
+
|
|
|
+Expected: the `seed` container exits 0 within ~5s after Postgres is
|
|
|
+healthy. The other services come up.
|
|
|
+
|
|
|
+### 2. Verify the seed
|
|
|
+
|
|
|
+```bash
|
|
|
+docker compose logs seed
|
|
|
+```
|
|
|
+
|
|
|
+Expected last line: `seed: ok`.
|
|
|
+
|
|
|
+```bash
|
|
|
+docker compose exec postgres psql -U ba -d ba -c \
|
|
|
+ "SELECT id, name FROM companies; SELECT id, full_name FROM individuals; SELECT count(*) FROM fcm_tokens;"
|
|
|
+```
|
|
|
+
|
|
|
+Expected:
|
|
|
+```
|
|
|
+ id | name
|
|
|
+---------+----------
|
|
|
+ acme-001 | Acme Corp
|
|
|
+ id | full_name
|
|
|
+------------+---------------
|
|
|
+ ind-acme-001 | Alice Operator
|
|
|
+ count
|
|
|
+-------
|
|
|
+ 1
|
|
|
+```
|
|
|
+
|
|
|
+### 3. fakefcmd health
|
|
|
+
|
|
|
+```bash
|
|
|
+curl -fsS http://localhost:8820/health
|
|
|
+```
|
|
|
+
|
|
|
+Expected: `{"failed":0,"received":0,"service":"fakefcmd","status":"ok"}`.
|
|
|
+
|
|
|
+### 4. Signed POST to ingestd
|
|
|
+
|
|
|
+```bash
|
|
|
+SECRET='s3cret-acme'
|
|
|
+BODY='{"company_id":"acme-001","source_id":"prom-prod","severity":"critical","category":"storage","title":"Disk full on db-prod-03","body":"92% used","data":{"host":"db-prod-03"},"dedupe_key":"m1-test-001"}'
|
|
|
+TS=$(date +%s)
|
|
|
+SIG=$(printf '%s.%s' "$TS" "$BODY" | openssl dgst -sha256 -hmac "$SECRET" -hex | awk '{print $2}')
|
|
|
+
|
|
|
+curl -fsS -X POST http://localhost:8800/v1/ingest \
|
|
|
+ -H "Content-Type: application/json" \
|
|
|
+ -H "X-BA-Signature: t=$TS,v1=$SIG" \
|
|
|
+ --data "$BODY"
|
|
|
+```
|
|
|
+
|
|
|
+Expected: `{"alert_id":"...","dedupe_count":1,"received_at":"..."}`.
|
|
|
+
|
|
|
+### 5. delivery row appears
|
|
|
+
|
|
|
+Within ~2s:
|
|
|
+
|
|
|
+```bash
|
|
|
+docker compose exec postgres psql -U ba -d ba -c \
|
|
|
+ "SELECT alert_id, individual_id, channel, target, status, attempts
|
|
|
+ FROM deliveries ORDER BY id DESC LIMIT 5;"
|
|
|
+```
|
|
|
+
|
|
|
+Expected: one row with `channel='fcm'`, `target='fake-fcm-token-acme-alice-001'`,
|
|
|
+`status='sent'`, `attempts=1`, `alert_id` matching step 4.
|
|
|
+
|
|
|
+### 6. fakefcmd logged the send
|
|
|
+
|
|
|
+```bash
|
|
|
+docker compose logs fakefcmd | tail -3
|
|
|
+```
|
|
|
+
|
|
|
+Expected last line:
|
|
|
+```
|
|
|
+fakefcmd send path=/v1/projects/fakefcmd/messages:send token=fake-fcm-token-acme-alice-001…
|
|
|
+ title=Disk full on db-prod-03 data_keys=8 alert_id=018f…
|
|
|
+```
|
|
|
+
|
|
|
+### 7. M0 protections still fire (regression check)
|
|
|
+
|
|
|
+Bad signature:
|
|
|
+```bash
|
|
|
+curl -i -X POST http://localhost:8800/v1/ingest \
|
|
|
+ -H "Content-Type: application/json" \
|
|
|
+ -H "X-BA-Signature: t=1,v1=deadbeef" \
|
|
|
+ --data "$BODY"
|
|
|
+```
|
|
|
+Expected: `401 {"error":"bad_signature"}`.
|
|
|
+
|
|
|
+Payload too large:
|
|
|
+```bash
|
|
|
+python3 -c 'import sys; sys.stdout.write("{\"x\":\"" + "a"*300000 + "\"}")' \
|
|
|
+ | curl -i -X POST http://localhost:8800/v1/ingest \
|
|
|
+ -H "Content-Type: application/json" --data-binary @-
|
|
|
+```
|
|
|
+Expected: `413 {"error":"payload_too_large"}`.
|
|
|
+
|
|
|
+### 8. Burst through loadgen
|
|
|
+
|
|
|
+```bash
|
|
|
+docker compose --profile loadgen up loadgen-http
|
|
|
+```
|
|
|
+
|
|
|
+Expected: 30s of 50/s; 1500 messages accepted; ~1500 deliveries
|
|
|
+written (1:1 with the single seeded token in M1); no failures.
|
|
|
+
|
|
|
+```bash
|
|
|
+docker compose exec postgres psql -U ba -d ba -c \
|
|
|
+ "SELECT status, count(*) FROM deliveries GROUP BY status;"
|
|
|
+```
|
|
|
+
|
|
|
+Expected: ~1500 `sent`, 0 `failed`, 0 `dlq`.
|
|
|
+
|
|
|
+### 9. Dedupe still works end-to-end
|
|
|
+
|
|
|
+Send the same `dedupe_key` twice in quick succession. Both
|
|
|
+deliveries succeed (one per the dedupe window), but
|
|
|
+`dedupe_count` in the 202 response is 2 on the second call.
|
|
|
+
|
|
|
+## What's NOT in M1 (the honest list)
|
|
|
+
|
|
|
+- **Recipient resolution rules** (subscriptions, opt-in per
|
|
|
+ source/severity, quiet hours, routing rules). M1 is broadcast.
|
|
|
+ **M2**.
|
|
|
+- **Real FCM** (Google service account, real HTTP v1 endpoint).
|
|
|
+ M1 talks to fakefcmd. **M3**.
|
|
|
+- **Other channels** (Telegram, SMS, email, Slack, Teams,
|
|
|
+ webhook). **M3+**.
|
|
|
+- **Retry with exponential backoff** (SPEC §9). M1 marks
|
|
|
+ terminal status on the first attempt. **M3** with the per-channel
|
|
|
+ retry worker.
|
|
|
+- **DLQ table reads, replay UI**. **M8**.
|
|
|
+- **Per-source `allowed_targets`** in the schema. **M2**.
|
|
|
+- **ClickHouse archive, Timescale hypertables** for `alerts` and
|
|
|
+ `deliveries`. **M7**.
|
|
|
+- **HMAC secret in DB** (M1 reads from env). **M2**.
|
|
|
+- **Multi-tenant query filters** at every query site. M1 has
|
|
|
+ `WHERE company_id = $1` in routing. M2 audits every other query.
|
|
|
+- **Per-IP concurrency cap, circuit breaker, quarantine** (M0
|
|
|
+ honest scope). **M5 / M9**.
|
|
|
+
|
|
|
+## If something fails
|
|
|
+
|
|
|
+| symptom | first thing to check |
|
|
|
+|---|---|
|
|
|
+| seed: `connect: dial tcp ...:5432: connect: connection refused` | is postgres healthy? `docker compose ps postgres` |
|
|
|
+| seed: `extension pgcrypto does not exist` | wrong image; we use `timescale/timescaledb:latest-pg16` which has it |
|
|
|
+| ingestd returns 401 on a "valid" call | openssl pipe vs Go HMAC mismatch; recheck `printf '%s.%s'` |
|
|
|
+| no rows in `deliveries` | is routerd healthy? `docker compose logs routerd` — it might not have created the consumer yet |
|
|
|
+| `deliveries` row but `status=failed` | is fakefcmd up? `curl localhost:8820/health` |
|
|
|
+| `last_error` says `status 503: UNAVAILABLE` | fakefcmd fault injection; check its log |
|
|
|
+| consumer never picks up messages | NATS JetStream state retained from a previous run with different consumer config; `docker compose down -v` to wipe |
|
Luis Rosales