|
|
@@ -0,0 +1,169 @@
|
|
|
+# M0 Verification
|
|
|
+
|
|
|
+How to prove the M0 milestone is done.
|
|
|
+
|
|
|
+## What "M0 done" means
|
|
|
+
|
|
|
+SPEC §23: "Skeleton + docker-compose up — Postgres + NATS + Redis up,
|
|
|
+all 4 services start, /health green."
|
|
|
+
|
|
|
+Concretely, the **M0 exit criteria** are:
|
|
|
+
|
|
|
+1. `docker compose up -d` brings up the data tier (postgres, redis,
|
|
|
+ nats, emqx, clickhouse) cleanly.
|
|
|
+2. `ingestd`, `routerd`, `deliverd`, `admind` all start, log
|
|
|
+ "http listening", expose `/health` returning 200.
|
|
|
+3. `curl -X POST /v1/ingest` with a properly HMAC-signed alert
|
|
|
+ returns `202` with `{alert_id, dedupe_count, received_at}`.
|
|
|
+4. The same `dedupe_key` sent twice within 60s returns the same
|
|
|
+ `alert_id` shape but with `dedupe_count=2` on the second call.
|
|
|
+5. `loadgen-http --mode normal --rate 50 --duration 30s` runs
|
|
|
+ cleanly against `ingestd` with no errors.
|
|
|
+6. Prometheus on `:9090` shows `ba_ingestd_alerts_received_total`
|
|
|
+ with `result="accepted"` incrementing.
|
|
|
+
|
|
|
+## Step-by-step
|
|
|
+
|
|
|
+### 1. Bring up the stack
|
|
|
+
|
|
|
+```bash
|
|
|
+cd /root/.openclaw/workspace/broad-announce
|
|
|
+docker compose up -d
|
|
|
+docker compose ps # all services "Up (healthy)"
|
|
|
+```
|
|
|
+
|
|
|
+Expected: postgres, redis, nats, emqx, clickhouse come up healthy
|
|
|
+in ~30s. The four app services depend on them and start last.
|
|
|
+
|
|
|
+### 2. Check /health
|
|
|
+
|
|
|
+```bash
|
|
|
+curl -fsS http://localhost:8080/health
|
|
|
+curl -fsS http://localhost:8081/health
|
|
|
+curl -fsS http://localhost:8082/health
|
|
|
+curl -fsS http://localhost:8083/health
|
|
|
+```
|
|
|
+
|
|
|
+Expected: all four return `{"status":"ok","service":"..."}`.
|
|
|
+
|
|
|
+### 3. Signed webhook end-to-end
|
|
|
+
|
|
|
+From a host that can reach ingestd (`localhost:8080` if running
|
|
|
+docker compose on the same host):
|
|
|
+
|
|
|
+```bash
|
|
|
+SECRET="s3cret-acme"
|
|
|
+BODY='{"company_id":"acme-001","source_id":"prom-prod","severity":"critical","category":"storage","title":"Disk full on db-prod-03","body":"92% used","data":{"host":"db-prod-03"},"dedupe_key":"disk:db-prod-03:full"}'
|
|
|
+TS=$(date +%s)
|
|
|
+SIG=$(printf '%s.%s' "$TS" "$BODY" | openssl dgst -sha256 -hmac "$SECRET" -hex | awk '{print $2}')
|
|
|
+
|
|
|
+curl -fsS -X POST http://localhost:8080/v1/ingest \
|
|
|
+ -H "Content-Type: application/json" \
|
|
|
+ -H "X-BA-Signature: t=$TS,v1=$SIG" \
|
|
|
+ --data "$BODY"
|
|
|
+```
|
|
|
+
|
|
|
+Expected:
|
|
|
+```json
|
|
|
+{"alert_id":"018f...","dedupe_count":1,"received_at":"2026-06-13T..."}
|
|
|
+```
|
|
|
+
|
|
|
+Run the same call again — `dedupe_count` should be `2`.
|
|
|
+
|
|
|
+### 4. Bad signature
|
|
|
+
|
|
|
+Same call but with `v1=deadbeef`:
|
|
|
+
|
|
|
+```bash
|
|
|
+curl -i -X POST http://localhost:8080/v1/ingest \
|
|
|
+ -H "X-BA-Signature: t=$TS,v1=deadbeef" \
|
|
|
+ -H "Content-Type: application/json" \
|
|
|
+ --data "$BODY"
|
|
|
+```
|
|
|
+
|
|
|
+Expected: `401 {"error":"bad_signature"}`.
|
|
|
+
|
|
|
+### 5. Payload too large
|
|
|
+
|
|
|
+```bash
|
|
|
+python3 -c 'import sys; sys.stdout.write("{\"x\":\"" + "a"*300000 + "\"}")' \
|
|
|
+ | curl -i -X POST http://localhost:8080/v1/ingest \
|
|
|
+ -H "Content-Type: application/json" \
|
|
|
+ --data-binary @-
|
|
|
+```
|
|
|
+
|
|
|
+Expected: `413 {"error":"payload_too_large"}`.
|
|
|
+
|
|
|
+### 6. Schema invalid
|
|
|
+
|
|
|
+```bash
|
|
|
+curl -i -X POST http://localhost:8080/v1/ingest \
|
|
|
+ -H "Content-Type: application/json" \
|
|
|
+ --data '{"company_id":"acme-001","source_id":"prom-prod","severity":"BOGUS","title":"x"}'
|
|
|
+```
|
|
|
+
|
|
|
+Expected: `400 {"error":"invalid","detail":"invalid alert: [severity: ...]"}`.
|
|
|
+
|
|
|
+### 7. loadgen-http
|
|
|
+
|
|
|
+```bash
|
|
|
+docker compose --profile loadgen up loadgen-http
|
|
|
+```
|
|
|
+
|
|
|
+Expected: 30 seconds of progress lines, then a "done" summary with
|
|
|
+`failed=0`. The `ba_ingestd_alerts_received_total{result="accepted"}`
|
|
|
+counter in Prometheus should be ~1500 (= 30s × 50/s).
|
|
|
+
|
|
|
+### 8. Metrics
|
|
|
+
|
|
|
+```bash
|
|
|
+curl -fsS http://localhost:8080/metrics | grep ba_ingestd
|
|
|
+```
|
|
|
+
|
|
|
+Expected output:
|
|
|
+```
|
|
|
+# HELP ba_ingestd_alerts_received_total Number of inbound alerts by result.
|
|
|
+# TYPE ba_ingestd_alerts_received_total counter
|
|
|
+ba_ingestd_alerts_received_total{result="accepted",service="ingestd"} 1500
|
|
|
+ba_ingestd_alerts_received_total{result="deduped",service="ingestd"} 450
|
|
|
+...
|
|
|
+```
|
|
|
+
|
|
|
+## If something fails
|
|
|
+
|
|
|
+| symptom | first thing to check |
|
|
|
+|---|---|
|
|
|
+| `connection refused` on `/health` | `docker compose ps` — is the service running? |
|
|
|
+| `nats:4222: dial: connection refused` | nats container unhealthy; `docker compose logs nats` |
|
|
|
+| `redis:6379: dial: connection refused` | redis container unhealthy |
|
|
|
+| `401 bad_signature` on a "valid" call | openssl vs Go HMAC mismatch; re-check the `printf '%s.%s'` format |
|
|
|
+| `404` on `/v1/ingest` | wrong path; must be `POST /v1/ingest` (the handler is registered with `POST` method) |
|
|
|
+| `nats: no stream` | broker.Connect failed; logs will say so |
|
|
|
+| nothing in `alerts.*` subject | did `EnsureStreams` run? Check the ingestd log on first start |
|
|
|
+
|
|
|
+## What's NOT in M0 (M0 honest scope)
|
|
|
+
|
|
|
+These all *work* (return correct HTTP code, log correctly) but
|
|
|
+return no-ops. M-stuff that lands them is called out:
|
|
|
+
|
|
|
+- **routerd**: starts, /health works, but does not consume `alerts.*`
|
|
|
+ or resolve recipients. **M2**.
|
|
|
+- **deliverd**: starts, /health works, but does not consume
|
|
|
+ `deliveries.*` or call any sink. **M3** (FCM), **M5** (Telegram).
|
|
|
+- **admind**: starts, /health + /v1/ping work, no tenant CRUD yet.
|
|
|
+ **M8**.
|
|
|
+- **Multi-tenant isolation**: source registry is env-based. M2
|
|
|
+ swaps for DB and adds `WHERE company_id = $1` everywhere.
|
|
|
+- **Per-IP concurrency cap** (SPEC §22 layer 2): not yet wired.
|
|
|
+ **M5** with WS.
|
|
|
+- **Circuit breaker** (layer 6), **quarantine** (layer 7): **M9**.
|
|
|
+- **Schema-based protection** is a single Validate() call today.
|
|
|
+ Future: per-source allowlist, JSON Schema files.
|
|
|
+- **gRPC, MQTT, WebSocket ingest**: not yet. **M4**, **M5**, **M11**.
|
|
|
+- **ClickHouse archive job**: not yet. **M7**.
|
|
|
+- **Real FCM**: fakes only. **M3**.
|
|
|
+- **Source registry in DB**: env-only. **M2**.
|
|
|
+
|
|
|
+If M0 verification passes, M1 is unlocked: add a no-op `deliverd-fcm`
|
|
|
+worker that consumes `deliveries.fcm.<company_id>` and writes to
|
|
|
+the testfakes FCM server.
|