Jelajahi Sumber

M5(3/3): README + PROMPT + SPEC bump to 'shipped 2026-06-14'

README: M5 in the status banner; M5_VERIFICATION + M5_SMOKE_LOG
in the docs index; WS ingest + live tail + WS loadgen noted in
the file tree; ingestd comment updated to include WS.

PROMPT: full M5 section appended. The three real bugs found
during verification are documented inline (subscribe-before-
upgrade, defer ordering, envelope shape). 'What's NOT in M5'
list calls out the deferred items (HTTP per-IP cap = M10,
JWT tail auth = M11, cross-node fan-out = M15+).

SPEC §23: M5 row updated to '✅ shipped 2026-06-14' with the
smoke summary inline.

Push: 3 commits ahead of origin/master, ready to push.
Luis Rosales 1 bulan lalu
induk
melakukan
873677235a
3 mengubah file dengan 118 tambahan dan 9 penghapusan
  1. 103 0
      PROMPT.md
  2. 14 8
      README.md
  3. 1 1
      SPEC.md

+ 103 - 0
PROMPT.md

@@ -493,3 +493,106 @@ What's NOT in M4 (and not supposed to be):
 Pushed: bc907d9 on master (1 commit for the code; this PROMPT
 bump + README + SPEC bump + the EMQX env-var refactor
 consolidated into a single follow-up commit).
+
+**2026-06-14 — M5 shipped (WebSocket ingest + live tail + per-IP cap)**
+
+What landed (~650 LoC Go + 2 SQL files across these commits):
+- `migrations/005_ws.up.sql`: adds `max_concurrent_connections`
+  column to `sources` and `companies` (defaults 32 and 64).
+- `internal/concurrency/perip.go`: tiny `sync.Map[ip]atomic.Int64`
+  with `Acquire/Release/InUse` and a janitor goroutine that
+  prunes entries idle > 5 min. No Redis on this hot path —
+  would self-DoS.
+- `internal/tailhub/hub.go`: in-process pub/sub. `Subscribe`
+  returns a `*Subscription` with a buffered `chan *Event`
+  (size 64) and a per-sub `Drops` counter; `Publish` is
+  best-effort, drops on full. `Filter.CompanyID` matches
+  single-company or all (empty).
+- `internal/wsclient/client.go`: thin `gorilla/websocket`
+  wrapper. `Connect` does the auth round-trip and stashes
+  the auth reply; `SendAlert` does a single WS write; `Close`
+  sends `CloseMessage(1000)` then TCP close. Used by
+  `loadgen-ws` and the smoke drivers.
+- `loadgen/cmd/ws/main.go`: `loadgen-ws` binary. Default
+  `normal` profile (30% dedupe). 3 modes: `normal`, `burst`.
+- `loadgen/cmd/m5drivers/tail/main.go`: `/tmp/m5-tail-test`
+  driver used by `scripts/m5_smoke.sh` steps 6 + 7.
+- `cmd/ingestd/ws.go`: WS ingest at `GET /v1/ingest/ws`.
+  Auth via first frame `{api_key}`. The handler does
+  per-IP `Acquire` (rejects + 503 on cap exceeded), then
+  loops on alert frames, sniffs `{alert, auth}` envelope
+  vs bare body, and calls the shared `scoped.ProcessAlert`.
+  Per-IP `Release` runs in the handler's defer.
+- `cmd/ingestd/wstail.go`: live tail at `GET /v1/tail/ws`.
+  Token via `Authorization: Bearer`, `X-BA-Tail-Token`, or
+  `?token=`. Optional `?company_id=` filter. **Subscribe
+  happens BEFORE the upgrade** so events in the
+  dial→subscribe window are not lost (a real bug we
+  caught in M5 verification — see PROMPT bug #1 below).
+- `cmd/ingestd/process.go`: `processDeps` now has
+  `Tail *tailhub.Hub` and `Transport string`. `ProcessAlert`
+  calls `d.Tail.Publish(tailhub.FromAlert(&a, d.Transport))`
+  on the `accepted` path, so the tail fans out alerts
+  from HTTP, MQTT, and WS through the same hub.
+- `cmd/ingestd/main.go`: wires `concurrency.NewPerIP`,
+  `tailhub.NewHub`, and the two new dep structs.
+- `internal/observability/metrics.go`: 5 new metrics —
+  `WSMessages`, `WSConnections`, `ConnectionRejected`,
+  `TailSubscribers`, `TailDropped` — all registered with
+  the ingestd service label.
+- `docker-compose.yml`: `BA_INGESTD_TAIL_TOKEN` and
+  `BA_INGESTD_MAX_CONCURRENT_PER_IP` env vars on ingestd.
+- `scripts/m5_smoke.sh`: 7-step live driver. Auto-builds
+  `/tmp/loadgen-ws`, `/tmp/m5-tail-test`, `/tmp/m5-perip-test`,
+  `/tmp/m5-badkey` on first run.
+
+Bug #1 — Tail subscribe vs upgrade race
+The first M5 verification run showed the live tail seeing
+`subs=0` at publish time even though the tail's gauge
+was 1. Root cause: `Subscribe` was called *after* the
+WS upgrade, leaving a window where the client thought
+it was connected but ingestd hadn't subscribed yet.
+The WS alert sent during that window was published to
+0 subscribers. Fix: subscribe first, *then* upgrade
+(so the client's dial-completion implies the subscription
+is in place). This is documented inline in
+`cmd/ingestd/wstail.go`.
+
+Bug #2 — Tail defer ordering
+The first run of the gauge-update defer fired while the
+subscription was still in the hub's map (defers run LIFO,
+so the metric defer fired *before* the unsubscribe
+defer). Result: the gauge stayed at 1 after the
+client disconnected. Fix: combine unsubscribe + gauge
+update into a single defer that runs after the stream
+loop exits.
+
+Bug #3 — Badkey envelope shape
+The first M5 step-4 test sent the HMAC as
+`auth: {"hmac": "v1=..."}` (a map). The server's envelope
+parser declared `auth` as a `string`, so the unmarshal
+failed, the parser fell back to using the whole envelope
+as the alert body, and validation rejected the alert
+with "company_id missing". Fix: send `auth` as a string
+`"t=...,v1=..."`, same shape as the MQTT envelope.
+`scripts/m4_smoke.sh` already had this correct — copy-
+pasted the pattern into M5.
+
+What's NOT in M5 (and not supposed to be):
+- HTTP `POST /v1/ingest` per-IP cap (M10 — the M5-bump
+  in SPEC §22; we ship the gate now and the wire-up to
+  HTTP is its own small PR)
+- JWT-based tail auth (M11 security milestone)
+- Cross-node tail fan-out via Redis pub/sub (M15+ when
+  ingestd runs as a cluster)
+- A "send test alert" UI button in the tail (deferred —
+  operators can `wscat` and a `loadgen-ws --count 1`)
+
+Pushed: 3 commits on master mirroring M0–M4's pattern:
+1. M5(1/3): code (migrations + 3 new packages + ws.go +
+   wstail.go + process.go + main.go + metrics + loadgen-ws)
+2. M5(2/3): verified (M5_VERIFICATION.md +
+   scripts/m5_smoke.sh + M5_SMOKE_LOG.md, 3 consecutive
+   green runs, 13/13 checks each, +36 deliveries
+   cumulative)
+3. M5(3/3): this PROMPT bump + README + SPEC §23.

+ 14 - 8
README.md

@@ -7,7 +7,7 @@ normalizes them, resolves recipients via `companies` → `groups` →
 Telegram, SMS, email, voice, Slack, MS Teams, and arbitrary outbound
 webhooks.
 
-> **Status**: M0 + M1 + M2 + M3 + M4 **shipped** 2026-06-14. M0 is the
+> **Status**: M0 + M1 + M2 + M3 + M4 + M5 **shipped** 2026-06-14. M0 is the
 > single-host docker-compose stack + 4 Go services + loadgen-http
 > + alert schema. M1 is the end-to-end: signed webhook → broker →
 > router → deliverd-fcm → fakefcmd (live-verified, 1530 deliveries
@@ -22,11 +22,14 @@ webhooks.
 > ingest (EMQX 5.10.4 broker with per-source ACL on
 > `ba/<co>/<src>/incoming`, ingestd's MQTT subscriber reuses the
 > same `ProcessAlert` pipeline as HTTP, loadgen-mqtt publisher,
-> 12 deliveries in 5-step smoke with 0 failures).
-> See `M0_VERIFICATION.md`, `M1_VERIFICATION.md`,
-> `M2_VERIFICATION.md`, `M3_VERIFICATION.md`, `M4_VERIFICATION.md`,
-> `M1_SMOKE_LOG.md`, `M2_SMOKE_LOG.md`, `M3_SMOKE_LOG.md`, and
-> `M4_SMOKE_LOG.md` for the smoke tests.
+> 12 deliveries in 5-step smoke with 0 failures). M5 is
+> WebSocket ingest (`GET /v1/ingest/ws` with `{api_key}` auth
+> frame, 35-conn load test, per-IP cap 32) + live tail
+> (`GET /v1/tail/ws?token=...&company_id=...` for operators,
+> in-process pub/sub fan-out, 13/13 checks green across 3
+> consecutive smoke runs).
+> See `M0_VERIFICATION.md` … `M5_VERIFICATION.md` and
+> `M1_SMOKE_LOG.md` … `M5_SMOKE_LOG.md` for the smoke tests.
 > Spec is in `SPEC.md`, diagrams in `ARCHITECTURE.md`, build log
 > in `PROMPT.md`.
 
@@ -50,14 +53,16 @@ M1_VERIFICATION.md  — M1 smoke test (end-to-end → fakefcmd)
 M2_VERIFICATION.md  — M2 smoke test (recipient resolution)
 M3_VERIFICATION.md  — M3 smoke test (Telegram delivery + bot)
 M4_VERIFICATION.md  — M4 smoke test (MQTT ingest + EMQX ACL)
+M5_VERIFICATION.md  — M5 smoke test (WS ingest + live tail + per-IP cap)
 M1_SMOKE_LOG.md     — M1 live run results
 M2_SMOKE_LOG.md     — M2 live run results
 M3_SMOKE_LOG.md     — M3 live run results
 M4_SMOKE_LOG.md     — M4 live run results
-docker-compose.yml  — single-host M0–M3 stack
+M5_SMOKE_LOG.md     — M5 live run results (3 consecutive green)
+docker-compose.yml  — single-host M0–M5 stack
 Dockerfile          — multi-stage build for all 7 binaries
 .env.example        — every BA_* knob documented
-cmd/ingestd/        — HTTP POST handler (M0) + MQTT subscriber (M4); M5 = WS, M11 = TLS
+cmd/ingestd/        — HTTP POST handler (M0) + MQTT subscriber (M4) + WS ingest (M5); M11 = TLS
 cmd/routerd/        — M2 rules engine + M3 channel union
 cmd/deliverd-fcm/   — M1 FCM HTTP v1 delivery (renamed from deliverd M3)
 cmd/deliverd-telegram/ — M3 Telegram Bot API delivery
@@ -65,6 +70,7 @@ cmd/telegramd/      — M3 long-polling bot loop + command handler
 cmd/admind/         — scaffold + /v1/ping (M8)
 loadgen/cmd/http/   — HTTP traffic generator (M0)
 loadgen/cmd/mqtt/   — MQTT traffic generator (M4)
+loadgen/cmd/ws/     — WebSocket traffic generator (M5)
 internal/alert/     — Alert v1 type + Validate() + Severity.Rank
 internal/broker/    — NATS JetStream wrapper
 internal/config/    — env-driven config

+ 1 - 1
SPEC.md

@@ -887,7 +887,7 @@ ingestd_rejection_latency_seconds_bucket{transport,reason} histogram
 | M2 | Recipient resolution | per-source `allowed_targets` honored, subscriptions applied | **✅ shipped 2026-06-13** (live smoke test all 7 steps green; see `M2_SMOKE_LOG.md`) |
 | M3 | Telegram delivery + bot commands | user can `/subscribe` and receive an alert via Telegram | **✅ shipped 2026-06-14** (live smoke test all 8 steps green; 14 deliveries, 8 sendMessage calls; see `M3_VERIFICATION.md` + `M3_SMOKE_LOG.md`) |
 | M4 | MQTT ingest | EMQX up, QoS 1, per-company topic ACLs | **✅ shipped 2026-06-14** (live smoke test all 5 steps green; 12 deliveries, 0 failures; see `M4_VERIFICATION.md` + `M4_SMOKE_LOG.md`) |
-| M5 | WebSocket ingest + live tail | admin UI (or wscat) sees alerts as they arrive; layer 2 in |
+| M5 | WebSocket ingest + live tail | admin UI (or wscat) sees alerts as they arrive; layer 2 in | **✅ shipped 2026-06-14** (live smoke test all 7 steps green; 3 consecutive green runs; 13/13 checks each; +36 deliveries cumulative; see `M5_VERIFICATION.md` + `M5_SMOKE_LOG.md`) |
 | M6 | Dedupe + dedupe_count | burst of 100 identical alerts → recipient sees "×100"; dedupe-aware rate shaping in |
 | M7 | Timescale + ClickHouse | 7d retention + archive job |
 | M8 | DLQ + replay UI | operator can replay a failed delivery |