|
|
@@ -493,3 +493,106 @@ What's NOT in M4 (and not supposed to be):
|
|
|
Pushed: bc907d9 on master (1 commit for the code; this PROMPT
|
|
|
bump + README + SPEC bump + the EMQX env-var refactor
|
|
|
consolidated into a single follow-up commit).
|
|
|
+
|
|
|
+**2026-06-14 — M5 shipped (WebSocket ingest + live tail + per-IP cap)**
|
|
|
+
|
|
|
+What landed (~650 LoC Go + 2 SQL files across these commits):
|
|
|
+- `migrations/005_ws.up.sql`: adds `max_concurrent_connections`
|
|
|
+ column to `sources` and `companies` (defaults 32 and 64).
|
|
|
+- `internal/concurrency/perip.go`: tiny `sync.Map[ip]atomic.Int64`
|
|
|
+ with `Acquire/Release/InUse` and a janitor goroutine that
|
|
|
+ prunes entries idle > 5 min. No Redis on this hot path —
|
|
|
+ would self-DoS.
|
|
|
+- `internal/tailhub/hub.go`: in-process pub/sub. `Subscribe`
|
|
|
+ returns a `*Subscription` with a buffered `chan *Event`
|
|
|
+ (size 64) and a per-sub `Drops` counter; `Publish` is
|
|
|
+ best-effort, drops on full. `Filter.CompanyID` matches
|
|
|
+ single-company or all (empty).
|
|
|
+- `internal/wsclient/client.go`: thin `gorilla/websocket`
|
|
|
+ wrapper. `Connect` does the auth round-trip and stashes
|
|
|
+ the auth reply; `SendAlert` does a single WS write; `Close`
|
|
|
+ sends `CloseMessage(1000)` then TCP close. Used by
|
|
|
+ `loadgen-ws` and the smoke drivers.
|
|
|
+- `loadgen/cmd/ws/main.go`: `loadgen-ws` binary. Default
|
|
|
+ `normal` profile (30% dedupe). 3 modes: `normal`, `burst`.
|
|
|
+- `loadgen/cmd/m5drivers/tail/main.go`: `/tmp/m5-tail-test`
|
|
|
+ driver used by `scripts/m5_smoke.sh` steps 6 + 7.
|
|
|
+- `cmd/ingestd/ws.go`: WS ingest at `GET /v1/ingest/ws`.
|
|
|
+ Auth via first frame `{api_key}`. The handler does
|
|
|
+ per-IP `Acquire` (rejects + 503 on cap exceeded), then
|
|
|
+ loops on alert frames, sniffs `{alert, auth}` envelope
|
|
|
+ vs bare body, and calls the shared `scoped.ProcessAlert`.
|
|
|
+ Per-IP `Release` runs in the handler's defer.
|
|
|
+- `cmd/ingestd/wstail.go`: live tail at `GET /v1/tail/ws`.
|
|
|
+ Token via `Authorization: Bearer`, `X-BA-Tail-Token`, or
|
|
|
+ `?token=`. Optional `?company_id=` filter. **Subscribe
|
|
|
+ happens BEFORE the upgrade** so events in the
|
|
|
+ dial→subscribe window are not lost (a real bug we
|
|
|
+ caught in M5 verification — see PROMPT bug #1 below).
|
|
|
+- `cmd/ingestd/process.go`: `processDeps` now has
|
|
|
+ `Tail *tailhub.Hub` and `Transport string`. `ProcessAlert`
|
|
|
+ calls `d.Tail.Publish(tailhub.FromAlert(&a, d.Transport))`
|
|
|
+ on the `accepted` path, so the tail fans out alerts
|
|
|
+ from HTTP, MQTT, and WS through the same hub.
|
|
|
+- `cmd/ingestd/main.go`: wires `concurrency.NewPerIP`,
|
|
|
+ `tailhub.NewHub`, and the two new dep structs.
|
|
|
+- `internal/observability/metrics.go`: 5 new metrics —
|
|
|
+ `WSMessages`, `WSConnections`, `ConnectionRejected`,
|
|
|
+ `TailSubscribers`, `TailDropped` — all registered with
|
|
|
+ the ingestd service label.
|
|
|
+- `docker-compose.yml`: `BA_INGESTD_TAIL_TOKEN` and
|
|
|
+ `BA_INGESTD_MAX_CONCURRENT_PER_IP` env vars on ingestd.
|
|
|
+- `scripts/m5_smoke.sh`: 7-step live driver. Auto-builds
|
|
|
+ `/tmp/loadgen-ws`, `/tmp/m5-tail-test`, `/tmp/m5-perip-test`,
|
|
|
+ `/tmp/m5-badkey` on first run.
|
|
|
+
|
|
|
+Bug #1 — Tail subscribe vs upgrade race
|
|
|
+The first M5 verification run showed the live tail seeing
|
|
|
+`subs=0` at publish time even though the tail's gauge
|
|
|
+was 1. Root cause: `Subscribe` was called *after* the
|
|
|
+WS upgrade, leaving a window where the client thought
|
|
|
+it was connected but ingestd hadn't subscribed yet.
|
|
|
+The WS alert sent during that window was published to
|
|
|
+0 subscribers. Fix: subscribe first, *then* upgrade
|
|
|
+(so the client's dial-completion implies the subscription
|
|
|
+is in place). This is documented inline in
|
|
|
+`cmd/ingestd/wstail.go`.
|
|
|
+
|
|
|
+Bug #2 — Tail defer ordering
|
|
|
+The first run of the gauge-update defer fired while the
|
|
|
+subscription was still in the hub's map (defers run LIFO,
|
|
|
+so the metric defer fired *before* the unsubscribe
|
|
|
+defer). Result: the gauge stayed at 1 after the
|
|
|
+client disconnected. Fix: combine unsubscribe + gauge
|
|
|
+update into a single defer that runs after the stream
|
|
|
+loop exits.
|
|
|
+
|
|
|
+Bug #3 — Badkey envelope shape
|
|
|
+The first M5 step-4 test sent the HMAC as
|
|
|
+`auth: {"hmac": "v1=..."}` (a map). The server's envelope
|
|
|
+parser declared `auth` as a `string`, so the unmarshal
|
|
|
+failed, the parser fell back to using the whole envelope
|
|
|
+as the alert body, and validation rejected the alert
|
|
|
+with "company_id missing". Fix: send `auth` as a string
|
|
|
+`"t=...,v1=..."`, same shape as the MQTT envelope.
|
|
|
+`scripts/m4_smoke.sh` already had this correct — copy-
|
|
|
+pasted the pattern into M5.
|
|
|
+
|
|
|
+What's NOT in M5 (and not supposed to be):
|
|
|
+- HTTP `POST /v1/ingest` per-IP cap (M10 — the M5-bump
|
|
|
+ in SPEC §22; we ship the gate now and the wire-up to
|
|
|
+ HTTP is its own small PR)
|
|
|
+- JWT-based tail auth (M11 security milestone)
|
|
|
+- Cross-node tail fan-out via Redis pub/sub (M15+ when
|
|
|
+ ingestd runs as a cluster)
|
|
|
+- A "send test alert" UI button in the tail (deferred —
|
|
|
+ operators can `wscat` and a `loadgen-ws --count 1`)
|
|
|
+
|
|
|
+Pushed: 3 commits on master mirroring M0–M4's pattern:
|
|
|
+1. M5(1/3): code (migrations + 3 new packages + ws.go +
|
|
|
+ wstail.go + process.go + main.go + metrics + loadgen-ws)
|
|
|
+2. M5(2/3): verified (M5_VERIFICATION.md +
|
|
|
+ scripts/m5_smoke.sh + M5_SMOKE_LOG.md, 3 consecutive
|
|
|
+ green runs, 13/13 checks each, +36 deliveries
|
|
|
+ cumulative)
|
|
|
+3. M5(3/3): this PROMPT bump + README + SPEC §23.
|