# M3 Verification — Telegram delivery + bot commands This is the live smoke test for the M3 milestone (**Telegram delivery + bot commands**, SPEC §8). It exercises the new path end-to-end: 1. `telegram_bots` table (one bot per company in M3) 2. `individuals.telegram_chat_id` + `telegram_user_id` + `telegram_invite_code` + `mute_until` 3. **Resolver** resolves channel='telegram' targets from the same subscription set when `channel_mask` contains 'telegram' AND the individual has a non-null `telegram_chat_id` 4. **`telegramd`** — long-polling bot loop that handles `/start`, `/subscribe`, `/unsubscribe`, `/preferences`, `/status`, `/mute`, `/unmute` 5. **`deliverd-telegram`** — per-channel delivery worker that consumes `deliveries.telegram.` and posts to the Bot API (`sendMessage`) 6. **`faketgmd`** — fake Bot API server for the smoke test 7. **Channel split** — `deliverd` was renamed to `deliverd-fcm` (per the user's M3 Q2 answer: two binaries, not a registry) 8. **Smoke flow** — bot commands drive the test; alerts drive the delivery check; faketgmd `/admin/sent` proves the messages actually went out Hard-fail: if the resolver returns zero targets, routerd drops the alert with a log line. Same as M2. ## Prerequisites - Stack is up: `docker compose up -d` - Migrations applied: `migrations/004_telegram.up.sql` is part of the seed sidecar's `*.up.sql` walk - Seed has been run: `docker compose up seed` (apply M1, M2, M3 migrations + all three seeds) - HMAC secret for the `acme-001:prom-prod` source is `s3cret-acme` (matches `BA_INGESTD_SOURCES` env) - faketgmd is reachable on `:8830` (host port mapped from the compose) ## Seed summary (M3 additions on top of M2) | individual | telegram_chat_id | telegram_user_id | telegram_invite_code | notes | |---|---|---|---|---| | ind-acme-001 (Alice) | 1001 | 900001 | (burned) | **pre-linked** in `seed_m3.sql` so the very first POST exercises both fcm and telegram delivery paths | | ind-acme-002 (Bob) | NULL | NULL | `acme-bob-002` | unlinked at start; linked via `/start ` in step 4 | | ind-acme-003 (Carol) | NULL | NULL | `acme-carol-003` | **never linked** in M3 — exercises "telegram skipped when no chat_id" path; quiet hours 00:00–23:59 UTC keep her out of non-inminent alerts | Subscriptions: all three individuals now have `channel_mask = ["fcm","telegram"]` (M2 had just `["fcm"]`). Bob's `min_severity` is `critical` (M2 default). One telegram bot per company: `(acme-001, "primary")` with token `fake-tg-bot-token-acme-001`. ## Automated smoke ```bash bash scripts/m3_smoke.sh ``` The script: 1. **Resets state** — Bob is unlinked, his subscription is restored to `min_severity=critical`, the `deliveries` table is truncated, and `faketgmd` is reset. **It also `docker compose restart telegramd`** so the bot's in-memory long-poll offset clears (see "Known quirks" below). 2. Walks steps 2–8 below and asserts the expected row counts. 3. Exits 0 on success, $fail_count on failure. ## Step-by-step (also documented in `M3_SMOKE_LOG.md`) ### Step 1 — confirm M3 seed applied ```bash docker exec -i broad-announce-postgres-1 psql -U ba -d ba -c " SELECT id, full_name, telegram_chat_id, telegram_user_id, telegram_invite_code, mute_until FROM individuals ORDER BY id; " ``` Expected: 3 rows. Alice has `telegram_chat_id=1001`, `telegram_user_id=900001`, no invite code (she was already linked). Bob and Carol have NULL chat_id/user_id and a non-null invite code. ### Step 2 — send a `warning` storage alert ```bash SECRET=s3cret-acme BODY='{"company_id":"acme-001","source_id":"prom-prod","severity":"warning","category":"storage","title":"Disk 80% full","body":"db-prod-04","data":{"host":"db-prod-04"},"dedupe_key":"m3-s2"}' TS=$(date +%s) SIG=$(printf '%s.%s' "$TS" "$BODY" | openssl dgst -sha256 -hmac "$SECRET" -hex | awk '{print $2}') curl -s -X POST http://localhost:8800/v1/ingest \ -H "Content-Type: application/json" \ -H "X-BA-Signature: t=$TS,v1=$SIG" \ --data "$BODY" ``` Wait 4s, then check `deliveries`: ```bash docker exec -i broad-announce-postgres-1 psql -U ba -d ba -c " SELECT individual_id, channel, status FROM deliveries ORDER BY id; " ``` **Expected**: 2 rows — Alice fcm (sent) + Alice telegram (sent). Bob has no telegram_chat_id (resolver returns no telegram endpoint for him). Carol is in quiet hours. ### Step 3 — send a `critical` storage alert (Bob unlinked) ```bash BODY='{"company_id":"acme-001","source_id":"prom-prod","severity":"critical","category":"storage","title":"Disk full","body":"db-prod-04","data":{"host":"db-prod-04"},"dedupe_key":"m3-s3"}' # same HMAC dance ``` **Expected** (cumulative): 3 more rows this step. - Alice fcm + Alice telegram - Bob fcm (Bob's min_severity=critical passes, so he gets fcm) - Bob telegram is **not** delivered because he is unlinked. Totals after step 3: fcm=3, telegram=2. ### Step 4 — Bob links his Telegram account ```bash curl -s -X POST http://localhost:8830/admin/queue \ -H "Content-Type: application/json" \ --data '{ "bot_token": "fake-tg-bot-token-acme-001", "user_id": 900002, "chat_id": 2002, "text": "/start acme-bob-002", "first_name":"Bob SRE" }' ``` Wait 4s. The bot (`telegramd`) long-polls faketgmd, sees the queued update, runs `/start acme-bob-002` through the handler. The handler: 1. Atomically `UPDATE individuals SET telegram_user_id=900002, telegram_chat_id='2002', telegram_invite_code=NULL WHERE telegram_invite_code='acme-bob-002'` 2. Fetches Alice's name (oh wait, Bob's name) and replies "Linked. Welcome, Bob SRE. …" **Expected**: ```bash docker exec -i broad-announce-postgres-1 psql -U ba -d ba -c " SELECT id, telegram_chat_id, telegram_user_id, telegram_invite_code FROM individuals WHERE id='ind-acme-002';" # ind-acme-002 | 2002 | 900002 | NULL curl -sS http://localhost:8830/admin/sent | jq '.items[-1]' # should be a "Linked. Welcome, Bob SRE. …" reply ``` ### Step 5 — send another `critical` storage alert ```bash BODY='{"company_id":"acme-001","source_id":"prom-prod","severity":"critical","category":"storage","title":"Disk full","body":"db-prod-04","data":{"host":"db-prod-04"},"dedupe_key":"m3-s5"}' ``` **Expected** (cumulative): 4 more rows this step. - Alice fcm + Alice telegram - Bob fcm + Bob telegram (Bob is now linked!) Totals after step 5: fcm=5, telegram=4. ### Step 6 — send an `inminent_colapse` alert ```bash BODY='{"company_id":"acme-001","source_id":"prom-prod","severity":"inminent_colapse","category":"power","title":"PDU overload imminent","body":"rack-B","data":{"host":"rack-b"},"dedupe_key":"m3-s6"}' ``` **Expected** (cumulative): 5 more rows this step. - Alice fcm + Alice telegram - Bob fcm + Bob telegram (Bob's min=critical passes since inminent_colapse > critical) - Carol fcm (Carol's quiet hours 00:00–23:59 are bypassed by inminent_colapse, but she has no telegram_chat_id, so no telegram) Totals after step 6: fcm=8, telegram=6. ### Step 7 — Bob runs `/preferences` ```bash curl -s -X POST http://localhost:8830/admin/queue \ -H "Content-Type: application/json" \ --data '{ "bot_token": "fake-tg-bot-token-acme-001", "user_id": 900002, "chat_id": 2002, "text": "/preferences", "first_name":"Bob SRE" }' ``` **Expected**: faketgmd receives a bot reply whose text starts with "Your subscriptions:" and lists the prom-prod row with `min=critical, channels=["fcm","telegram"]`. ### Step 8 — verify faketgmd totals ```bash curl -sS http://localhost:8830/admin/sent | jq '.count' # should be 8: 4 alert alerts (alice×2, alice, alice+bob) + 1 linked-reply + 1 prefs-reply = 8 ``` (Wait, count: 4 alert notifications: step 2 alice, step 3 alice, step 5 alice+bob, step 6 alice+bob = 6 alert msgs + 1 linked-reply + 1 prefs-reply = 8.) ## Known quirks (and why the smoke script does what it does) 1. **faketgmd's `getUpdates` is a fake.** In production, Telegram's getUpdates tracks the offset server-side and returns updates >= offset. Our fake uses an in-memory `offsetByBot` map. The bot (`telegramd`) keeps its offset in a goroutine-local `var offset int64` that advances as updates come in. The smoke script calls `docker compose restart telegramd` after every reset so the bot's in-process offset goes back to 0 and aligns with faketgmd's cleared state. In production this is a non-issue (Telegram is the real source of truth). 2. **faketgmd is in-process only.** It keeps `sent`/`queue` in memory. Restarting the container (which the smoke script does not do by default) clears all state. M3 does not promise persistence for the fake. 3. **`deliverd` was renamed to `deliverd-fcm`.** The old container is gone. If you have an older compose, the `deliverd-telegram` service will start but the old `deliverd` service won't, leaving you with FCM disabled. Always `docker compose up -d` with the M3 compose. ## What this proves about M3 1. **Multi-channel resolver works.** The single CTE `internal/routing/routing.go` UNION-ALLs an `fcm_rows` branch and a `tg_rows` branch. `fcm_rows` joins on `fcm_tokens`; `tg_rows` filters to `telegram_chat_id IS NOT NULL`. Same CTE, same cost, two channels. 2. **Per-channel delivery binaries are independent.** `deliverd-fcm` and `deliverd-telegram` each subscribe to `deliveries..`. Restarting one does not affect the other. 3. **Bot long-polling works.** `telegramd` reads the active bots from `telegram_bots`, long-polls each, dispatches commands to the handler, and replies via `sendMessage`. 4. **`/start` is the only link path.** Admin creates the individual with an invite code; user runs `/start `. No self-registration. The handler atomically claims the code so two simultaneous `/start`s can't race. 5. **Subscriptions have channel_mask.** Bob's M2 subscription was `["fcm"]`; M3 expanded it to `["fcm","telegram"]` via the seed migration. The resolver honors the mask. 6. **Quiet hours still apply to telegram.** Carol's 00:00–23:59 window blocked her from everything except `inminent_colapse`, where the bypass held. 7. **Same severity bypass as M2.** `inminent_colapse` bypasses quiet hours regardless of channel.