notify
Every service's events in one place, delivered where people want them: Telegram, e-mail, browser push,
webhooks, and plain topics anyone can post to. Host: notify.<d>. Binary: notifyd. Stage 4.
Concepts
- Event: from a service's outbox:
type (app.crashed), service, team, project, environment,
actor, target, severity (info, warning, critical), title, body, link, time. Stored with
retention (90 days by default); the activity feed in home reads it.
- Channel: Telegram (the instance bot: a user links their chat with
/start <code>; groups too),
e-mail (through instance e-mail delivery), Web Push (panels register a service worker), webhook
(signed with HMAC), and later Slack / Discord / ntfy-compatible apps.
- Subscription: a user's or a team's rule: which events (types, services, projects, minimum
severity) go to which channels, with quiet hours and digest (instant, hourly, daily).
- Topic:
POST https://notify.<d>/t/<topic> with a body (and optional X-Title, X-Priority,
X-Tags) publishes to a team topic with a token; subscribers get it on their channels. This is the
"curl to Telegram" use for scripts and cron jobs.
- Delivery: per channel, with retries, rate limits per channel, a delivery log, and folding of
repeated events (one message "app crashed 5 times in 10 minutes").
Telegram bot
Go (ADR 0011). Commands: /start, /link, /mute 1h, /subs, /status (failing monitors and apps of
your teams). Inline buttons on events: open, acknowledge, rollback (for deploy failures, with a confirm).
API
POST /api/v1/events services (client credentials), batches from kit/events; idempotent by (service, id)
GET /api/v1/events?team=&project=&service=&type=&severity=&before= GET /api/v1/events/{seq} (with deliveries)
POST /api/v1/events/{seq}/ack developers and up
GET/POST /api/v1/channels · POST /api/v1/channels/telegram (a /start code) · GET/PATCH/DELETE /api/v1/channels/{id}
POST /api/v1/channels/{id}/test · /verify (e-mail code) · /secret (new webhook secret, shown once)
GET/POST/PATCH/DELETE /api/v1/subscriptions · /topics
GET/PUT /api/v1/prefs quiet hours and time zone
GET /api/v1/activity · /api/v1/me · /api/v1/stats
POST /t/{topic} plain text or JSON; X-Title, X-Priority (1–5, low…urgent), X-Tags, X-Link
/mcp notify_list_events, notify_get_event, notify_ack_event, notify_publish,
notify_list_topics, notify_create_topic, notify_list_channels
How it works
-
Routing (internal/route): a person's channel hears events of the teams they are in (id's
/api/v1/directory, refreshed every minute; superusers also hear events of no team), a team's channel
its own team only. A subscription narrows by team, project, services, types (deploy.*) and minimum
severity. Critical events skip digests, quiet hours and mutes.
-
Delivery (internal/deliver): repeats (same service, type and target) within 10 minutes are
counted into the first message, which Telegram edits; several due at once for a channel go as one
message; failures retry with back-off six times; a chat that blocked the bot is removed.
-
Telegram: messages use premium emoji of the TgAndroidIcons pack only; buttons Open and
Acknowledge. /start <code> links a private chat (person) or a group (team, admins only); /mute 1h,
/subs, /status.
-
Webhooks: JSON {delivery, event, team, project, url}, X-Xvii-Signature: sha256=<HMAC of the body>,
X-Xvii-Event, X-Xvii-Delivery; private and loopback addresses are refused unless PRIVATE_WEBHOOKS.
-
Limits (ADR 0013): notify.channels (a person's count against their personal team),
notify.topics, notify.posts (topic posts a day).
-
Events from pages: deploy.created, deploy.failed (critical in production), deploy.expired,
domain.verified, domain.lost.
-
Web Push: a browser subscribes in the panel (service worker /sw.js, VAPID key in DATA_DIR/vapid.pem); messages are encrypted per RFC 8291 and dropped subscriptions remove the channel.
Left: Slack / Discord / ntfy apps, the bot's rollback button (with apps), home's feed of events.
E-mails carry the text and the same as HTML in the instance's look (home's name, accent and logo, read
from its catalog every ten minutes), quoted-printable; subjects start with the instance's name.