Metrics, logs and traces

analytics keeps the platform's telemetry beside its web analytics, under Infrastructure. Nothing to install: every service pushes it, and every node reports its server and the apps it runs.

What is collected

From	What
Every service	requests by route and status, response time, Go memory and goroutines, database connections, its logs, traces of requests across services
Services' own	ai: upstream calls, tokens, cost and quota windows; mail: deliveries; notify: notifications sent; backups: runs, their time and size
Servers	CPU, load, memory, disk, network: the control plane and every node
Apps	CPU, memory, restarts, network; requests through the node (status, response time); their logs; their own metrics endpoint

Services push every minute. Raw points are kept 7 days and hourly roll-ups 90 days; logs and traces 7 days. A superuser changes this under Alerts → Retention.

Your app's metrics

Serve the Prometheus text format inside the container and name it in the app's Settings → Metrics port (and path, /metrics by default). The node reads it every 30 seconds (up to 500 series); counters show as rates, gauges as they are. The app's page links to its requests, logs and metrics.

Logs are kept unless Keep logs in analytics is off; a container may keep 200 lines a second, and longer lines are cut at 8 KB. A line's level is read from what it says (error, warn, …).

Explore

Overview: every service up or down with its requests, errors, p95, memory; every server's CPU, memory, disk and network.
Metrics: pick a metric, a function (rate, sum, average, max, p50…p99), group by a label, filter by values; the address keeps the view, so a link shares it.
Logs: search, filter by source and level, follow live; a line inside a request opens its trace.
Traces: requests across services as a waterfall. A trace is kept when it is sampled (10%), failed (5xx) or took over a second.

Superusers see the platform; a team sees its own apps.

Alerts

A rule watches a function of a metric over the last five minutes — say rate(http_requests_total{code:5xx, source:ai}) > 0.1 for 5 minutes — and tells notify when it fires (analytics.alert.firing) and when it resolves. A service that stops pushing for three minutes is reported down (analytics.service.down) without a rule. A team's admins make rules on their apps' series.

Prometheus and Grafana

Every service answers /metrics to callers on its own machine. An outside Prometheus can scrape every series' latest value from https://analytics.xvii.network/api/v1/telemetry/metrics with the instance's telemetry token (/etc/xvii/telemetry.token on the control plane).