Metrics
Metrics — in-house observability
Section titled “Metrics — in-house observability”apps/metrics is a scoped Rust/Axum service that ingests client telemetry into a
self-hosted ClickHouse telemetry database. It replaces SaaS error/metrics tooling
(Sentry / Datadog / PostHog) with one pipeline that exposes three lenses — errors,
performance, and product — over a single wide table correlated by session_id.
Pipeline
Section titled “Pipeline”@kbve/observ (browser SDK) → POST metrics.kbve.com/api/v1/ingest/errors → apps/metrics (axum, port 5500) → jedi ClickHouseConfig::execute_insert (JSONEachRow, batched) → telemetry.errors_distributedThe ingest path reuses the shared jedi ClickHouse client — no new database
plumbing. A background tokio mpsc flusher batches rows by size and interval.
Requests are CORS-allowlisted, per-IP rate-limited, body-capped, and run through a
noise/PII sanitizer. Error grouping uses a sha256(project + error_type + top-5 stack frames) fingerprint. /readiness checks ClickHouse so a degraded backend
sheds load instead of silently 503ing.
Client-agnostic
Section titled “Client-agnostic”The platform field (web / unreal / unity / …) lets every client feed the
same pipe, so the errors lens shipped for the frontend extends to game clients
without schema changes.
Deployment
Section titled “Deployment”Runs in the kbve namespace behind the Cilium Gateway at metrics.kbve.com,
reusing the existing clickhouse-credentials ExternalSecret. Schema setup is a
job in the vector namespace that creates the telemetry database, the
errors_raw / errors_distributed tables, and the error_groups view.