In-house telemetry pipelineMetrics
KBVE's in-house observability platform — a scoped Rust/axum service that ingests client telemetry into a self-hosted ClickHouse database, replacing Sentry, Datadog, and PostHog with one pipeline over a single wide table.
Three lenses, one table
One pipeline exposes errors, performance, and product lenses over a single wide table correlated by session_id — no SaaS error/metrics tooling.
- Sink — telemetry.errors_distributed on ClickHouse.
- Grouping — sha256(project + type + top-5 frames).
What it gives you
Features
Three lenses over one table
Errors, performance, and product views over a single wide table correlated by session_id, replacing Sentry / Datadog / PostHog.
Off-thread batched inserts
A background tokio mpsc flusher batches rows by size and interval through the shared jedi ClickHouse client — no new database plumbing.
Hardened ingest
Requests are CORS-allowlisted, per-IP rate-limited, body-capped, and run through a noise/PII sanitizer before insert.
Client-agnostic
A platform field (web / unreal / unity / …) lets every client feed the same pipe, so the errors lens extends to game clients with no schema changes.
What it is
In-house observability
apps/metrics is a scoped Rust/Axum service that ingests client telemetry into a
self-hosted ClickHouse telemetry database. It replaces SaaS error/metrics tooling
(Sentry / Datadog / PostHog) with one pipeline that exposes three lenses — errors,
performance, and product — over a single wide table correlated by session_id.
Ingest path
Pipeline
@kbve/observ (browser SDK) → POST metrics.kbve.com/api/v1/ingest/errors → apps/metrics (axum, port 5500) → jedi ClickHouseConfig::execute_insert (JSONEachRow, batched) → telemetry.errors_distributedThe ingest path reuses the shared jedi ClickHouse client — no new database
plumbing. A background tokio mpsc flusher batches rows by size and interval.
Requests are CORS-allowlisted, per-IP rate-limited, body-capped, and run through a
noise/PII sanitizer. Error grouping uses a sha256(project + error_type + top-5 stack frames) fingerprint. /readiness checks ClickHouse so a degraded backend
sheds load instead of silently 503ing.
Client-agnostic
Section titled “Client-agnostic”The platform field (web / unreal / unity / …) lets every client feed the
same pipe, so the errors lens shipped for the frontend extends to game clients
without schema changes.
Runtime
Deployment
Runs in the kbve namespace behind the Cilium Gateway at metrics.kbve.com,
reusing the existing clickhouse-credentials ExternalSecret. Schema setup is a
job in the vector namespace that creates the telemetry database, the
errors_raw / errors_distributed tables, and the error_groups view.
Questions
Frequently asked
What is the Metrics service?
Metrics (apps/metrics) is KBVE's in-house observability platform, a scoped Rust/axum service that ingests client telemetry into a self-hosted ClickHouse telemetry database. It replaces SaaS tooling like Sentry, Datadog, and PostHog with one pipeline exposing errors, performance, and product lenses over a single wide table correlated by session_id.
How does the Metrics ingest pipeline work?
The browser SDK @kbve/observ POSTs to metrics.kbve.com/api/v1/ingest, the axum service on port 5500 reuses the shared jedi ClickHouse client to batch JSONEachRow inserts via a tokio mpsc flusher, and rows land in telemetry.errors_distributed. Requests are CORS-allowlisted, rate-limited, body-capped, and PII-sanitized.
How is Metrics client-agnostic across web and game clients?
A platform field (web, unreal, unity, and more) lets every client feed the same pipe, so the errors lens built for the frontend extends to Unreal and Unity game clients with no schema changes.