Operating Husaria

Tracing

Tracing #

Husaria accepts a distributed-trace identifier on every inbound request, echoes it back on the response, and synthesises a valid context when no upstream sent one. This means an OpenTelemetry collector downstream of Husaria always sees a well-formed trace ID — even if the request originated from a curl call that didn't bother to set the header.

There are no env vars to enable this — it runs unconditionally on every request.

Supported header formats #

Husaria checks three header formats on inbound, in priority order:

PriorityHeaderFormatSource
1traceparentW3C Trace Context: 00-{32-hex-trace-id}-{16-hex-span-id}-{flags}OpenTelemetry SDKs, modern instrumented services
2X-Trace-ID32-hex (legacy)Pre-OTel internal services
3X-B3-TraceId16- or 32-hexZipkin / B3 propagation

The first one found wins. Malformed values are rejected — Husaria mints a fresh traceparent rather than echoing back a broken header.

Normalisation #

All inbound IDs are normalised to a 32-hex string internally. A 16-hex Zipkin ID is left-padded with zeros to 32 hex chars before it leaves Husaria. This is how the W3C spec lifts shorter IDs.

Synthesis #

If no upstream sent any of the three headers — or if the value was malformed — synthesizeTraceparent mints a fresh well-formed header:

traceparent: 00-{random 32-hex trace-id}-{random 16-hex span-id}-01

The flags byte is 01 (sampled). This is intentional: if a client didn't bother sending a header, you almost certainly do want this request traced.

Outbound #

Husaria echoes both of these headers on the response:

  • X-Trace-ID: <32-hex> — for legacy log-aggregation pipelines that grep this field.
  • traceparent: 00-{32-hex}-{16-hex}-{flags} — for OTel-aware proxies and collectors downstream.

Logs are correlated #

The trace ID is attached to every log entry for that request, under the field key trace_id. With JSON logging (which ENVIRONMENT=production selects automatically), this is a top-level field — easy to join against traces in Tempo, Jaeger, etc.

{
  "level": "info",
  "ts": "...",
  "msg": "REQ",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "status": 200,
  "duration_ms": 1.96
}

Wiring example — OpenTelemetry collector #

Husaria does not export spans itself — it only propagates trace IDs. To capture real spans, run an OTel collector in front of (or beside) Husaria.

Sidecar pattern #

# kubernetes deployment fragment
spec:
  template:
    spec:
      containers:
        - name: husaria
          image: ghcr.io/lukaszraczylo/husaria:latest
          # …
        - name: otel-collector
          image: otel/opentelemetry-collector-contrib:latest
          args: ["--config=/conf/otel.yaml"]
          # collector receives spans from Husaria's upstream (Envoy, NGINX
          # service-mesh sidecar, gateway, etc.) and exports them to
          # your trace backend (Jaeger, Tempo, Honeycomb, …)

The collector observes the traceparent headers Husaria emits and (if your ingress layer is OTel-instrumented) the spans the ingress layer captures around Husaria's request handler.

Gateway pattern #

If you already run an ingress controller with OTel support — Envoy, NGINX with the OTel module, Traefik with the OTel middleware — that layer can capture spans for every request flowing through. Husaria's trace-ID passthrough means the parent-child relationship is preserved across the ingress → Husaria → Postgres boundary.

What you don't get #

  • No span export from Husaria itself. Husaria is a propagation participant, not a span emitter. The request/response is one logical span owned by the layer in front; Husaria's logs are the correlation breadcrumb.
  • No automatic Postgres span. Queries to Postgres run inside Husaria's process — they're not instrumented as separate child spans on the wire. Use pg_stat_statements (and the console Performance tab) for SQL-level diagnostics.

Debugging tip #

Want to confirm tracing is working without a full collector? Send a request and check the response headers:

curl -i http://localhost:8080/v1/graphql \
  -H "X-Husaria-Admin-Secret: ..." \
  -H 'traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01' \
  -d '{"query":"{ __typename }"}'

The response should carry the same traceparent (or a synthesised one if you omit the header). Logs should show trace_id matching the trace-ID portion of traceparent.