UUIDv7 ingestion validation

Opik lets a client choose the id of a trace or span it creates. That id must be a UUIDv7, and the timestamp embedded in its top 48 bits is what ClickHouse uses to place the row in a partition. UUIDv7 ingestion validation bounds that timestamp at write time, so a client embedding wrong timestamps fails loudly instead of quietly degrading the partition layout.

Why it exists

A client that embeds a wrong timestamp does more than mislabel one row. An id claiming a date years in the future lands in a partition that retention never reaches, and one claiming a date years in the past lands in a partition that has already been retained away. Either way the partition layout degrades, silently, and the row is hard to find again.

Bounding the timestamp at ingestion turns that into an HTTP 400 on the offending write, so a broken client surfaces as a failed request rather than as partition corruption discovered months later.

What is validated

Two separate checks run on a client-supplied id.

The version check. The id must be a version 7 UUID, or the write is rejected with reason: not_v7. This check is always on and none of the settings below affect it.

The window check. This is the configurable part, and it applies in two forms:

• Both bounds — when a trace or span is created with a caller-chosen id. An embedded timestamp further than the window into either the past (too_old) or the future (too_far_future) is rejected.
• The future bound only — when a single trace or span is updated by id, and wherever an existing id is referenced: a span’s traceId and parentSpanId, and ids referenced by dataset items, feedback scores, comments, attachments and guardrails. Old ids are legitimate on those paths — a late span on a month-old trace is normal — so only future-dating is rejected.

Where the ids come from

The checks live in the backend’s service layer, so the creation paths are covered alike: the REST API, the batch ingestion endpoints, and the OpenTelemetry endpoint.

Worth knowing if you only send OTel: the OpenTelemetry endpoint does not carry Opik ids, so the backend derives them — but it derives them from your spans’ start timestamps, not from the current time, so that a trace and its spans share one reference point. An exporter replaying spans whose start times are outside the window is therefore rejected too, even though it never set an id itself. Backfilling historical traces over OTel needs a window wide enough to cover them.

Before you turn it on

Configuration

Together the first two settings give three modes:

Docker Compose

The Compose file reads all three from the environment, so exporting them before starting Opik is enough:

$
export UUID_VALIDATION_ENABLED=true
$
export UUID_VALIDATION_AUDIT_ONLY=false
$
export UUID_VALIDATION_WINDOW=24h
$
$
./opik.sh

Helm

1
databaseAnalytics:
2
  uuidV7ValidationEnabled: true
3
  uuidV7ValidationAuditOnly: false
4
  uuidV7ValidationWindow: 24h

Audit (shadow) mode

Audit mode is the safe way to introduce validation on a deployment whose clients you do not fully control. Out-of-window ids are counted and logged, but the write still goes through:

1
databaseAnalytics:
2
  uuidV7ValidationEnabled: true
3
  uuidV7ValidationAuditOnly: true

Each detection is logged at INFO with a fixed, searchable prefix:

UUIDv7 audit: would-reject id, embedded timestamp '...' outside window '...', reason '...', resource '...', workspace '...'

Once no new detections are arriving for the clients you care about, set uuidV7ValidationAuditOnly back to false to start enforcing. Query the counter’s rate, not its value, when deciding that — see below.

Monitoring

Both modes increment the opik.ingestion.uuid_v7.rejected counter, so one query covers the whole rollout:

opik.ingestion.uuid_v7.rejected is a cumulative, monotonic counter: it only ever increases, so its value never returns to zero once a single id has failed. Gate the rollout on an interval delta or rate — increase(...[1h]) or rate(...[5m]) in PromQL, with the labels you care about — rather than on the raw total.

Watch it for a full traffic cycle before switching from audit to reject, not just the first hours: a client that only runs nightly will not show up in a one-hour sample.

Turning it back off

If legitimate traffic starts being rejected, you have three options, in order of preference:

1. Drop to audit mode — keeps the signal, stops the 400s:

   1
   databaseAnalytics:
   2
     uuidV7ValidationAuditOnly: true
2. Widen the window — if the rejected ids are legitimate but just outside 24h. The accepted range is 12h–45d:

   1
   databaseAnalytics:
   2
     uuidV7ValidationWindow: 48h
3. Turn the check off entirely:

   1
   databaseAnalytics:
   2
     uuidV7ValidationEnabled: false

All three take effect once the backend has restarted. Nothing is migrated and no data is rewritten, so any of them is safe to do at any time — writes that were rejected while the check was on are not recovered, though, so the client has to send them again.

Note that none of these switch off the version check: a non-UUIDv7 id is rejected regardless.

Next steps

• Self-host troubleshooting — other ingestion and startup failures
• Self-hosting overview — how the backend picks up configuration