feat: compute deterministic timeseries_id column at ingest

[g-talbot]

Summary

• Adds a timeseries_id column (Int64) to the metrics Arrow batch, computed as a deterministic SipHash-2-4 of the series identity columns
• Hash includes metric_name, metric_type, and all tags — excludes temporal columns (timestamp_secs, start_timestamp_secs, timestamp) and value columns (value, plus DDSketch components from [metrics] Support DDSketch in the parquet pipeline #6257: count, sum, min, max, flags, keys, counts)
• Column is already declared in the metrics default sort schema (metric_name|service|env|datacenter|region|host|timeseries_id|timestamp_secs/V2), so the writer automatically sorts by it and places it in the correct physical position
• Adds TimeseriesId variant to ParquetField enum and updates SORT_ORDER

Design reference

Sorted Series Column for QW Parquet Pipeline — this PR implements the Timeseries ID component; the full Sorted Series composite key is a follow-up.

Test plan

• 8 unit tests for compute_timeseries_id (determinism, exclusions, order independence, key/value non-interchangeability)
• All 200 existing tests in quickwit-parquet-engine and quickwit-opentelemetry pass with updated column counts
• Clippy clean (pre-existing warning in reorder_columns not introduced by this PR)

🤖 Generated with Claude Code

[alanfgates]

One question on metadata changes, other than that lgtm.

[alanfgates]

Adding FOR UPDATE means your locking these rows. Is that what you want? I'm not clear why this change requires the addition of this lock.

[alanfgates]

FWIW, Claude likes this addition, saying:

The addition of FOR UPDATE in the CTE subquery correctly prevents a TOCTOU race where concurrent mark_metrics_splits_for_deletion could change split state between the CTE read and the DELETE. This matches the established pattern in the non-metrics delete_splits at line 1076-1083 of the same file. Good fix.

[alanfgates]

Claude comments:

The stability contract relies on Rust's Hash trait implementation for str (writes bytes ++ [0xFF]) and u8 (writes [byte]). While this has been stable since Rust 1.0 and the pinned tests would catch a regression, this is not a guaranteed API contract — it's an implementation detail of the standard library. A Rust toolchain update could theoretically change it.
Recommendation: Consider hashing raw bytes directly (e.g., hasher.write(metric_name.as_bytes()); hasher.write(&[0xFF]);) rather than going through the Hash trait. This would make the stability explicit in the code rather than relying on an undocumented trait implementation. The pinned tests are a good safety net but catch the problem only after a build — explicit byte feeding prevents it.

It's not clear to me if the added safety here is worth the effort, especially considering a change to the hash implementation for str would probably break a lot of things.

[alanfgates]

Comment from Claude:

compute_timeseries_id is called per-row and sorts the tags each time. For batches where most rows share the same tag key set, this is redundant work. For typical batch sizes this is unlikely to be a bottleneck, but worth being aware of for large batches.

This seems valid, especially since we're storing one value per row, as compaction proceeds we'll be getting identical tag sets next to each other. Maybe it's too early to optimize this, but leaving a TODO noting we could do it later seems good.