Skip to content

Catalog

Governance tag: overwatch · Min version: overwatch-serve 0.0.1 · Anchor: #catalog

The catalog is the inventory of a tenant's own streams: what each stream was declared to produce, reconciled against what actually landed, with a headline freshness signal. It is the only dimension that catches a stream producing zero rows — completeness scoring is blind to a stream with no buckets to check.

Two layers reconciled

Layer Answers Keyed on
Declared What the stream is meant to produce (exchange, market_type, symbol, datatype)
Observed What actually landed (exchange, market_type, symbol, datatype)
Reconciliation Declared LEFT JOIN observed → presence same tuple

The reconciliation is declared-driven: it shows exactly the streams a tenant declared, not whatever else happened to land. A stream declared but never observed still appears — flagged absent — which is how a dead producer is caught.

Declared row

Field Type Meaning
exchange · market_type · symbol · datatype string The stream identity
expected_cadence_ms u32 Grid quantum for a state stream; 0 = event-based
kind string state or event
is_active u8 1 = live in the reconciliation
binding_id string The deploy that owns the stream
deleted u8 Tombstone: 1 = producer released
updated_at u64 Unix ms; latest declaration wins

Latest declaration per stream wins (ReplacingMergeTree on updated_at); a released producer tombstones its row (deleted = 1) and drops out of the view.

Observed row

Field Type Meaning
first_seen_ms · last_seen_ms u64 Earliest / latest event timestamp
row_count u64 Rows present for the stream
measured_cadence_ms u32 Median inter-row delta (state streams)
observed_at u64 Unix ms of this refresh

Presence signal

Reconciliation derives one headline presence per stream:

presence Condition
absent row_count = 0 or no observed row
stale observed_at - last_seen_ms > 60000 (over 60s since last event)
present fresh and non-empty 
flowchart TD
  D[Declared stream] --> J{Observed?}
  J -- no rows --> AB[absent]
  J -- rows, last event > 60s ago --> ST[stale]
  J -- rows, recent --> PR[present]
Each declared stream resolves to one presence value — absent when no rows landed, stale when the last event is over sixty seconds old, otherwise present.

Read it

The Governance API returns one reconciled row per declared stream, filterable by any of exchange, market_type, symbol, datatype, presence.