SQLStreams

the messaging platform that is just Postgres

You last visited on 9999-99-99 Show what's new since then
Posted: 2026-09-12 · Report this thread
brandon Site Admin brandon profile Posts: 677

client.Manager().Run(ctx) claims the system’s manager row and reconciles every worker row in the deployment until ctx cancels, then returns nil. Most deployments never call it: Consume and SchedulerInstance.Schedule run the same manager beside their session, so a process with one live consumer already keeps the fleet-wide workers running, the stream janitor, the schedule producer, the metrics collector, and the built-in checks, for every stream in the deployment. Run is for the process with no consumer to hang upkeep on: sqlstreams manager run, or a service that carries it explicitly.

return client.Manager().Run(ctx)

Verbs

verbreturnsnotes
Run(ctx)errorblocks; safe to run N-way; nil when ctx cancels

The dial

Nothing is refused. Every Consume, Run, and Schedule call in the fleet starts its own reconcile loop, and they all try to claim one row: manager under the system’s owner, declared target_instances = 1. One claim wins, the losers retry every 30 seconds and take over within that window when the holder stops. That column takes effect with no deploy:

target_instanceswhat runs
0nothing, everywhere; the loop logs SQL0035 and reconciles no rows
1one process reconciles, the rest wait. The default
NN processes reconcile
-1every process reconciles, no cap

Gotchas

  • ClientConfig.DisableManager is the per-process opt-out for consumer pods beside a dedicated sqlstreams manager run. A consumer keeps running its own group’s consumers, its cursor advancer, and its stream’s janitor and vacuum either way; the opt-out only stops it from carrying the deployment-wide rows. The stream janitor creates and drops partitions, so a consumer’s database role needs DDL rights on its stream’s tables.
  • Admin verbs run nothing and the producer runs nothing. A deployment that only produces accumulates partitions; ProducerHandle.Register logs SQL0063 naming the unclaimed rows, and the one-minute worker_liveness check keeps an alert active on __system.alerts until the rows are claimed again.
  • The stream janitor gives each of its five cleanup steps its own five-second deadline, retries included. A step that times out returns its error and the remaining steps still run; a pass with any error counts as a failed tick and backs off. The start log reports cleanup_timeout.