SQLStreams

the messaging platform that is just Postgres

You last visited on 9999-99-99 Show what's new since then
View previous thread::View next thread

0749 — Benchmark runner lives at the .bench root

Edit this page
Posted: 2026-09-11 · Report this thread
brandon Site Admin brandon profile Posts: 677

Superseded by 0755 for command naming; the root layout and evidence preservation continue unchanged.

Context

The reliability lab is the shared running system for benchmark scenarios. Its extra directory puts that system beside older independent experiments.

Decision

  • Promote the lab packages, entry point, scripts, Compose stack, and results from .bench/reliability/ to .bench/.
  • Keep just reliability-lab and just reliability-report as the commands.
  • Preserve recorded evidence bytes and frozen source layouts; update current evidence links. Historical reproduction commands target the frozen source.
  • The evidence retention policy in 0748 continues at .bench/results/published/.
  • Classifying or migrating the other benchmark programs is separate work.

Consequences

New scenarios extend the existing root runner. Imports, Docker build paths, and native source capture follow the shallower directory. Existing workloads and verdicts are unchanged; old result paths in historical records describe where those artifacts lived when recorded.

The team