Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Where This Plugs Into Panoptes

Kind: Wrap-up.

You have built panoptes-control. Before we talk about where it goes next, let us place it — precisely — inside the harness it serves. You did not build a toy distributed system that happens to shuttle eval jobs. You built the control plane for the EXECUTE stage of the Panoptes harness, and every design choice you made was in service of one job in a larger pipeline.

The stage you now own

The harness has stages on either side of you, and they hand work across file contracts, not function calls. Upstream, the generation stage produces a manifest of vignettes — decision-scenario prompts, parameterized and pinned. That manifest, plus a set of models and a number of epochs, is the submission your coordinator accepts. Downstream, the coding and analysis stages consume a log of raw model responses. That log — one JSONL record per response, prompt and raw output and model version and token usage, append-only — is the evidentiary record the rest of Panoptes is built on. It is the dataset of record for the thesis, the benchmark, and the client deliverable alike.

Your coordinator is the engine that turns the first artifact into the second. Submitted as a Run, split into Jobs — one per model × epoch — scheduled across workers, each response logged as it lands, the run marked done when every job has. That is the EXECUTE stage, promoted from a batch loop that waits on the network into a long-lived service that schedules the waiting across machines.

Why a control plane, and not just a loop The naive execute stage is a for loop over vignette × model × epoch that calls a model and appends a record. It works until it doesn't — until a run is thousands of calls long, until one call in the middle fails, until you want a second machine helping. Promoting the loop to a control plane buys three things the loop never had: durability (a submitted run survives a crash), parallelism (jobs fan out across a pool), and accountability (every response and every token is recorded as it happens, not tallied at the end).

The artifacts you built, and how they compose

Five crates, each one thing, the dependency arrow always pointing inward at the core:

  • control-core — the seams. The WorkerHandle trait the scheduler dispatches through, the Run/Job/JobOutcome domain, the id newtypes, the ControlError taxonomy that knows what is worth retrying, and the wire Message enum with its framed codec. Nothing in the workspace that matters is not defined here first.
  • control-eval — the workload. A mockable ModelClient, run_eval that drives a job's prompts through it, and the append-only JSONL contract that writes the evidentiary record. This is the code that actually produces what Panoptes consumes.
  • control-store — the coordination point. sqlx/SQLite, the atomic claim_next_run, and the transactional, idempotent outcome recording. Every fact about a run's progress lives here, which is exactly why it can survive a crash and referee two schedulers.
  • panoptes-control — the coordinator binary. The axum API that accepts submissions, the scheduler that fans jobs out with bounded concurrency and retries only the retryable, the /stats endpoint that accounts for tokens and cost per model, and the pool that fronts remote workers.
  • panoptes-worker — the networked worker binary. Connect, register, run run_eval, heartbeat, reconnect.

Read that list as a pipeline and a spine at once. The coordinator accepts a run and persists it (store); the scheduler claims it and splits it (store + core); each job is dispatched through a WorkerHandle (core); the worker runs the eval (eval) and appends to the log (eval); the outcome is recorded idempotently (store); /stats reads the tally back out (store). The same run_eval runs whether the job executed in-process or across a socket — only the transport differed.

The spine, one more time

Here is the sentence the whole course was built to earn. The scheduler holds Vec<Arc<dyn WorkerHandle>> and calls dispatch. It never learns whether the handle on the other side is a LocalWorker running the eval in an in-process task or a RemoteWorker shipping it over framed TCP to another machine.

That one seam is why the same coordinator binary runs two deployments without a line of scheduler code changing: a laptop's in-process pool for a quick sweep, and a rack of networked workers for a real run. Parts II through VII built the coordinator against LocalWorker alone. Part VIII added RemoteWorker as a new impl WorkerHandle and a pool to hand it to the scheduler — additive, not a rewrite, exactly as the orientation chapter promised. The distributed system did not require you to rebuild the control plane. It required you to plug a new mechanism into a seam you had already installed. Dependency inversion is not a slogan here; it is the load-bearing wall you can now see holding up two roofs.

Where the verified build lives

Everything here has a worked, compiling reference behind it — the same discipline as courses 1 and 2. The full panoptes-control build, all five crates and 38 tests (including the capstone that kills a worker mid-run and proves no eval is lost or double-counted), lives at github.com/tbar4/panoptes_control. When your version diverges from the answer key, that repo is the arbiter.

Turn the page for the honest accounting: what the hand-rolled protocol costs, and the production paths that pick up where this course deliberately stops.