GEAK pipeline#

2026-07-17

3 min read time

Applies to Linux

The e2e_workflow takes a running LLM server and a workload (ISL/OSL/concurrency) and raises its serving throughput. Control flow is deterministic; each phase is owned by a specialized agent, and every accepted change is gated on a measured end-to-end delta before it is kept. The single-kernel kernel_workflow runs the same closed loop on one kernel and is called recursively when a kernel needs to be authored or tuned.

Inputs#

  • Model + serving backend: a model on disk served by sglang or vllm (selected per run).

  • Workload: input/output sequence length and concurrency (ISL/OSL/conc). Profiling and benchmarking use the same workload so numbers are comparable.

Phases#

The following phases make up an end-to-end run.

Setup#

The Director builds an isolated evaluation directory, launches a warm server, and records the **true baseline throughput and its noise band. Model weights and installed packages stay read-only.

Profile#

The Profiler traces the warm server under the real workload and produces one canonical, Amdahl-ranked hot-kernel list, separating the prefill and decode regimes.

Strategize#

The System Architect routes each hot kernel to a track by Amdahl leverage (pct_gpu_time × achievable_speedup): config, head kernel, editable kernel, or host overhead.

ConfigSweep#

The Config Tuner tries service-level switches one at a time — attention backend, cuda/HIP graph, scheduling and memory knobs, backend toggles, and (when enabled) lower precision — keeping only changes that measurably help. This is the cheapest, landscape-reshaping lever, so it runs first; the server is re-profiled afterward.

HeadKernel#

For the heaviest GEMM/attention kernels: the Kernel Extractor builds a standalone, immutable unit test from the real workload; the Op Benchmarker bakes off backends and tunes them; and the recursive kernel_workflow authors/optimizes a kernel against that test. The e2e Integrator overlays the winner back and gates it end-to-end.

Milestone#

The remaining editable kernels above a threshold are swept the same way — extract, optimize using the kernel layer, integrate, re-profile — until the budget or the Amdahl stop rule ends the loop.

Finalize, Report, Validate#

The Integrator assembles the deliverable (reversible overlay + patch + final_launch.sh); the Architect writes final_report.md; and the Director independently re-measures the combined result against the true baseline and arbitrates the official number. final_launch.sh is a self-contained script that launches the server with all accepted overlays and configuration changes applied — run it in place of your original launch command to use the optimized configuration.

Integration and gating#

Each accepted kernel is folded back into the live server through a reversible overlay (never editing installed packages). A change is accepted only when it actually runs live (engagement proof), clears the noise band under a tight back-to-back A/B, and preserves output quality (greedy parity, or a task accuracy gate for reduced-precision kernels).

Outputs#

Everything lands under exp/e2e_<model>_<timestamp>/: final_report.md, architect_report.md, the final/ bundle (overlay, final_patch.diff, final_launch.sh), and per-stage artifacts.