Environment variables

Environment variables#

User-configurable environment variables for Hyperloom, grouped by purpose. Runtime parameters such as framework, tensor parallelism, prompt lengths, and phase toggles are configured with CLI flags; internal subprocess handoff envs are intentionally not listed as user configuration.

Variables marked Required must be set (using shell or $REPO_ROOT/.env) or the CLI will exit fast at startup. Variables marked Optional have sensible defaults; the default is shown in the Default column.

Precedence rule (applies everywhere): shell-exported env wins over .env. See Hyperloom authentication and credentials.

Credentials#

These variables configure LLM gateway access and optional backend credentials.

Variable

Required

Default

Description

SAFE_API_KEY

Conditional

—

AMD primus-safe large language model (LLM) gateway key. Format ak-.... Required for the single-gateway setup; split-gateway deployments can instead provide provider-specific keys. Source for GEAK / Claude / Codex / Critic / Robustness credentials downstream (auto-aliased).

OPENAI_BASE_URL

Conditional

—

LLM gateway URL. Required for the single-gateway setup; split-gateway deployments can provide provider-specific base URLs instead. Example: https://<your-gateway-host>/api/v1/llm-proxy/v1.

ANTHROPIC_BASE_URL

No

Derived from OPENAI
_BASE_URL

Claude-side base URL for split-gateway deployments.

ANTHROPIC_AUTH_TOKEN

No

Inherits SAFE_API_KEY

Claude CLI auth token alias; set explicitly only for split-gateway deployments.

GEAK_API_KEY

No

Inherits SAFE_API_KEY

Only set explicitly to override the default inheritance.

GEAK_BASE_URL

No

Inherits OPENAI
_BASE_URL

Only set explicitly to override the default inheritance.

GEAK_CLAUDE_MODEL

No

Inherits CLAUDE_MODEL

GEAKv4 Claude Code workflow model id.

ANTHROPIC_API_KEY

No

Inherits SAFE_API_KEY (using preflight alias fan-out)

Only set explicitly to override.

OPENAI_API_KEY

No

Inherits SAFE_API_KEY (using preflight alias fan-out)

Only set explicitly to override.

LANGFUSE_HOST

No (required
only
when HYPER
LOOM_LA
NGFUSE
_ENABLE=1)

Unset

Base URL of your Langfuse deployment (for example, https://langfuse.<your-domain>). Used by both the live trace push and the offline backfill_langfuse CLI.

LANGFUSE
_PUBLIC_KEY

No (required
only
when HYPER
LOOM_LA
NGFUSE
_ENABLE=1)

Unset

Langfuse project public key (pk-...).

LANGFUSE
_SECRET_KEY

No (required
only
when HYPER
LOOM_LA
NGFUSE
_ENABLE=1)

Unset

Langfuse project secret key (sk-...).

Path environment#

The following variables configure filesystem paths for Hyperloom’s runtime dependencies and session data.

Variable

Required

Default

Description

REPO_ROOT

No (recommended)

$(pwd)

This Hyperloom checkout. Used to locate .env, skills, scripts. Falls back to the current working directory when unset.

INFERENCEX_PATH

Conditional

Auto-cloned by install.sh

Path to the SemiAnalysisAI/InferenceX repo, used by baseline / target analysis. install.sh clones it when unset; only required if that auto-clone fails.

TRACELENS_ROOT

No (installer auto-clones)

${HYPER
LOOM_CA
CHE_DIR:-
$REPO_ROOT
/.cache}/Tr
aceLens@<resolved-sha> (auto-clone of AMD-AGI/TraceLens pinned to a fixed SHA)

src/hyperloom/agents/kernel/scripts/install.sh clones the public repo into the repo-local cache root when unset. Export it to opt into a pre-existing checkout you maintain — that is an explicit operator override and skips both the clone and the SHA pin.

GEAK_CLAUDE_BIN

No (installer auto-resolves)

First of $HOME/.local/bin/claude, /usr/local/bin/claude, $(command -v claude); written to kernel-agent.env.sh

Pins the Claude Code binary the GEAK SDK path uses, so claude_agent_sdk doesn’t fall back to its older bundled CLI. Export to force a specific build.

USER_DATA_PATH

No

/workspace/hyperloom

Session directory root (logs, runs, mirrors, breakdown). Replaces the retired INFERENCE_OPTIMIZER_SESSION_DIR and WORKSPACE_PATH.

INFERENCE_
OPTIMI
ZER_CU
RRENT_S
ESSION_DIR

No (set by CLI)

Set at session boot

Absolute path to the active session directory. Written by the CLI when a session starts and inherited by every benchmark subprocess; session-path resolution prefers it over scanning USER_DATA_PATH. Do not set by hand.

HYPERLOOM_ROOT

No

$HYPER
LOOM_R
UNTIME_
DIR/sou
rce-mirrors

Legacy source-mirror root kept for compatibility. Current open-source dependency checkouts default to the repo-local cache root (${HYPER
LOOM_CA
CHE_DIR:-
$REPO_ROOT
/.cache}), not this path.

HYPERLOOM
_CACHE_
DIR

No

$REPO_ROOT
/.cache

Writable, repo-local base for auto-cloned open-source deps (TraceLens, Magpie, etc.), cloned per revision as <name>@<sha>. Not under $TMPDIR so a reaper cannot wipe it mid-run.

MAGPIE_PATH

No

Resolved from installed Magpie package unless explicitly set

Magpie package root for benchmark wrappers and patch inspection.

INFERENCE_
OPTIMIZER
_MODEL_PATH_ROOTS

No

Built-in model roots such as /models and /shared_nfs

os.pathsep-separated allowlist for absolute model paths restored from state.json during --resume. HuggingFace-style repo IDs remain allowed. Set this when production models live outside the built-in roots.

SESSION_DIR

No (robustness-agent)

Scan known paths

Path containing storage/coordinator.db; the robustness FindingSink writes under {session_
dir}/ag
ents/ro
bustne
ss/fin
dings/
{sess
ion_id}.jsonl.

ROBUSTNESS_SERVER_URL

No (robustness-agent)

Scan known DNS

M1 primary data source; empty disables the primary path and forces local-only probes.

WORKSPACE_PATH (legacy)

No

Unset

Legacy path variable. Still consumed in two narrow spots: the CLI setdefaults it to the repo root for the critic subprocess’s static assets, and TraceLens uses it as a USER_DATA_PATH fallback. Prefer USER_DATA_PATH. See Upgrade Hyperloom version.

INFERENCE_
OPTIMI
ZER_SES
SION_DIR (deprecated)

No

Unset

Retired — replaced by USER_DATA_PATH. No longer read.

Workload configuration#

Set with CLI flags, not env vars. Pre-set ISL / OSL / CONC / PRECISION / TP / EP env vars are ignored and overwritten (GPU_TYPE is a fallback when --gpu-type is omitted).

  • Model / workload shape: --model, --model-class, --framework, --framework-version, --precision, --tp, --ep, --isl, --osl, --conc, --max-model-len, --profile-osl.

  • Goal / budget: --target-gain, --max-hours, --target-summary, --target-tput, --compare-against-gpu.

  • Cluster topology & multi-node backend: --nodes, --gpus-per-node, --cpus-per-node, --mem-per-node, --gpu-type, --mn-backend (rayjob / infera), --mn-image, --server-args (rayjob), --rayjob-extra-env (repeatable; the supported way to inject server-side env such as SGLANG_USE_AITER=0 into RayJob pods — the sandbox’s own SGLANG_* env is not forwarded).

  • PD disaggregation (infera): --pd-mode disaggregated, --pd-prefill-nodes / --pd-prefill-tp / --pd-prefill-ep / --pd-prefill-extra-args, --pd-decode-nodes / --pd-decode-tp / --pd-decode-ep / --pd-decode-extra-args, --pd-transfer-backend, --pd-ib-device.

  • Phase toggles: --enable-roofline / --no-enable-roofline, --enable-conc-sweep / --no-enable-conc-sweep, --conc-sweep-concs, --no-framework-agent, --no-framework-local-explore, --no-kernel, --no-explore.

  • Agent models: --claude-model, --codex-model.

  • Session / resume: --resume, --resume-from, --session-dir, --reset-state.

  • Quantization: --quantize, --quantize-scheme.

Run inference_optimizer optimize --help for the exhaustive flag list.

Kernel-opt backend selection#

The following variables control the kernel optimization backend ladder.

Variable

Default

Description

KERNEL_OPT_BACKEND_ORDER

Unset

Comma-separated override for the kernel-opt backend. Bare-metal defaults to geak (whole-pipeline GEAK). Use forge to opt into the forge backend.

KERNEL_OPT_MAX_PARALLEL

8 (GPU-adaptive cap)

Max parallel kernel-opt attempts per request (per-kernel race fan-out). The runtime caps this by visible GPUs and per-attempt GPU reservation when it can detect them.

INFERENCE_OPTIMIZER
_KERNEL_OPT_MAX_PARTIAL

Unset

Cap on how many PARTIAL kernel-opt verdicts an action can yield before it short-circuits to NEEDS_REVIEW. Useful for keeping budget contained when GEAK is consistently timing out.

Single-node Ray execution#

Variable

Default

Description

INFERENCE_OPTIMIZER_RAY_EXEC

Unset (off)

Opt-in switch for routing single-node serving benchmarks and needs_gpu specialists through Ray actors. Set to 1 / true / yes / on to enable Ray-managed leases. Leave unset, or set 0 / false / no / off, to use the local subprocess path. Multi-node serving remains controlled by the multi-node backend.

Multi-node / prefill-decode (PD)#

Use CLI flags for multi-node topology and prefill-decode configuration:

--nodes, --mn-backend, --mn-image, --gpus-per-node, --cpus-per-node, --mem-per-node, --tp, --ep, --pd-mode, --pd-prefill-nodes, --pd-decode-nodes, --pd-prefill-tp, --pd-decode-tp, --pd-transfer-backend, and --pd-ib-device.

When the optimizer provisions the cluster itself (Primus-SaFE flow), those flags are all you need — no environment variables are required.

External-mode variables (SaFE-less cluster)#

When SaFE is unavailable (SAFE_API_URL / SAFE_API_KEY not both set) and HYPERLOOM_MN_EXT_SERVICE_URL is set, the optimizer skips all provisioning and benchmarks an already-running cluster described by these variables. When both SAFE_API_* are present these are ignored.

Variable

Backend

Required

Description

HYPERLOOM_MN_EXT_SERVICE_URL

both

yes

Benchmark frontend URL (http(s)://…; infera frontend typically :8000). Its presence triggers external mode.

HYPERLOOM_MN_EXT_SSH_KEY

infera

yes

Private SSH key already authorized on the pods (SaFE injects one otherwise).

HYPERLOOM_MN_EXT_PREFILL_IPS / _DECODE_IPS

infera

PD

Prefill / decode pod IPs (comma-separated) for PD-disaggregated runs.

HYPERLOOM_MN_EXT_WORKER_IPS

infera

aggregated

Worker pod IPs (comma-separated) for aggregated (non-PD) runs. At least one of _PREFILL_IPS / _DECODE_IPS / _WORKER_IPS is required.

HYPERLOOM_MN_EXT_SSH_PORT

infera

No (default 2233)

SSH base port; decode role is offset +10.

HYPERLOOM_MN_EXT_SSH_KNOWN_HOSTS

infera

No

known_hosts path; else a relaxed host-key check is used.

HYPERLOOM_MN_EXT_HEAD_IP

rayjob

No (recommended)

Ray head IP (Dashboard :8265, GCS :6379). Enables per-round restarts; omit for benchmark-only.

HYPERLOOM_MN_EXT_RAY_DASHBOARD_TOKEN

rayjob

No

Ray Dashboard auth token, only if the dashboard is authenticated.

Infera external mode requires HYPERLOOM_MN_EXT_SSH_KEY plus at least one *_IPS list, or the run fails fast at startup. RayJob external mode ignores the SSH / IP vars and uses HYPERLOOM_MN_EXT_HEAD_IP for restarts.

Multi-node SSH fanout creates session-scoped keys under the active session directory. Treat mn_id_ed25519 and mn_id_ed25519.pub as sensitive session artifacts: keep the session directory on an access-controlled filesystem and do not publish it unchanged in support bundles.

Quantization prelude#

Variable

Default

Description

HYPERLOOM_QUANTIZE_ENABLED

Unset

Primary switch (1 to enable) for the AMD Quark PTQ quantization prelude driven by --quantize / --quantize-scheme.

QUARK_ROOT

Unset

AMD Quark checkout used by the quantization-agent. Set this explicitly when quantization is enabled.

Framework / source-tree discovery#

The following variables configure framework source discovery and path overrides.

Variable

Default

Description

INFERENCE_
OPTIMIZER_
FRAMEWORK_
SOURCE_ROOTS

Union with /sgl-workspace
/{aiter,sglang
,vllm}

Colon-separated list of source roots used by PolicyGate and flag discovery. Populated automatically by src/hyperloom/inference_optimizer/assets/install.sh’s _probe_framework_source_roots step (using hyperloom.orchestrator.framework.paths.probe_framework_source_roots_for_env).

INFERENCE_
OPTIMIZER
_RESCUE_PATHS

Unset

Colon-separated list of extra directories the harvest step scans for stray result.json files written outside the session dir (InferenceX-native scripts that hardcode --result-dir).

INFERENCE_
OPTIMIZER
_AITER_JIT_DIR

Aiter default

Override the aiter just-in-time (JIT) cache root for cold-cap sizing.

INFERENCE_
OPTIMIZER
_STRICT_PATHS

1 when CLI bootstraps

When 1, missing path env raises instead of falling back to discovery. Set by the CLI at session start; do not override unless debugging.

HYPERLOOM_
SGLANG_PA
TCH_EXACT
_VERSIONS

Unset

Pin the sglang server-patch step to specific upstream versions; advanced compatibility option.

HYPERLOOM_
ENABLE
_PATCH

1

Set to 0 to skip the in-place server patch step (useful when the upstream is already pre-patched).

AITER_REF

Unset

Optional bare-metal AITER install pin. When unset, the installer selects the newest tag compatible with the installed torch/triton stack.

INFERENCE_
OPTIMIZER_
FRAMEWORK_
AUDIT_USE_LLM

auto

Controls the FRAMEWORK phase semantic-audit LLM deep-read. off keeps the hermetic static verdict only; on always runs the evidence-gated LLM refine; auto (default) escalates to the LLM only when the static verdict is unknown or confidence < 0.5. The refine never upgrades to an already_* status the static layer did not already back with evidence.

Security compatibility switches#

These switches keep production-compatible behavior by default while still allowing operators to turn off credential/env persistence in hardened deployments.

Variable

Default

Description

HYPERLOOM_SPECIALIST_INHERIT_SECRET_ENV

Unset (1)

Bash-enabled specialist subprocesses inherit the limited provider credential set by default: ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, OPENAI_API_KEY, ANTHROPIC_CUSTOM_HEADERS, and AWS Bedrock credential/config vars. Set to 0 only when the claude CLI is authenticated through its own config and env credentials must be suppressed. Unrelated secrets such as GitHub and KB tokens remain blocked.

HYPERLOOM_SPECIALIST_ALLOW_MCP_AUTH_HEADERS

Unset (1)

Allows the specialist MCP config file to include auth headers such as Authorization for cortex_kb by default for production compatibility. The generated config is chmod 0600. Set to 0 to skip bearer-auth Cortex KB MCP wiring when no auth header should be persisted.

HL_ALLOW_DANGEROUS_AGENT_PERMISSIONS

Unset (0)

Slurm carrier only. Set to 1 only in dedicated internal containers to re-enable legacy Claude/Codex approval and sandbox bypass flags.

Critic / Robustness / knowledge base (KB)#

The following variables configure the Critic, Robustness, and knowledge base components.

Variable

Default

Description

HYPERLOOM_
LOCAL_KB_ROOT

$USER_DATA_PATH/kb

Filesystem root for the local recipe-snapshot KB store (always the write target). Overridden by --local-kb-root. See Integrate Recipe/Cortex knowledge base in Hyperloom.

GBRAIN_BASE_URL

Unset

Base URL of the remote recipe-snapshot page store — the read side of the recipe KB. When unset, recipe reads are local-only.

GBRAIN_TOKEN

Unset

Bearer token for GBRAIN_BASE_URL.

RECIPE_KB_MIRROR_MODE

external

external (default): a separate ingest process mirrors the local store into the remote recipe KB. inline: best-effort mirror each local write into the remote store in-process (local write stays authoritative).

CORTEX_KB_URL

Unset

Optional Cortex KB URL used only by the Critic agent’s per-proposal assess enrichment (/v2/reasoning/assess) — not the recipe KB. Also set by --cortex-kb-url. No Cortex call is made unless configured.

CRITIC_AGENT_ROOT

Derived from REPO_ROOT

Override location of the critic-agent runtime.

ROBUSTNESS_AGENT_ROOT

Derived from REPO_ROOT

Override location of the robustness-agent runtime.

ROBUSTNESS_LLM_RCA_DISABLED

Unset

Set to 1 to forcibly disable the LLM root cause analysis (RCA) engine even when credentials are present.

Session / observability hand-off#

These are read by src/hyperloom/inference_optimizer/session/manifest.py and the src/hyperloom/inference_optimizer/breakdown/collectors/ package to populate session_breakdown.json for downstream consumers.

Variable

Description

CLAW_SESSION_ID

Hosted SaFE / Claw session id, written to session.claw_session_id in session_breakdown.json. Set by the Primus-Claw sandbox; unset for local runs.

SANDBOX_USER_ID

Hosted SaFE / Claw user id, written to session.sandbox_user_id. Set by Primus-Claw; unset for local runs.

HYPERLOOM_LANGFUSE_ENABLE

Primary switch (default off) for live Langfuse trace push. See details below.

HYPERLOOM_LANGFUSE_ENABLE details:

Primary switch (default off) for live Langfuse trace push.

  • SDK install: when this flag is on, src/hyperloom/inference_optimizer/assets/install.sh auto-installs the optional langfuse SDK on demand and skips it entirely when off — no separate pip install '...[trace]' is required.

  • Live push: when set to 1/true/yes/on and the three LANGFUSE_* credentials are present, every in-process LLM call is mirrored into Langfuse while the run is live. A session-end flush backfills out-of-process children (geak, forge, robustness, specialist) and KEEP/REVERT decision Scores.

  • Local ledger: reports/trace/*.jsonl is always written regardless of this flag. If the SDK is unavailable, live push degrades to a no-op.

  • Correlation: the Langfuse trace ID and session_id grouping are derived from claw_session_id (env CLAW_SESSION_ID), falling back to the internal session ID for standalone runs. Live push and the offline backfill_langfuse CLI collapse onto one trace per Primus-Claw session.

  • Span layout: trace → phase span (PRELUDE/FRAMEWORK_AGENT/EXPLORE/KERNEL_AGENT/SWEEP/…) → agent span (component: orchestration/kernel/specialist/critic/geak/forge/…) → Generation. Each KEEP/REVERT/gain_pct Score attaches to the agent span that produced the decision, with a trace-level fallback when no matching span exists.

  • Receipt: every session records a langfuse section in session_breakdown.json (and reports/trace/langfuse_receipt.json) noting:

    • Whether push was enabled (or the disabled_reason)

    • The redacted connection config (host and key-presence booleans — never the keys themselves)

    • The derived trace_id and session_id

    • How many generations, scores, and spans were sent

    This lets an operator confirm post-hoc whether a run reached Langfuse.

Langfuse and artifact-package — security and known limitations#

  • Sensitive data surface: When live push is on, conversations.jsonl (and Langfuse Generations) carry full prompt/response text. redact_secrets scrubs common token shapes (Bearer, sk-/pk-, GitHub tokens, some KEY=value) but is not a complete data loss prevention (DLP) filter — bare keys without a recognizable prefix (for example, raw AWS AKIA…) can slip through. The artifact packager also copies reports/trace/*.jsonl and, with the loose mode on by default (HYPERLOOM_SESSION_PACKAGE_LOOSE), drops them under /workspace for the Claw sync. If a session might contain customer code or secrets, define an explicit retention + access-control policy for both the Langfuse project and the /workspace package destination, and consider disabling live push or loose packaging for those runs.

  • live push + backfill_langfuse overlap: Both derive the same trace_id from claw_session_id, so running the offline backfill after a live run re-emits the out-of-process children onto the same trace and can duplicate observations. Use one path per session, or treat backfill as a recovery tool only when live push did not run.

  • flush_session is idempotent: A second flush only re-writes the receipt (no re-emit), so a duplicated CLOSE step won’t double-push.

  • Package truncation: The bundle caps at 5000 files / 256 MB. On a very long session the cap can stop the bundle short; the PACKAGE_MANIFEST then sets truncated: true and lists dropped_files, so consumers must not treat a truncated package as complete.

  • Generation duration is ~0: Both live and backfill stamp a single timestamp (end == start), so Langfuse shows no meaningful per-Generation duration — counts/usage are accurate, latency is not captured.

token_usage section (in session_breakdown.json)#

Every breakdown carries a top-level token_usage section: a promoted, discoverable rollup of LLM token spend derived from the per-call ledger (reports/trace/llm_calls.jsonl + ext/*.jsonl). It is purely derived from decision_trace.token_rollup, so it always reconciles with that section. No env var controls it; it is always present (zeroed on pre-trace sessions).

  • session_total: whole-session total across every call, with two convenience figures: total_in_out (prompt + completion only) and grand_total (in + out + all cache-creation + cache-read tokens).

  • by_component: per-agent breakdown (orchestration / kernel / critic / specialist / proposal_scorer / geak / forge / …), each with the same convenience totals.

  • by_phase: per-phase breakdown (PRELUDE / FRAMEWORK_AGENT / EXPLORE / KERNEL_AGENT / SWEEP / CLOSE).

  • attribution: attributed_to_decisions vs unattributed split plus attributed_calls_pct. Only calls that carry a task_id / dyn_id joining to a KEEP/REVERT or dynamic_action decision (for example, specialist subprocess turns) are attributed; orchestration / kernel / critic / proposal_scorer turns are LLM-internal and land in unattributed (this is expected, not a gap in the data).

  • timeline: each action_timeline row annotated with the tokens that join to it on task_id. Rows whose action has no LLM spend show tokens: null (rather than a zero bucket) to make the sparsity explicit.

To get the single “total tokens for this run” number, read token_usage.session_total.grand_total (all-in) or .total_in_out (prompt+completion only).

Variables intentionally not exposed#

These are read by os.environ somewhere in the codebase but are internal-only — do not set them by hand:

  • HYPERLOOM_KERNEL_AGENT_ROOT: internal CLI-only handoff to the kernel subprocess (Python constant _KERNEL_AGENT_ROOT_ENV).

  • Any _INFERENCE_OPTIMIZER_*_INTERNAL_* symbol: internal toggles for the test suite.

If you find one of these in a log message, treat it as diagnostic detail rather than something you should tune.

More info#

Use these resources for related configuration and reference information:

Contents