# Environment variables

Every environment variable Infera reads, grouped by subsystem. These are set on
the **process** (engine, daemon, or server) — most have a CLI-flag equivalent
where noted, and flags win over env vars. Each feature one-pager repeats the
subset relevant to it; this page is the single place that lists them all.

```{tip}
The three that bite people most: `PYTHONHASHSEED=0` (vLLM cross-restart cache
hits), `MC_GID_INDEX=1` (cross-node RDMA), and `--ipc=host` so the engine and kvd
share the L2 arena.
```

## kvd cache — connector (set on the engine process)

| Env | Default | What it does |
|---|---|---|
| `INFERA_KVD_SOCKET` | `/var/run/infera-kvd.sock` | Unix socket the engine uses to reach the kvd daemon. |
| `INFERA_KVD_HIPFILE_ROOTS` | *(none)* | L3 write roots, `retention=path` CSV (e.g. `long=/nvme/kvd-long`). Set this to enable a disk tier; also turns on the GPU-direct chunk-file path. |
| `INFERA_KVD_AIS` | `auto` | GPU-direct (hipFile) read/write: `auto` probes P2PDMA, `1`/`0` force. Use `1` on NFS/AIC, measure on local NVMe. Legacy name: `INFERA_KVD_GPU_DIRECT`. |
| `INFERA_KVD_CHUNK_TOKENS` | `auto` | Tokens per L3 chunk; `auto` targets `INFERA_KVD_CHUNK_TARGET_MIB` (default `128`). |
| `INFERA_KVD_LAYERWISE_LOAD` | `parallel` | L3 load mode: `off` / `stepped` / `prefetch` / `parallel`. |
| `INFERA_KVD_L3_BUDGET_BYTES` | `0` (uncapped) | Hard cap on the **GPU-direct** L3 file volume. These files are connector-owned, so the daemon's `--long-bytes` does **not** bound them — this does. |
| `INFERA_KVD_L3_FREE_FLOOR` | `0.05` | Free-space fraction the connector's L3 reaper keeps; evicts (retention-priority, LRU) when violated. Matters most on a shared filesystem. |
| `INFERA_KVD_L3_REAP_INTERVAL` | `30` (s) | L3 reaper tick period. |
| `INFERA_KVD_SAVE_WORKERS` / `INFERA_KVD_LOAD_WORKERS` | `auto` | L3 I/O worker counts (`auto` = mount `nconnect`; load drops to 1 without P2PDMA). |
| `INFERA_KVD_RETENTION_DEFAULT` | `long` (vLLM) | Default per-request retention when the request carries none. **Required on SGLang** to exercise the long tier (its SPI doesn't propagate per-request retention). |
| `HIPFILE_UNSUPPORTED_FILE_SYSTEMS` | *(unset)* | **libhipfile** env (not Infera): set `1` to allow hipFile DMA reads on NFS. |

Per-request (not env) cache hint goes through `extra_body.kv_transfer_params`:
`infera_retention` — the vLLM connector honors `none|short|long` (default
`long`). See [KV-Cache Offload](../features/kv_cache_offload.md).

```{admonition} Advanced kvd knobs
:class: note
Edge-case tuning (async-load depth, ring buffers, save pool, fsync policy, arena
hugetlb/prefault, per-shard workers) exists as additional `INFERA_KVD_*` vars.
A normal deployment never needs them; run `python -m infera.kvd --help` for the
ones that matter.
```

## PD disaggregation — RDMA transport (set on each engine)

| Env | Typical | What it does |
|---|---|---|
| `MC_GID_INDEX` | `1` | Mooncake **RoCEv2 (routable) GID index** — `1` on our ionic fleet; verify with `show_gids` (often 1 or 3). Index 0 is the link-local/RoCEv1 GID and times out cross-node. |
| `NCCL_IB_GID_INDEX` | `1` | Same idea for the collectives (RCCL) path. |
| `MC_DISABLE_HIP_TRANSPORT` | `1` | Force RDMA instead of the intra-node HIP (hipIpc) transport. Belt-and-braces only: our image already defaults that transport OFF. A hipIpc handle is host-local, so a peer node's `hipIpcOpenMemHandle` fails with `201 - invalid device context` and cross-node PD dies. |
| `MC_ENABLE_HIP_TRANSPORT` | unset | Escape hatch: opt the HIP intra-node P2P transport back **in**. Single-node deployments only — it breaks cross-node PD. `MC_DISABLE_HIP_TRANSPORT=1` overrides it. |
| `MORI_IB_GID_INDEX` | `1` | RoCEv2 GID index for the MoRI transport. |
| `VLLM_HOST_IP` | *(none)* | Routable IP a vLLM worker advertises for KV transfer (cross-node). Pair with `--advertise-host`. |
| `RDMAV_FORK_SAFE` | `1` | libibverbs fork-safety; set for RDMA workers. |

See [PD disaggregation](../features/pd_disaggregation.md) for the bring-up
checklist and the RDMA self-check.

## Routing & transport (set on the server and every worker)

| Env | Default | What it does |
|---|---|---|
| `NATS_SERVER` / `NATS_URL` | *(injected by operator)* | NATS broker address when `--request-transport nats`. |
| `INFERA_NATS_REQ_IDLE_TIMEOUT` | `900` (s) | Max wait for the next reply chunk; expiry → 504 + cancel. A backstop; a stall is reported by the two warnings below. |
| `INFERA_HTTP_REQ_IDLE_TIMEOUT` | `0` (off) | Same, for `--request-transport http`; expiry fails the stream. |
| `INFERA_STREAM_ADMISSION_WARN` | `240` (s) | Log a stream still awaiting its first byte, with worker + request id. Reports only. |
| `INFERA_STREAM_STALL_WARN` | `60` (s) | Log a stream that went silent after producing bytes. Reports only. |
| `INFERA_NATS_REQ_MAX_DURATION` | `0` (off) | Hard wall-clock cap on a whole request. |
| `INFERA_NATS_REQ_MAX_PENDING` | `0` (off) | Per-worker admission limit; backlog at the cap → 429. |
| `INFERA_REQUEST_MAX_RETRIES` | `1` | Retry on an alternate worker before the first token (never mid-stream). `0` = fail fast. |
| `INFERA_K8S_LABEL_SELECTOR` | *(operator default)* | Pod selector for `--discovery-backend kubernetes`. |
| `INFERA_DRAIN_TIMEOUT` | `30` (s) | Graceful-drain window on shutdown. |

Each has a CLI flag (flag > env > default); see [Routing & transport](../features/routing_and_transport.md)
and the [CLI reference](cli.md).

## Engine startup (set on the engine process)

| Env | Default | What it does |
|---|---|---|
| `INFERA_NODEPORT_RANGE` | `30000-32767` | Port window every auto-allocated port must avoid (kv-event publishers, single or a per-DP-rank block, and the ATOM rendezvous port) — Kubernetes' NodePort range, where a Service claim makes the port unreachable from the node IP we advertise. Set `lo-hi` for a cluster that moved the range, `none` to drop the guard. Only `none`/`off` drop it — an empty or malformed value keeps the default. |
| `INFERA_ENGINE_READY_TIMEOUT` | `1800` (s) | Maximum time the SGLang/vLLM wrapper waits for the child engine `/health` endpoint during startup. Increase for slow model storage or long JIT compilation; non-numeric, non-finite or non-positive values fall back to 1800. This is separate from disaggregation bootstrap and runtime watchdog timeouts. |

## Engine correctness

| Env | Value | What it does |
|---|---|---|
| `PYTHONHASHSEED` | `0` | **Mandatory on vLLM** — vLLM v1 salts block hashes per process, so without a fixed seed you get **0 cross-restart cache hits**. SGLang doesn't need it. |
| `VLLM_SERVER_DEV_MODE` | `1` | Mounts dev routes including `POST /reset_prefix_cache` (used to clear L1 + kvd L3 without a restart). |

## Related

- [CLI reference](cli.md) — the flag equivalents.