Environment variables#
2026-10-29
5 min read time
FlashInfer on ROCm reads the variables on this page at import time or when an
operation runs. Set a variable that is read at import before import flashinfer.
In a shell, export it for the process that starts Python:
export FLASHINFER_LOGGING_LEVEL=DEBUG
Build-time variables, including FLASHINFER_ROCM_ARCH_LIST,
PYTORCH_ROCM_ARCH, FLASHINFER_JIT_VERBOSE, FLASHINFER_EXTRA_LDFLAGS,
FLASHINFER_EXTRA_CFLAGS, FLASHINFER_EXTRA_CUDAFLAGS,
FLASHINFER_OWN_HEADERS_NON_SYSTEM, MAX_JOBS, and
FLASHINFER_JIT_DEBUG, are documented in
CONTRIBUTING.md.
Compilation and kernel cache#
These variables control torch.compile, JIT compilation, and the on-disk kernel cache.
Environment variable |
Default |
Description |
|---|---|---|
|
|
Set to |
|
|
Parent directory of the JIT cache, |
|
Unset |
Any non-empty value disables JIT compilation, including the string |
|
Unset |
Any non-empty value skips the version check between FlashInfer and the JIT-cache package. |
|
Unset |
Any non-empty value, including |
|
Unset |
Set to |
AITER#
These variables control the AITER backend and the kernels it selects. For operation coverage, see the kernel support matrix.
Environment variable |
Default |
Description |
|---|---|---|
|
|
Set to |
|
|
Selects the single-prefill implementation inside AITER. |
|
|
Any value other than |
|
|
Set to |
|
|
Applies to |
|
AITER default |
Directory of AITER shared libraries that the FlashInfer C++ shim loads. Overrides the path recorded when FlashInfer was built. |
|
FlashInfer stores |
List of prebuilt AITER variant directories, separated by |
|
Set by |
Directory of AITER assembly code objects. When the variable is unset or empty, single prefill stays on the CK Tile kernel. |
|
Autodetected |
GPU architecture list for AITER JIT compilation. If you leave it unset, a shim build may set it from |
Logging#
FLASHINFER_LOGGING_LEVEL controls the FlashInfer logger. FLASHINFER_LOGLEVEL and FLASHINFER_LOGDEST control a separate API-call log.
Environment variable |
Default |
Description |
|---|---|---|
|
|
Logger verbosity. Typical values are |
|
|
API-call log detail. |
|
|
Destination of the API-call log: |
ROCm installation#
Environment variable |
Default |
Description |
|---|---|---|
|
|
ROCm installation prefix used by |
Runtime helpers#
Two helpers report whether the current process can use AITER and whether the PyTorch build matches the ROCm installation. Importing flashinfer already runs the ROCm compatibility check.
is_aiter_supported returns True when the device is gfx942 (MI300X or MI325X) or gfx950 (MI350X or MI355X) and PyTorch is a ROCm build. It does not import the aiter package. Catch ImportError around import aiter when you also need to confirm that the package is installed.
check_torch_rocm_compatibility warns when the ROCm version reported by PyTorch differs from the ROCm version on the system. It raises RuntimeError when PyTorch has no ROCm build, and ImportError when PyTorch is not installed.
PyTorch addresses AMD GPUs with the device string "cuda".
import torch
from flashinfer.rocm.aiter_utils import is_aiter_supported
from flashinfer.rocm.hip_utils import check_torch_rocm_compatibility
device = torch.device("cuda")
aiter_ready = is_aiter_supported(device)
check_torch_rocm_compatibility()