Compatibility matrix

Compatibility matrix#

This page lists the exact versions Infera 0.1.0 has been validated against. Everything here is what we have tested — combinations not listed are outside the validated set for this release, not necessarily unsupported.

Four ROCm baselines are covered. ROCm 7.2 is the release baseline: all three engines, every scenario. ROCm 10.1, ROCm 10.2 and ROCm 10.3 are early-enablement previews on pre-release vendor bases, each validated over a deliberately narrow scope — see ROCm 10.1 preview, ROCm 10.2 preview and ROCm 10.3 preview for exactly how narrow.

How to read this page

Where a component is pinned to a specific image tag, use that exact artifact. The validated engine images bundle the Infera connector, the sitecustomize hook, and the RDMA transport shims on top of the vendor ROCm base — so the tag is the tested unit, not just the engine version inside it.

Engines#

Infera orchestrates one or more engines. Install at least one. The 0.1.0 release is validated against these engine versions and images.

vLLM

SGLang

ATOM

Engine version

0.25.1

0.5.18 (MI355X) / 0.5.16 (MI325X)

0.1.4

Docker image

vllm/vllm-openai-rocm:v0.25.1 (ROCm base, digest-pinned)

lmsysorg/sglang:v0.5.18-rocm720-mi35x / lmsysorg/sglang:v0.5.16-rocm720-mi30x

rocm/atom:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.10.0_atom0.1.4_20260612

ROCm (image base)

7.2.3

7.2.0

7.2.4

OS (image base)

Ubuntu 22.04.5

Ubuntu 22.04.5

Ubuntu 24.04.4

Python (image base)

3.12.13

3.10.12

3.12.3

PyTorch (image base)

2.11.0

2.9.1

2.10.0

vLLM

SGLang

ATOM

Engine version

0.26.1.dev0 (g568afb3a1)

0.5.15.post1 (ge35a33b34a)

not validated

Docker image

rocm/ufb-private:vllm-0.26.0-…-rocm10.1.0a20260807-568afb3a1

rocm/ufb-private:sglang-v0.5.15.post1-…-rocm10.1.0a20260811-e35a33b34a

—

ROCm (image base)

10.1.0 (nightly a20260807)

10.1.0 (nightly a20260811)

—

OS (image base)

Ubuntu 24.04.4

Ubuntu 24.04.4

—

Python (image base)

3.14.7

3.14.6

—

PyTorch (image base)

2.12.0+rocm10.1.0a20260807

2.12.0+rocm10.1.0a20260811

—

Build date

2026-08-08

2026-08-11

—

These are nightly tags in an access-gated repository, so the tag alone is not a version. Build against the digest:

rocm/ufb-private:vllm-0.26.0-ubuntu24.04-py3.14-nightlies-device-all-cdna-rocm10.1.0a20260807-568afb3a1@sha256:11657b464b9c2b7f285a68ca1f622f2cdf3c5ef30a48cc6e560c4141acca8556
rocm/ufb-private:sglang-v0.5.15.post1-ubuntu24.04-py3.14-nightlies-device-all-rocm10.1.0a20260811-e35a33b34a@sha256:e87ecda2624687727d253e0c74863dd7f5008759ab09a9f7bc14e3360de181b6

vLLM

SGLang

ATOM

Engine version

not validated

0.5.21 (gaaaf7c860d)

not validated

Docker image

—

rocm/ufb-private:sglang-v0.5.21-…-rocm10.2.0rc0-aaaf7c860d

—

ROCm (image base)

—

10.2.0 (pre-release rc0)

—

OS (image base)

—

Ubuntu 24.04.5

—

Python (image base)

—

3.14.10

—

PyTorch (image base)

—

2.14.0+rocm10.2.0rc0

—

Build date

—

2026-10-29

—

This is a pre-release tag in an access-gated repository, so the tag alone is not a version. Build against the digest:

rocm/ufb-private:sglang-v0.5.21-ubuntu24.04-py3.14-prereleases-device-all-rocm10.2.0rc0-aaaf7c860d@sha256:3385b5c137e4fe6d482d508d80ce661387a905b3dc60983a5e8bd5113b5189ea

The Infera engine image for this base is built from deploy/docker/Dockerfile.sglang.rocm102, not Dockerfile.sglang.

vLLM

SGLang

ATOM

Engine version

not validated

0.5.22 (g8dee3f6bdb), swapped onto the 10.3 base

not validated

Docker image

—

rocm/ufb-private:sglang-v0.5.21-…-rocm10.3.0a20261103-aaaf7c860d + sglang 0.5.22

—

ROCm (image base)

—

10.3.0 (alpha nightly a20261103)

—

OS (image base)

—

Ubuntu 24.04.5

—

Python (image base)

—

3.14.10

—

PyTorch (image base)

—

2.14.0+rocm10.3.0a20261103

—

Build date

—

2026-11-03

—

The vendor base ships sglang 0.5.21, whose disaggregated decode is not usable (below), so the engine runs the sglang source of the v0.5.22 production base on it. Both images are pinned by digest in deploy/docker/Dockerfile.sglang-base.rocm103:

rocm/ufb-private:sglang-v0.5.21-ubuntu24.04-py3.14-nightlies-device-all-rocm10.3.0a20261103-aaaf7c860d@sha256:bb27e0f56ba66497a9419f79ebf5d298e6bb690e573797f3b8c4b18bef5350bd
lmsysorg/sglang-rocm:v0.5.22-rocm720-mi35x-20261018@sha256:8f7d8034546b17cc4ae57c63f3f65c584eb73e62378243d1416b8385aa7be286

The Infera engine image is Dockerfile.sglang, unchanged, built on that base.

OS / Python / PyTorch above were read from inside each validated base image (not the tag). See the platform table below for the baseline that applies across all engines.

Engine images bundle more than the engine

The validated engine images layer the Infera connector, the sitecustomize hook, and the Mooncake / ionic RDMA shims on top of the vendor ROCm base. For serving, prefer the prebuilt image over a manual pip install of the engine. See Deployment → Engine images.

Platform#

Component

Validated version

GPU

AMD Instinct MI355X (gfx950)

ROCm

7.2 — plus a 10.1 nightly preview (below), a 10.2 pre-release preview (below) and a 10.3 alpha-nightly preview (below)

Operating system

Ubuntu 24.04 (Linux x86-64)

Python

3.10+

Docker

Required — runs etcd in dev and builds engine images

RDMA NIC

AMD AINIC — required only for cross-node prefill-decode

ROCm 10.1 preview#

ROCm 10.1 changes how ROCm itself ships. It arrives as a pip wheel (_rocm_sdk_devel under site-packages, with ROCM_HOME pointing at it) and the bases carry no /opt/rocm; Python moves to 3.14. The Infera engine images restore /opt/rocm as a symlink, because third-party code that hardcodes the path degrades silently rather than erroring.

Validated with gpt-oss-120b on 2 × 8 MI355X (gfx950), both engines, both serving topologies:

Scenario

vLLM

SGLang

Aggregated — prefill + decode in one worker

✅ PASS · 169 s

✅ PASS · 342 s

Disaggregated — prefill and decode on separate nodes

✅ PASS · 166 s

✅ PASS · 367 s

Both disaggregated runs were confirmed to carry KV over RDMA rather than silently falling back to TCP, and all four runs’ correctness probes passed.

Outside the ROCm 10.1 preview scope

  • Models — gpt-oss-120b only. GLM-5.2 and DeepSeek-v4 are not covered: the 10.1 SGLang base is v0.5.15.post1, while the DSA patch set Infera applies for those models is pinned against v0.5.17.

  • aiter sampling (vLLM) — every 10.1 vLLM run passes --use-fp64-gumbel. aiter’s top-k/top-p kernel does not build against 10.1’s hipCUB, so sampling falls to the PyTorch implementation. This is an upstream aiter issue; MXFP4 MoE still runs through aiter.

  • kvd GPU-direct tier — not built into the 10.1 images.

  • ATOM — not exercised on 10.1.

  • gfx942 — the same bases and scenarios also passed on 8 × MI300X, but that path needs image-side patches and aiter overrides which are not part of this release’s images.

ROCm 10.2 preview#

ROCm 10.2 keeps 10.1’s packaging: ROCm ships as a pip wheel (rocm-sdk-* 10.2.0rc0, _rocm_sdk_devel under site-packages) and Python is 3.14. Unlike the 10.1 bases, the 10.2 SGLang base already carries /opt/rocm as a symlink to _rocm_sdk_devel.

The 10.2 SGLang base is v0.5.21, between the v0.5.18 and v0.5.22 recipes, so the engine image has its own Dockerfile, Dockerfile.sglang.rocm102. It applies the v0.5.22 patch set with these differences:

  • GLM NextN fused shared experts — backported. It is upstream only from v0.5.22, where the DSA patch script just asserts it.

  • Mooncake KV wait event — the mooncake/conn.py consumer half is added. v0.5.21 records the early-send event but never waits on it, so chunked prefill over Mooncake can read half-written KV without it.

  • MLA absorb-BMM config — dropped. This build has no _absorb_bmm_config.

  • Decode radix under speculation — the patch defaults off. The opt-in it enables (SGLANG_EXPERIMENTAL_DECODE_RADIX_SPEC=1) is unavailable on this image.

Validated with GLM-5.2-MXFP4 on 1 × 8 MI355X (gfx950), SGLang only, both serving topologies, using the examples/sglang_mix_glm5.2 and examples/sglang_2p1d_glm5.2 kits:

Scenario

SGLang

Aggregated — TP4 + DP4 attention + EAGLE MTP + HiCache, one worker on 4 GPUs

✅ PASS · 6/6

Disaggregated — 1 × TP1·PP4 prefill + TP4 / DP4-attention decode, on one node

⚠️ smoke 6/6, but degrades after a long request (below)

Each run is the kit’s smoke.sh: worker registration, completion, chat, streaming, and a ~66k-token needle prompt sent cold and again from the prefix cache. The disaggregated run additionally answered two fresh, uncached ~99.5k-token needle prompts correctly, and its prefill and decode both moved KV through Mooncake’s RDMA transport over the ionic rails, with no TCP fallback.

Known defect — disaggregated decode on this sglang build

The smoke run above does not catch it, because its short questions come before its long prompt. On sglang 0.5.21+amd.march.gaaaf7c860d a disaggregated decode is correct on a fresh worker, but after the first long request (~70k tokens) every later response, short prompts included, degrades to runs of “!”, and nothing is logged. Reproduced on the 10.2 and 10.3 bases (same sglang build), on a node other than the one above; the aggregated engine and sglang v0.5.22 answer the same request sequence correctly. Do not serve disaggregated traffic from this image. The ROCm 10.3 preview replaces this sglang build with v0.5.22; the same swap has not been validated on the 10.2 base.

Outside the ROCm 10.2 preview scope

  • Engines — SGLang only. vLLM and ATOM are not exercised on 10.2.

  • Models — GLM-5.2-MXFP4 only. gpt-oss-120b, the 10.1 preview model, is not re-run on 10.2.

  • Cross-node prefill-decode — not covered. Prefill (GPUs 0-3) and decode (GPUs 4-7) ran on the same node, still over Mooncake RDMA.

  • Accuracy and performance — smoke checks only. GSM8K and AgentX were not run on 10.2.

  • First-request warm-up — the first long prompts after a cold start pay a one-time AITER / Triton JIT cost. A ~66k-token prompt took 530 s the first time, while a fresh ~99.5k-token prompt took 4.4 s once warm. Warm up with long prompts before taking traffic.

  • gfx942 — not exercised on 10.2.

ROCm 10.3 preview#

ROCm 10.3 keeps 10.2’s packaging: ROCm ships as a pip wheel (rocm-sdk-* 10.3.0a20261103, _rocm_sdk_devel under site-packages, /opt/rocm symlinked to it) and Python is 3.14. The base is an alpha nightly, not a release candidate; no 10.3 pre-release exists yet.

The vendor 10.3 SGLang base ships the same sglang build as the 10.2 base, 0.5.21+amd.march.gaaaf7c860d, and with it the disaggregated-decode defect described under the 10.2 preview. No 10.3 image ships a newer sglang, so the preview keeps the vendor base’s ROCm, PyTorch and native kernels and runs the sglang v0.5.22 source of the production base on them. That makes the engine image the release recipe, unchanged:

docker build -f deploy/docker/Dockerfile.sglang-base.rocm103 -t sglang-rocm103:v0522 .
docker build -f deploy/docker/Dockerfile.sglang \
    --build-arg SGLANG_BASE_IMAGE=sglang-rocm103:v0522 -t infera-sglang:rocm103 .

The swap is sglang’s pure-Python tree and the transformers version it pins. The native libraries stay the 10.3 base’s (sglang-kernel 0.4.7, aiter acf8fdf93), older than the ones the v0.5.22 base ships (sglang-kernel 0.4.10, aiter 2bb549102).

Validated with GLM-5.2-MXFP4 on 1 × 8 MI355X (gfx950), SGLang only, both serving topologies, using the examples/sglang_mix_glm5.2 and examples/sglang_2p1d_glm5.2 kits:

Scenario

SGLang

Aggregated — TP4 + DP4 attention + EAGLE MTP + HiCache, one worker on 4 GPUs

✅ PASS · 6/6 · 23/24 around a ~70k prompt

Disaggregated — 1 × TP1·PP4 prefill + TP4 / DP4-attention decode, on one node

✅ PASS · 6/6 · 23/24 around a ~70k prompt

Each scenario ran the kit’s smoke.sh (worker registration, completion, chat, streaming, a ~66k-token needle prompt cold and again from the prefix cache) and a history check aimed at the 0.5.21 defect: eight short needle prompts (35 to 3.6k tokens), one fresh ~70k-token needle, then sixteen more short ones. The long needle was answered correctly in both, and 23 of the 24 short ones (the miss is the residual below); sglang 0.5.21 on the same node fails every prompt after the long one. The disaggregated run also answered two fresh, uncached ~99.5k-token needles, and its prefill and decode moved KV through Mooncake’s RDMA transport on all eight ionic rails, with no TCP fallback and no transfer errors.

Outside the ROCm 10.3 preview scope

  • Engines — SGLang only. vLLM and ATOM are not exercised on 10.3.

  • Models — GLM-5.2-MXFP4 only. Other models are not checked against the base’s older native kernels, and multimodal input that needs sglang’s Rust extension is unavailable: the v0.5.22 tree carries it built for Python 3.10.

  • Cross-node prefill-decode — not covered. Prefill (GPUs 0-3) and decode (GPUs 4-7) ran on the same node, still over Mooncake RDMA.

  • Accuracy and performance — smoke and history checks only. GSM8K and AgentX were not run on 10.3.

  • Memory fraction on dma-buf nodes — on the validation nodes (amdgpu 6.16.13, ionic 26.08.17) no peer-memory client is registered, so Mooncake registers VRAM through dma-buf, and HIP counts that registered VRAM twice. With the 2p1d kit’s default fractions the prefill and then the decode run out of memory on their first allocation after KV registration. The disaggregated run used PREFILL_EXTRA_ARGS="--mem-fraction-static 0.65" and DECODE_EXTRA_ARGS="--mem-fraction-static 0.60".

  • Residual garbled starts — one short response in 24 per topology began correctly and then turned to garbage (for example ASPMJKPKEMNDMD for ASPEN-5787). This is not the 0.5.21 defect, which breaks every response after a long one. The decode CUDA-graph code is byte-identical to the ROCm 7.2 release image’s v0.5.22, which shows the same pattern at a few percent when DP ranks replay different decode-graph variants; it is not specific to 10.3.

  • First-request warm-up — as on 10.2, the first long prompts after a cold start pay a one-time AITER / Triton JIT cost.

  • gfx942 — not exercised on 10.3.

Infera#

Package

Validated version

amd-infera

0.1.0
Contents