Architecture & compilation pipeline guide#
This guide covers the FlyDSL project structure, compilation stages, key abstractions, and environment configuration.
Quick Reference#
Component |
Description |
Key File |
|---|---|---|
FlyDSL |
Python DSL front-end for authoring GPU kernels |
|
FlyDSL Compiler |
Trace-based JIT compilation, specialization, and C object export |
|
FlyDSL Expr |
DSL expression ops (arith, vector, gpu, buffer, rocdl) |
|
Fly Dialect |
Flexible Layout IR — MLIR dialect with layout algebra |
|
MlirCompiler |
End-to-end MLIR pass pipeline (DSL → binary) |
|
CompiledArtifact |
Serializable compiled MLIR plus lazy ExecutionEngine state |
|
CompiledFunction |
Callable specialization with |
|
1. Project structure#
FlyDSL/
├── include/flydsl/ # C++ dialect headers
│ ├── Core/ # Layout algebra library (Trait/EmitOp categories)
│ └── Dialect/
│ ├── Fly/ # Fly layout dialect
│ │ ├── IR/
│ │ │ ├── FlyDialect.td # Dialect declaration (name = "fly")
│ │ │ ├── FlyOps.td # Layout ops (make_shape, crd2idx, composition, ...)
│ │ │ ├── FlyTypeDefs.td # Custom types (!fly.int_tuple, !fly.layout, ...)
│ │ │ ├── FlyAttrDefs.td # Attributes
│ │ │ └── FlyInterfaces.td # Op interfaces
│ │ └── Transforms/
│ │ └── Passes.td # Pass declarations (fly-layout-lowering, etc.)
│ └── FlyROCDL/ # FlyROCDL dialect (copy/MMA atoms)
│ └── IR/
│ ├── Dialect.td # FlyROCDL dialect declaration
│ ├── CopyAtom.td # Copy atom ops
│ └── MmaAtom.td # MMA atom ops
│
├── lib/ # C++ dialect implementation
│ ├── Dialect/Fly/ # Fly dialect ops, type inference, lowering
│ ├── Dialect/FlyROCDL/ # FlyROCDL dialect implementation
│ ├── Conversion/ # Dialect conversion passes
│ └── Transforms/ # Optimization passes
│
├── python/flydsl/ # Python DSL package
│ ├── __init__.py # Package version
│ ├── compiler/
│ │ ├── __init__.py # Public API: jit, kernel, compile, argument adapters
│ │ ├── _aot.py # Internal C object/header export implementation
│ │ ├── jit_function.py # @jit decorator, MlirCompiler, JitCacheManager
│ │ ├── kernel_function.py # @kernel decorator, KernelFunction, KernelLauncher
│ │ ├── jit_executor.py # CompiledArtifact and ExecutionEngine wrapper
│ │ ├── jit_argument.py # Argument conversion (Tensor, Stream, Int32)
│ │ ├── ast_rewriter.py # AST rewriting for Python control flow → MLIR
│ │ └── protocol.py # DslType / JitArgument protocols
│ ├── expr/
│ │ ├── __init__.py # Public expr API
│ │ ├── typing.py # Types (T.f32, Tensor, Stream, Constexpr)
│ │ ├── numeric.py # DSL numeric types (Float32, Int32, ...)
│ │ ├── primitive.py # Primitive operations (layout algebra, copy, gemm)
│ │ ├── derived.py # Thread-level tiled copy/MMA views and helpers
│ │ ├── arith.py # Arithmetic dialect ops
│ │ ├── gpu.py # GPU dialect ops (thread_idx, block_idx, barrier)
│ │ └── rocdl/ # ROCm-specific intrinsics (MFMA/WMMA, buffer, TDM, cluster)
│ ├── runtime/
│ │ └── device.py # get_rocm_arch() — GPU architecture detection
│ └── utils/
│ ├── env.py # EnvManager — typed environment config
│ └── logger.py # Logging utilities
│
├── examples/ # Runnable examples
│ ├── 01-vectorAdd.py # Vector addition with layout algebra
│ ├── 02-tiledCopy.py # Tiled copy with partitioned tensors
│ ├── 03-tiledMma.py # Tiled MMA (GEMM) with MFMA atoms
│ └── 04-preshuffle_gemm.py # Preshuffle GEMM end-to-end example
│
├── kernels/ # Production GPU kernels (organized by domain)
│ ├── gemm/ # Dense GEMM kernels
│ │ ├── preshuffle_gemm.py # GEMM (preshuffle layout)
│ │ ├── mxfp4_preshuffle.py # MXFP4 preshuffle GEMM
│ │ ├── fp4_gemm_4wave.py # FP4 4-wave GEMM
│ │ ├── fp8_gemm_4wave.py # FP8 4-wave GEMM
│ │ ├── fp8_gemm_8wave.py # FP8 8-wave GEMM
│ │ ├── rdna_f16_gemm.py # RDNA FP16 GEMM
│ │ ├── rdna_fp8_preshuffle_gemm.py # RDNA FP8 GEMM
│ │ ├── gemm_common_gfx1250.py # GFX1250 GEMM common
│ │ ├── gemm_bf16_gfx1250.py # GFX1250 BF16/FP16 GEMM
│ │ ├── gemm_a8w8_gfx1250.py # GFX1250 FP8 GEMM (per-token/channel, blockscale)
│ │ ├── gemm_a4w4_gfx1250.py # GFX1250 general MXFP4 GEMM
│ │ ├── gemm_a8w4_gfx1250.py # GFX1250 FP8 x MXFP4 GEMM
│ │ └── fp8_gemm_utils.py # FP8 GEMM helpers
│ ├── norm/ # Normalization kernels
│ │ ├── layernorm_kernel.py # LayerNorm (layout API)
│ │ ├── rmsnorm_kernel.py # RMSNorm (layout API)
│ │ └── softmax_kernel.py # Softmax (layout API)
│ ├── attention/ # Attention kernels
│ │ ├── pa_decode_fp8.py # Paged attention decode (FP8)
│ │ ├── pa_decode_swa.py # Paged attention sliding-window
│ │ ├── flash_attn_generic.py # FlashAttention generic fallback
│ │ ├── flash_attn_gfx950.py # FlashAttention gfx950 fast path
│ │ ├── mla_fwd_decode.py # MLA forward decode
│ │ └── fused_rope_cache_kernel.py # Fused RoPE + KV cache
│ ├── moe/ # Mixture-of-experts kernels
│ │ ├── moe_gemm_2stage/ # MoE GEMM (2-stage gate/up + reduce)
│ │ ├── mxfp_moe/ # Fused a4w4/a8w4 MoE 2-stage (device fp4 re-quant)
│ │ ├── moe_sorting_kernel.py # MoE token sorting
│ │ └── topk_gating_softmax_kernel.py # Top-k gating softmax
│ ├── common/mma/ # Shared MMA pipeline helpers
│ │ └── mfma_preshuffle_pipeline.py # Preshuffle layout and XCD remapping
│ ├── conv/ # Convolution kernels
│ │ ├── conv3d_implicit.py # Implicit-GEMM 3D convolution
│ │ └── conv3d_implicit_fp8.py # FP8 implicit-GEMM 3D convolution
│ ├── comm/ # Multi-GPU communication
│ │ └── custom_all_reduce.py # Multi-GPU all-reduce
│ └── common/ # Cross-domain kernel utilities
│ ├── kernels_common.py # Common kernel utilities
│ ├── layout_utils.py # Layout helpers
│ └── tensor_shim.py # GTensor/STensor abstraction
│
├── tests/
│ ├── core/ # Core layout algebra unit tests (C++, gtest)
│ ├── mlir/ # MLIR-level tests (Conversion, Dialect, FlyKtrace, Transforms)
│ ├── kernels/ # GPU kernel tests + benchmarks
│ ├── python/ # Python integration tests and examples
│ ├── unit/ # Unit tests (streams, async, etc.)
│ ├── conftest.py # Pytest fixtures
│ ├── test_common.py # Shared test utilities
│ └── utils.py # Compilation helpers
│
└── scripts/ # Build and test helpers
├── build.sh # Build FlyDSL (CMake + ninja)
├── build_llvm.sh # Build MLIR from ROCm llvm-project
├── run_tests.sh # Run full test suite (pytest + examples + FileCheck)
├── run_benchmark.sh # Run benchmarks
└── dumpir.sh # Dump intermediate IR
2. Architecture#
The user-facing API lives in python/flydsl/. Kernel authors use @flyc.jit and @flyc.kernel decorators with expression operations from flydsl.expr:
Traces Python functions via AST rewriting and execution
Generates Fly dialect ops and standard MLIR dialects (gpu, arith, scf, memref, vector, rocdl)
Compiles through the
MlirCompilerpass pipeline (Fly → ROCDL → LLVM → HSACO)Caches compiled kernels to disk for fast reuse
Executes via MLIR ExecutionEngine
The Fly dialect (include/flydsl/Dialect/Fly/) provides the MLIR-level layout algebra (composition, product, divide, coordinate mapping). Python DSL operations in flydsl.expr lower to Fly dialect ops during tracing, and are then compiled through the MlirCompiler pipeline.
3. Compilation pipeline#
3.1 High-level flow#
Python Function (@flyc.kernel / @flyc.jit)
│
▼ AST Rewriting
Transformed Python Function
│
▼ Tracing (execution inside MLIR Context)
MLIR Module (fly, gpu, arith, scf, memref, vector dialects)
│
▼ MlirCompiler.compile()
┌────────────────────────────────────────────────────────┐
│ Stage A — pre_binary_fragments (Fly → ROCDL) │
│ fly-canonicalize │
│ fly-rocdl-expand-ops │
│ fly-layout-lowering │
│ fly-int-swizzle-simplify │
│ canonicalize │
│ fly-convert-atom-call-to-ssa-form │
│ fly-promote-regmem-to-vectorssa │
│ convert-fly-to-rocdl │
│ canonicalize │
│ gpu.module(convert-scf-to-cf, cse, │
│ convert-rocdl-fastmath-ops, │
│ convert-gpu-to-rocdl{chipset=gfxNNN ...}, │
│ fly-rocdl-cluster-attr) │
├────────────────────────────────────────────────────────┤
│ Stage B — binary_prep_fragments (→ LLVM) │
│ func.func(convert-scf-to-cf) │
│ convert-vector-to-llvm │
│ gpu-to-llvm{use-bare-pointers-...=true} │
│ reconcile-unrealized-casts │
│ ensure-debug-info-scope-on-llvm-func (optional) │
├────────────────────────────────────────────────────────┤
│ Stage C — binary_fragment │
│ fly-emit-gpu-binary{opts="..."} │
└────────────────────────────────────────────────────────┘
│
▼
CompiledArtifact (ExecutionEngine)
3.2 Pipeline stages in detail#
The pipeline is built by RocmBackend._pipeline_parts() in
python/flydsl/compiler/backends/rocm.py. The orchestrator
_pipeline_fragments_for_mode() in jit_function.py decides whether to run
the pipeline as a single combined pass list (pipeline_fragments()) or split
it for external LLVM codegen (external_binary_pipeline_fragments()). External
mode runs Stages A and B with the bundled MLIR runtime, then invokes the
external LLVM toolchain only for Stage C. That toolchain drives an upstream
mlir-opt which does not know FlyDSL passes, so external mode substitutes
gpu-module-to-binary{format=isa} (codegen only). FlyDSL then finishes
assemble+link with the same in-process LLD path as fly-emit-gpu-binary, so
external mode does not need <toolkit>/llvm/bin/ld.lld. Builds without
in-process LLD fall back to format=fatbin and still require ld.lld.
Stage A — pre_binary_fragments (Fly dialect → ROCDL lowering)
The tables explain the stages conceptually. The following excerpt is included directly from the backend at documentation build time, so it is the exact pass definition for this checkout:
pre_binary_fragments = [
"fly-canonicalize",
"fly-rocdl-expand-ops",
"fly-layout-lowering",
"fly-int-swizzle-simplify",
"canonicalize",
*resource_check_fragments,
"fly-convert-atom-call-to-ssa-form",
"fly-promote-regmem-to-vectorssa",
*ktrace_fragments,
"convert-fly-to-rocdl",
"canonicalize",
f"gpu.module(convert-scf-to-cf,cse,"
f"convert-rocdl-fastmath-ops,"
f"convert-gpu-to-rocdl{{chipset={chipset} index-bitwidth=0 runtime=HIP use-bare-ptr-memref-call-conv=true}},"
f"fly-rocdl-cluster-attr)",
]
binary_prep_fragments = [
# gpu.module bodies already went through convert-scf-to-cf above; only host functions remain.
"func.func(convert-scf-to-cf)",
# gpu-to-llvm applies every loaded dialect's ConvertToLLVM patterns.
"convert-vector-to-llvm",
"gpu-to-llvm{use-bare-pointers-for-host=true use-bare-pointers-for-kernels=true}",
"reconcile-unrealized-casts",
*(
["ensure-debug-info-scope-on-llvm-func{emission-kind=LineTablesOnly}"]
if env.debug.enable_debug_info
else []
),
]
opts = f'opts="{" ".join(bin_cli_opts)}"'
toolkit = rocm_toolkit_path(for_external=external)
if toolkit:
opts = f'toolkit="{toolkit}" {opts}'
if not external:
binary_fragment = f"{FLY_GPU_BINARY_PASS}{{{opts}}}"
elif HAS_INPROCESS_LLD:
# ROCDL's format=bin/fatbin always spawn ld.lld. Stop at isa so the
# external tree only does codegen; fly-emit-gpu-binary then
# assemble+links in process (see run_external_binary_codegen).
binary_fragment = f"gpu-module-to-binary{{format=isa {opts}}}"
else:
binary_fragment = f"gpu-module-to-binary{{format=fatbin {opts}}}"
# |
Pass |
Description |
|---|---|---|
1 |
|
FlyDSL-specific canonicalization (rebuilds every static-typed result as |
2 |
|
Expands target-specific FlyROCDL operations while Core layout profiles are available. |
3 |
|
Lowers layout algebra ( |
4 |
|
Algebraically simplifies the swizzle-shaped arith sequences emitted by |
5 |
|
Standard MLIR canonicalization (constant folding, etc.). |
6 |
|
Converts |
7 |
|
Promotes |
8 |
|
Converts Fly values everywhere, including function arguments, results, calls, kernel launches and SCF/CF values, to the nested packed ABI, omitting static fields. Lowers remaining Fly ops to MLIR upstream + ROCDL dialects (copy atoms → ROCDL buffer load/store operations, or gfx1250 TDM → |
9 |
|
Second canonicalization round after ROCDL lowering. |
10 |
|
Inside the GPU module: SCF→CF, CSE, ROCDL fast-math ops lowering, GPU intrinsics→ROCDL, then |
With the check_resource_limits compile hint, gpu.module(fly-check-resource-limits) also runs after #5 (canonicalize) and rejects a kernel whose statically known allocations hold more of a resource at once than its gpu.module target allows (GPUTargetInfoAttrInterface). Ops report what they allocate and free through ResourceUsageOpInterface; shared fly.make_ptr allocations are never freed, so they add up. Usage only known at run time, such as a launch’s dynamic shared memory, is not counted. An allocation counts whether or not the kernel uses it.
Stage B — binary_prep_fragments (LLVM lowering, host + kernel)
# |
Pass |
Description |
|---|---|---|
11 |
|
Host-side SCF → ControlFlow (GPU module bodies were already converted in #10). |
12 |
|
Vector rewrites (e.g. n-D |
13 |
|
Host launcher → LLVM, plus every loaded dialect’s |
14 |
|
Final cast cleanup. |
When FLYDSL_DEBUG_ENABLE_DEBUG_INFO=1, Stage B appends
ensure-debug-info-scope-on-llvm-func{emission-kind=LineTablesOnly} after
reconcile-unrealized-casts and before Stage C.
Stage C — binary_fragment
# |
Pass |
Description |
|---|---|---|
15 |
|
Invokes the LLVM AMDGPU backend and emits an HSA fatbin. |
Stage C wraps the upstream gpu-module-to-binary pass
(lib/Conversion/FlyToROCDL/FlyEmitGPUBinary.cpp): it runs upstream only as far
as format=isa, then assembles the ISA with mlir::ROCDL::assembleIsa and
links the HSA code object through the LLD ELF driver linked into FlyDSL.
Upstream would instead spawn <toolkit>/llvm/bin/ld.lld, resolved from
ROCM_PATH or a path baked into the LLVM build, which fails on any container
that installs ROCm elsewhere. Linking LLD as a library removes that lookup and
pins the linker to the LLVM revision that produced the ISA. An LLVM built
without the lld project falls back to the upstream behavior.
Device bitcode (ocml/ockl/hip/opencl) is loaded from
<toolkit>/amdgcn/bitcode when the module calls __ocml_* / __ockl_* — for
example fx.erfc, which has no LLVM intrinsic. FlyDSL bundles that bitcode into
the package at build time and points toolkit= at it, so this lookup does not
depend on where the container installs ROCm either.
rocm_toolkit_path() (module-level in rocm.py) resolves it in order:
FLYDSL_COMPILE_ROCM_PATH(ignored with a warning if the path lacks bitcode, or lacksld.lldwhen the build has no in-process LLD)the bitcode bundled with the package (when in-process LLD is available)
ROCM_PATH/ROCM_ROOT/ROCM_HOMEon Windows, a TheRock / HIP SDK root staged by
_rocm_toolkit_root()MLIR’s baked-in ROCm path (
_mlirDialectsFlyROCDL.rocm_path())/opt/rocm
Each install prefix is accepted in either the canonical ROCDL layout
(<prefix>/amdgcn/bitcode + <prefix>/llvm/bin/ld.lld) or TheRock’s nested
layout (<prefix>/lib/llvm/amdgcn/bitcode + <prefix>/lib/llvm/bin/ld.lld).
The last two fallbacks keep the JIT cache fingerprint off none for
source/dev builds that did not bundle bitcode and left ROCM_* unset.
External codegen (FLYDSL_COMPILE_LLVM_DIR) uses the same bitcode-only toolkit
resolution when in-process LLD is available: external mlir-opt emits
format=isa, then run_external_binary_codegen runs fly-emit-gpu-binary to
assemble and link. Without in-process LLD, both paths need a toolkit that
contains llvm/bin/ld.lld.
If none of them contains amdgcn/bitcode/ocml.bc, no toolkit= is passed and
upstream’s own lookup applies. Configure the build with
-DFLYDSL_ROCM_BITCODE_DIR=<dir> to choose which ROCm supplies the bundled
bitcode; CMake otherwise searches ROCM_PATH/ROCM_ROOT/ROCM_HOME and
/opt/rocm.
gpu-kernel-outlining is no longer a pass in the runtime pipeline. Kernel
outlining happens during Python tracing, when @flyc.kernel emits
gpu.func ops directly into a gpu.container_module.
3.3 JIT compilation flow#
At decoration, ASTRewriter.transform rewrites Python control flow for MLIR
tracing. Each ordinary @flyc.jit call then follows this path:
Argument conversion: Bind the signature and resolve JIT argument adapters.
Tracing: Execute the launcher and kernels to create the complete MLIR module.
Preparation: Apply compile hints and the debug-location policy; resolve the exact pass pipeline and LLVM options.
Artifact key: Hash that prepared module text with its configuration and target.
Cache lookup: Check memory, then the disk store (
FLYDSL_RUNTIME_CACHE_DIR), whose key also fingerprints the toolchain. A disk miss is rechecked under a per-artifact compile lock.Compilation on miss:
MlirCompiler.compile()lowers the prepared module using the same resolved configuration, then stores the artifact.Execution: Build the argument adapter for this call, lazily create the artifact’s execution engine, and launch.
There is no compiled-artifact hit before tracing. Explicit CompiledFunction
calls reuse a fixed specialization without tracing. See
compilation and caching
for invalidation boundaries, external inputs, and run-only behavior.
3.4 Precompilation and C export#
flyc.compile(launcher, *specialization_args) uses the same tracing, lowering,
and cache path as a normal @flyc.jit call, then returns a CompiledFunction
that retains the selected CompiledArtifact. Live device arguments preserve
the initial launch; null pointer wrappers and compatible non-device tensors are
compile placeholders, so compilation and export can run without initializing a
GPU runtime when the target architecture is specified explicitly.
CompiledFunction.export_to_c(file_path, file_name, function_prefix="")
rewrites a fresh copy of the already-lowered module for standalone linkage. It:
namespaces and internalizes the module’s definitions;
selects the backend GPU object and attaches the AOT module handler;
emits a packed
int32_t entry(void **args)wrapper and ABI metadata;emits a position-independent host object and matching C header; and
embeds the backend AOT runtime archive into the host object (on Windows, merges the runtime’s LLVM-dialect IR into the module instead). Only backend system libraries remain as link-time dependencies.
This export path reuses the same GPU binary as the compiled specialization; it does not retrace the Python launcher.
flyc.load_module(path) is the inverse: it reads the embedded metadata from an
exported object, links the object into the running process with LLVM ORC
JITLink (lib/CAPI/HostObject/object_loader.cc), loads the GPU module on the
current device (and on another device the first time it is called there), and
rebuilds a CompiledFunction whose argument packing comes from the recorded
ABI slots and launcher signature. The loader provides no platform runtime: it
runs no static constructors and services no thread-local storage, and rejects
objects that need either before linking. The embedded runtime needs neither.
The exported entry passes the launcher a pointer to an int32_t status on its
own stack, and the kernel launches keep the first error of the call there;
once a launch fails, the remaining launches of the call are skipped.
Backend-specific implementation hooks and the _AOTCompiledFunction helper
remain internal APIs.
4. Key abstractions#
4.1 @flyc.jit — Host launcher#
Decorates a Python function as a JIT-compiled host launcher:
import flydsl.compiler as flyc
import flydsl.expr as fx
@flyc.jit
def launch(a: fx.Tensor, b: fx.Tensor, n: fx.Constexpr[int],
stream: fx.Stream = fx.Stream(None)):
my_kernel(a, b, n).launch(grid=(n // 256,), block=(256,), stream=stream)
Key behaviors:
Every ordinary call traces; matching MLIR and compilation configuration reuse a cached binary.
Constexpr[T]parameters become compile-time constants and affect the cache key.Tensorparameters map to memref descriptors via DLPack.Streamparameters pass a CUDA/HIP stream to the GPU runtime.When called inside an existing MLIR context, the function acts as a normal composable function.
4.2 @flyc.kernel — GPU kernel#
Decorates a Python function as a GPU kernel:
@flyc.kernel
def my_kernel(a: fx.Tensor, b: fx.Tensor, n: fx.Constexpr[int]):
tid = fx.thread_idx.x
bid = fx.block_idx.x
# ... kernel body ...
Key behaviors:
You can only call this inside a
@flyc.jitfunction.Calling the kernel returns a
KernelLauncher; you must call.launch()to emit the launch op.Supports
Constexpr[T]for compile-time specialization.Emits a
gpu.funcwithgpu.kernelattribute into thegpu.module.
4.3 KernelLauncher#
Calling a @kernel function returns a KernelLauncher. Use .launch() to configure and emit the GPU launch:
launcher = my_kernel(a, b, 1024)
launcher.launch(
grid=(num_blocks, 1, 1),
block=(256, 1, 1),
smem=shared_mem_bytes,
stream=stream_value,
)
4.4 CompiledArtifact#
Stores the compiled MLIR and lazily creates MLIR’s ExecutionEngine for JIT
execution:
Thread-safe with lazy engine initialization.
Serializable (pickle) for disk caching.
Supports packed calling convention via
ctypes.Provides
.dump(),.ir, and.source_irfor inspecting compiled or original IR..source_iris not pickled, so it is None after a disk-cache hit.
4.5 CompiledFunction#
The public result of flyc.compile(). It keeps the compiled artifact alive,
provides a positional-only fast call path for runtime arguments, and exposes
export_to_c() for producing a linkable host object and generated C header.
Runtime initialization is deferred for placeholder-based compilations until the
callable is invoked.
4.6 DslType / JitArgument protocols#
Extensible type system for mapping Python values to MLIR. The language-level
contracts, including Storable, are in DSL protocols.
# DslType protocol — for values used inside kernel/jit functions
class DslType(Protocol):
@classmethod
def __construct_from_ir_values__(
cls, values: List[ir.Value], exemplar: "DslType | None" = None
) -> "DslType": ...
def __extract_to_ir_values__(self) -> List[ir.Value]: ...
# JitArgument protocol — for values passed at the host boundary
class JitArgument(Protocol):
def __get_ir_types__(self) -> List[ir.Type]: ...
def __c_abi_spec__(self) -> List[Tuple[type, Callable]]: ...
# Optional: def __cache_signature__(self) -> object, read only by cache_signature()
Built-in types: Tensor, Stream, Int32, and Constexpr[T]
To register custom types:
from flydsl.compiler import JitArgumentRegistry
@JitArgumentRegistry.register(MyPythonType, dsl_type=MyDslType)
class MyJitArg:
def __get_ir_types__(self): ...
def __c_abi_spec__(self): ...
def __cache_signature__(self): ... # optional, for user-managed handle caches
__c_abi_spec__() returns ordered (ctype, fill) slots for the packed C
interface; they need not be one-to-one with __get_ir_types__(). Do not
implement __get_c_pointers__ — that hook has been removed.
4.7 ASTRewriter#
Transforms Python control flow to MLIR ops at the AST level:
for i in range(n)→scf.forfor i in range_constexpr(n)→ compile-time unrolled loopif condition→scf.ifconst_expr(value)→ compile-time constant
5. Environment variables#
5.1 Compilation options (FLYDSL_COMPILE_*)#
Variable |
Default |
Description |
|---|---|---|
|
|
Optimization level (0–3) |
|
|
If |
5.2 Debug options (FLYDSL_DEBUG_*)#
Variable |
Default |
Description |
|---|---|---|
|
|
Dump intermediate IR at each pipeline stage. |
|
|
Directory for IR dumps. |
|
|
Dump final AMD ISA assembly. |
|
|
Print AST diff during rewrite. |
|
|
Print origin IR before compilation. |
|
|
Print IR after each MLIR pass. |
|
|
Generate debug info in compiled code. |
|
|
Verify IR module. |
|
|
Logging level (DEBUG, INFO, WARNING, ERROR). |
5.3 Runtime options (FLYDSL_RUNTIME_*)#
Variable |
Default |
Description |
|---|---|---|
|
|
Directory for caching compiled kernels. |
|
|
Enable kernel disk caching (in-memory cache is always active). |
5.4 Architecture detection priority#
get_rocm_arch() in runtime/device.py checks in the following order:
FLYDSL_GPU_ARCHenv varHSA_OVERRIDE_GFX_VERSIONenv var (supports9.4.2→gfx942format)rocm_agent_enumerator -namesystem tool, which follows HSA runtime device visibilityrocminfosystem tool
The result is both the compile target and the arch kernels branch on;
pytest --flydsl-compile-arch sets FLYDSL_GPU_ARCH.
If neither tool identifies a GPU and no explicit override is set, detection
raises RuntimeError instead of assuming gfx942. On hosts without accessible
ROCm hardware or detection tools, set FLYDSL_GPU_ARCH to the
intended target (for example, FLYDSL_GPU_ARCH=gfx950 COMPILE_ONLY=1 for cross-compilation).
Failed detection is cached to avoid repeated slow probes.
Device counting uses the default rocm_agent_enumerator output instead:
-name can list multiple supported ISAs per GPU and overcount devices. The
physical count may ignore HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES; use
HIP’s hipGetDeviceCount when visibility matters. Counting returns zero if the
enumerator is unavailable or reports no GPU agents.
6. Target hardware#
Architecture |
GPU |
LDS per CU |
Notes |
|---|---|---|---|
|
MI300A / MI300X |
64 KB |
CDNA 3, primary development target |
|
MI350 / MI355X |
160 KB |
CDNA 4, larger LDS |
|
Radeon AI PRO R9700 |
64 KB |
RDNA 4 |
|
— |
320 KB |
CDNA 5, wave32, WMMA, TDM ops |
|
MI250X |
64 KB |
CDNA 2 (verified platform) |
7. IR dump workflow#
Enable with FLYDSL_DUMP_IR=1:
FLYDSL_DUMP_IR=1 FLYDSL_DUMP_DIR=./dumps python test_my_kernel.py
This produces numbered dump files (exact pass count tracks RocmBackend._pipeline_parts()):
dumps/my_func_name/
├── 00_origin.mlir
├── 01_fly_canonicalize.mlir
├── 02_fly_rocdl_expand_ops.mlir
├── 03_fly_layout_lowering.mlir
├── 04_fly_int_swizzle_simplify.mlir
├── 05_canonicalize.mlir
├── 06_fly_convert_atom_call_to_ssa_form.mlir
├── 07_fly_promote_regmem_to_vectorssa.mlir
├── 08_convert_fly_to_rocdl.mlir
├── 09_canonicalize.mlir
├── 10_convert_scf_to_cf_cse_convert_gpu_to_rocdl.mlir
│ # also runs fly-rocdl-cluster-attr
├── 11_func_func_convert_scf_to_cf.mlir
├── 12_convert_vector_to_llvm.mlir
├── 13_gpu_to_llvm.mlir
├── 14_reconcile_unrealized_casts.mlir
├── 15_gpu_module_to_binary.mlir
├── 16_llvm_ir.ll
└── 17_final_isa.s # AMD ISA assembly (best-effort)
If FLYDSL_DEBUG_ENABLE_DEBUG_INFO=1, the debug-info pass adds an extra numbered dump before gpu_module_to_binary.
8. Source files#
File |
Description |
|---|---|
|
|
|
Internal C ABI construction and object/header export used by |
|
|
|
|
|
|
|
|
|
|
|
|
|
Layout algebra primitives (make_shape, crd2idx, copy, gemm) |
|
|
|
DSL numeric types (Float32, Int32, …) |
|
|
|
|
|
Fly dialect op definitions |
|
Pass declarations (fly-layout-lowering, etc.) |