hipDNN frontend graph C++ API#
2026-03-31
27 min read time
Main Graph class for building and executing deep learning operations.
This header defines the Graph class — hipDNN’s top-level API for describing, compiling, and running DNN operations on AMD GPUs. It is included automatically via #include <hipdnn_frontend.hpp>.
Overview#
The Graph class provides a fluent API for:
Creating tensor descriptors (shape + dtype, no data yet)
Adding operations (convolution forward/dgrad/wgrad, batchnorm forward/backward/inference, layernorm, rmsnorm, pointwise, matmul, scaled dot-product attention, block-scale quantize/dequantize)
Building (compiling) an execution plan for the GPU
Executing the plan with real device pointers
Typical Workflow#
using namespace hipdnn_frontend;
using namespace hipdnn_frontend::graph;
// 1. Create and configure the graph
Graph graph;
graph.set_io_data_type(DataType::HALF)
.set_compute_data_type(DataType::FLOAT)
.set_name("my_conv_graph");
// 2. Create input tensors
auto x = Graph::tensor(TensorAttributes()
.set_dim({N, C, H, W})
.set_stride({C*H*W, H*W, W, 1})
.set_uid(0));
auto w = Graph::tensor(TensorAttributes()
.set_dim({K, C, R, S})
.set_uid(1));
// 3. Add operations
auto y = graph.conv_fprop(x, w, ConvFpropAttributes()
.set_padding({1, 1})
.set_stride({1, 1}));
y->set_output(true).set_uid(2);
// 4. Build and execute
hipdnnHandle_t handle;
hipdnnCreate(&handle);
graph.build(handle);
int64_t workspaceSize;
graph.get_workspace_size(workspaceSize);
void* workspace;
hipMalloc(&workspace, workspaceSize);
std::unordered_map<int64_t, void*> variantPack = {
{0, d_input}, {1, d_weights}, {2, d_output}
};
graph.execute(handle, variantPack, workspace);
-
namespace hipdnn_frontend
-
namespace graph#
-
class Graph : public hipdnn_frontend::graph::INode#
- #include <Graph.hpp>
The main class for building and executing hipDNN computational graphs.
You describe what operations to run (convolution, batchnorm, pointwise, matmul, layernorm, rmsnorm) and the library figures out how to execute them efficiently on AMD GPUs.
Typical workflow:
Step
What you do
hipDNN call
1. Describe tensor shapes
Define dims, strides, dtype
Graph::tensor(attrs)2. Add operations
Wire inputs to outputs
graph.conv_fprop(x, w, ...)3. Compile for GPU
Select engine, build plan
graph.build(handle)4. Execute
Pass device pointers
graph.execute(handle, ptrs, ws)The Graph uses a fluent API — setter methods return
*thisso you can chain calls:graph.set_io_data_type(DataType::HALF) .set_compute_data_type(DataType::FLOAT) .set_name("my_graph");
See also
hipdnn_frontend::graph::TensorAttributes, hipdnn_frontend::graph::ConvFpropAttributes, hipdnn_frontend::graph::BatchnormAttributes, hipdnn_frontend::graph::PointwiseAttributes
Public Functions
-
inline Error validate()#
Validate the graph structure and tensor configurations.
Validates that:
No duplicate tensor UIDs exist
Graph is a valid DAG (no cycles)
Graph is a single connected component
All tensor attributes are set (dims, type, strides)
All operation nodes have valid configurations
- Returns:
Error with ErrorCode::INVALID_VALUE or ErrorCode::ATTRIBUTE_NOT_SET on failure. Call get_message() for the specific failure reason.
-
inline Error checkNoDuplicateTensorIds()#
Verify that no two tensors in the graph share the same UID.
- Returns:
ErrorCode::OK if all UIDs are unique, or ErrorCode::INVALID_VALUE if duplicates exist. Call get_message() for the duplicate UIDs.
-
inline Error checkTensorUidsSet() const#
Check that all tensors in the graph have UIDs assigned.
- Returns:
ErrorCode::OK if all tensors have UIDs, or ErrorCode::ATTRIBUTE_NOT_SET if any are missing. Call get_message() for the affected tensors.
-
inline std::unordered_map<int64_t, std::shared_ptr<TensorAttributes>> getTensorsByUid() const#
Get all tensors in the graph indexed by UID.
Tensors without UIDs are skipped.
- Returns:
Map from tensor UID to tensor attributes
-
inline std::unordered_map<std::string, std::shared_ptr<TensorAttributes>> getTensorsByName() const#
Get all tensors in the graph indexed by name.
Tensors without names are skipped.
- Returns:
Map from tensor name to tensor attributes
-
inline Error topologicallySortGraph()#
Topologically sort the graph nodes.
Reorders internal nodes so that every node appears after its dependencies.
- Returns:
ErrorCode::OK on success, or ErrorCode::INVALID_VALUE if the graph contains a cycle or multiple disconnected components. Call get_message() for the specific failure reason.
-
inline flatbuffers::DetachedBuffer buildFlatbufferOperationGraph()#
Serialize the graph to a FlatBuffer operation graph.
Assigns UIDs to any tensors that do not already have them, then serializes the full graph structure into a FlatBuffer.
- Returns:
DetachedBuffer containing the serialized graph
-
inline Error build_operation_graph(hipdnnHandle_t handle)#
Build the operation graph descriptor.
Creates the backend operation graph descriptor from the frontend graph representation. Typically called internally by build().
- Parameters:
handle – The hipDNN handle
- Returns:
ErrorCode::OK on success, or ErrorCode::HIPDNN_BACKEND_ERROR / ErrorCode::INVALID_VALUE on failure. Call get_message() for the specific failure reason.
-
inline Error get_knobs_for_engine(int64_t engineId, std::vector<Knob> &knobs) const#
Get available configuration knobs for a specific engine.
See also
hipdnn_frontend::Knob, hipdnn_frontend::KnobSetting
- Parameters:
engineId – The engine ID to query
knobs – Output vector of available Knob objects
- Returns:
ErrorCode::OK on success, or ErrorCode::HIPDNN_BACKEND_ERROR if the graph has not been built. Call get_message() for the specific failure reason.
-
inline Error get_knob_lookup_for_engine(int64_t engineId, std::unordered_map<KnobType_t, Knob> &knobs) const#
Get knobs for a specific engine, indexed by knob type.
Convenience wrapper around get_knobs_for_engine() that populates a map keyed by KnobType_t for direct lookup.
See also
get_knobs_for_engine(), hipdnn_frontend::Knob
- Parameters:
engineId – The engine ID to query
knobs – Output map populated with available knobs, keyed by type
- Returns:
ErrorCode::OK on success, or ErrorCode::HIPDNN_BACKEND_ERROR if the graph has not been built. Call get_message() for the specific failure reason.
-
inline Error get_ranked_engine_ids(std::vector<int64_t> &rankedEngineIds, const std::vector<HeuristicMode> &modes = {HeuristicMode::FALLBACK})#
Get a ranked list of engine IDs based on heuristics.
- Parameters:
rankedEngineIds – Output vector of engine IDs, ranked by expected performance
modes – Heuristic modes to use for ranking
- Returns:
ErrorCode::OK on success, or ErrorCode::HIPDNN_BACKEND_ERROR on failure. Call get_message() for the specific failure reason.
-
inline Error create_execution_plans(const std::vector<HeuristicMode> &modes = {HeuristicMode::FALLBACK})#
Create execution plans using heuristics.
Queries the backend for available engines and selects based on the specified heuristic modes.
- Parameters:
modes – Heuristic modes to use for engine selection
- Returns:
ErrorCode::OK on success, or ErrorCode::HIPDNN_BACKEND_ERROR if the graph has not been built. Call get_message() for the specific failure reason.
-
inline Error create_execution_plan_ext(int64_t engineId, const std::vector<KnobSetting> &settings)#
Create an execution plan with specific engine and knob settings.
Creates an execution plan for a specific engine, configured via knob settings. Settings for deprecated knobs or knobs that are not supported by the engine are skipped and a log message is added describing this.
See also
hipdnn_frontend::Knob, hipdnn_frontend::KnobSetting, get_knobs_for_engine()
- Parameters:
engineId – The engine ID to use
settings – Knob settings to apply to the engine
- Returns:
ErrorCode::OK on success, or ErrorCode::HIPDNN_BACKEND_ERROR if the graph has not been built. Call get_message() for the specific failure reason.
-
inline Error check_support()#
Verify that the execution plan is valid and supported.
- Returns:
ErrorCode::OK if valid, or ErrorCode::HIPDNN_BACKEND_ERROR if the execution plan has not been created.
-
inline Error is_supported_ext(hipdnnHandle_t handle, const std::vector<HeuristicMode> &modes = {HeuristicMode::FALLBACK})#
Check if the graph is supported by any available engine plugin.
Performs a lightweight check to determine if any engine plugin can handle this graph. If the graph has not yet been validated and built, those steps are performed automatically. The graph’s internal state (operation graph descriptor) is preserved for subsequent operations.
- Parameters:
handle – The hipDNN handle
modes – Heuristic modes for engine ranking
- Returns:
Error with OK if supported, HIPDNN_BACKEND_ERROR if not
-
inline Error fromBackendDescriptor(hipdnnBackendDescriptor_t graphDesc)#
Reconstruct the Graph from a finalized backend OperationGraph descriptor.
Extracts operations and graph-level data types from a backend descriptor and rebuilds the frontend Graph representation. Tensors are shared across operations via UID-based lookup.
Currently supports: ConvolutionFprop operations (phased rollout — additional operation types will be added incrementally).
- Parameters:
graphDesc – A finalized backend OperationGraph descriptor
- Returns:
ErrorCode::OK on success, or ErrorCode::INVALID_VALUE / ErrorCode::HIPDNN_BACKEND_ERROR on failure. Call get_message() for the specific failure reason.
-
inline Error deserialize_via_backend(hipdnnHandle_t handle, const std::vector<uint8_t> &data)#
Deserialize the graph from binary via the backend descriptor path.
Creates a backend graph descriptor from serialized bytes and rebuilds the frontend Graph. If a handle is provided, the descriptor is finalized for full backend support.
Currently supports ConvolutionFprop operations (phased rollout — additional operation types will be added incrementally). Graphs containing unsupported operation types will fail.
- Parameters:
handle – The hipDNN handle (can be nullptr)
data – The serialized graph bytes
- Returns:
ErrorCode::OK on success, or ErrorCode::INVALID_VALUE / ErrorCode::HIPDNN_BACKEND_ERROR on failure. Call get_message() for the specific failure reason.
-
inline Error build_plans()#
Finalize the execution plan.
Called internally by build() after create_execution_plans().
- Returns:
ErrorCode::OK on success, or ErrorCode::HIPDNN_BACKEND_ERROR on failure. Call get_message() for the specific failure reason.
-
inline Error build(hipdnnHandle_t handle, const std::vector<HeuristicMode> &modes = {HeuristicMode::FALLBACK}, [[maybe_unused]] BuildPlanPolicy policy = BuildPlanPolicy::HEURISTICS_CHOICE, [[maybe_unused]] bool do_multithreaded_builds = false)#
Build the complete graph and create execution plans.
This is the main method to prepare a graph for execution. It performs:
Graph validation
Operation graph building
Execution plan creation
Execution plan support verification
Plan finalization
hipdnnHandle_t handle; hipdnnCreate(&handle); Error err = graph.build(handle); if(err.is_bad()) { handleError(); }
Note
This method does not allow setting engine knobs. If you need to configure knobs, use get_ranked_engine_ids(), get_knobs_for_engine(), and create_execution_plan_ext() instead.
- Parameters:
handle – The hipDNN handle
modes – Heuristic modes for engine selection
policy – Build plan policy (currently only HEURISTICS_CHOICE is used)
do_multithreaded_builds – Reserved for future use
- Returns:
ErrorCode::OK on success, or ErrorCode::INVALID_VALUE / ErrorCode::ATTRIBUTE_NOT_SET / ErrorCode::HIPDNN_BACKEND_ERROR on failure. Call get_message() for the specific failure reason.
-
inline Error get_workspace_size(int64_t &workspaceSize) const#
Get the workspace memory size required for execution.
Call this after build() to determine how much workspace memory to allocate.
- Parameters:
workspaceSize – Output parameter for the workspace size in bytes
- Returns:
ErrorCode::OK on success, or ErrorCode::HIPDNN_BACKEND_ERROR on failure. Call get_message() for the specific failure reason.
Execute the graph with tensor pointers mapped by tensor handles.
std::unordered_map<std::shared_ptr<TensorAttributes>, void*> tensorLookup = { {inputTensor, d_input}, {outputTensor, d_output} }; graph.execute(handle, tensorLookup, workspace);
- Parameters:
handle – The hipDNN handle
tensorLookup – Map from std::shared_ptr<TensorAttributes> (tensor handles) to device memory pointers
workspace – Pointer to workspace memory (can be nullptr if size is 0)
- Returns:
ErrorCode::OK on success, ErrorCode::INVALID_VALUE if a tensor in the lookup is null or missing a UID, or ErrorCode::HIPDNN_BACKEND_ERROR on backend failure. Call get_message() for the specific failure reason.
-
inline Error execute(hipdnnHandle_t handle, std::unordered_map<int64_t, void*> &variantPack, void *workspace) const#
Execute the graph with tensor pointers mapped by UID.
std::unordered_map<int64_t, void*> variantPack = { {0, d_input}, // UID 0 -> input tensor {1, d_weights}, // UID 1 -> weights {2, d_output} // UID 2 -> output tensor }; graph.execute(handle, variantPack, workspace);
- Parameters:
handle – The hipDNN handle
variantPack – Map from tensor UID to device memory pointers
workspace – Pointer to workspace memory (can be nullptr if size is 0)
- Returns:
ErrorCode::OK on success, or ErrorCode::HIPDNN_BACKEND_ERROR on failure. Call get_message() for the specific failure reason.
-
inline const std::string &get_name() const#
Get the graph name.
-
inline DataType get_compute_data_type() const#
Get the compute data type (precision used inside operations, e.g. accumulation)
-
inline DataType get_intermediate_data_type() const#
Get the intermediate data type (precision of virtual tensors between fused ops)
-
inline DataType get_io_data_type() const#
Get the I/O data type (precision of graph input and output tensors)
-
inline std::optional<int64_t> get_preferred_engine_id_ext() const#
Get the preferred engine ID, if set.
-
inline Graph &set_compute_data_type(DataType computeType)#
Set the compute data type (precision for internal math)
Controls the accumulation precision inside operations — the dtype used for arithmetic during execution. For mixed-precision training you might store tensors in fp16 (
io_data_type = HALF) but accumulate in fp32 (compute_data_type = FLOAT) for numerical stability.
-
inline Graph &set_intermediate_data_type(DataType intermediateType)#
Set the intermediate data type for virtual tensors between fused ops.
When the backend fuses multiple operations, intermediate results are stored in this precision. Usually matches compute_data_type.
-
inline Graph &set_io_data_type(DataType ioType)#
Set the I/O data type — the default precision for graph inputs/outputs.
This is the dtype of the tensors you feed in and read out. Individual tensors can override this by calling TensorAttributes::set_data_type().
Batch normalization forward pass for training.
Normalizes the input across the batch dimension and computes statistics.
Formula:
mean[c] = (1/m) * sum(x[n,c,h,w]) where m = N*H*W var[c] = (1/m) * sum((x[n,c,h,w] - mean[c])^2) invVar[c] = 1 / sqrt(var[c] + epsilon) y[n,c,h,w] = scale[c] * (x[n,c,h,w] - mean[c]) * invVar[c] + bias[c]
When previous running statistics are provided:
nextRunningMean = (1 - momentum) * prevRunningMean + momentum * mean nextRunningVar = (1 - momentum) * prevRunningVar + momentum * var
See also
hipdnn_frontend::graph::BatchnormAttributes
- Parameters:
x – Input tensor with batch, channel, and spatial dimensions
scale – Per-channel scale (gamma)
bias – Per-channel bias (beta)
attributes – Configuration including epsilon; optionally prev_running_mean, prev_running_variance, and momentum for exponential moving average of running statistics
- Returns:
Array of 5 output tensors:
[0] y: Normalized output (same shape as x)
[1] mean: Per-channel batch mean
[2] invVariance: Per-channel batch inverse variance
[3] nextRunningMean: Updated running mean (nullptr if not tracking)
[4] nextRunningVariance: Updated running variance (nullptr if not tracking)
Batch normalization backward pass.
Computes gradients with respect to input, scale, and bias.
Formula (using per-channel indexing for illustration):
where m = number of elements per channel (batch size * spatial dims).x_hat[c] = (x[c] - mean[c]) * invVariance[c] dbias[c] = sum(dy[c]) // sum over batch and spatial dims dscale[c] = sum(dy[c] * x_hat[c]) // sum over batch and spatial dims dx[c] = scale[c] * invVariance[c] * (dy[c] - (dbias[c] + x_hat[c] * dscale[c]) / m)
See also
hipdnn_frontend::graph::BatchnormBackwardAttributes
- Parameters:
dy – Upstream gradient (loss gradient w.r.t. output, same shape as x)
x – Original input from forward pass
scale – Per-channel scale (gamma)
attributes – Configuration; optionally set saved mean and inverse variance from the forward pass via set_saved_mean_and_inv_variance()
- Returns:
Array of 3 output tensors:
[0] dx: Gradient w.r.t. input (same shape as x)
[1] dscale: Per-channel gradient w.r.t. scale
[2] dbias: Per-channel gradient w.r.t. bias
Batch normalization inference.
Applies pre-computed normalization statistics for inference.
Formula:
y[n,c,h,w] = scale[c] * (x[n,c,h,w] - mean[c]) * invVariance[c] + bias[c]
See also
hipdnn_frontend::graph::BatchnormInferenceAttributes
- Parameters:
x – Input tensor with batch, channel, and spatial dimensions
mean – Pre-computed per-channel mean
invVariance – Pre-computed per-channel inverse variance (1/sqrt(var+epsilon))
scale – Per-channel scale (gamma)
bias – Per-channel bias (beta)
attributes – Additional configuration
- Returns:
y: Normalized output tensor (same shape as x)
Batch normalization inference with variance and epsilon tensors.
Variant that accepts variance (instead of inverse variance) and epsilon as separate input tensors, computing inverse variance internally.
Formula:
y[n,c,h,w] = scale[c] * (x[n,c,h,w] - mean[c]) / sqrt(variance[c] + epsilon) + bias[c]
See also
hipdnn_frontend::graph::BatchnormInferenceAttributesVarianceExt
- Parameters:
x – Input tensor with batch, channel, and spatial dimensions
mean – Pre-computed per-channel mean
variance – Pre-computed per-channel variance
scale – Per-channel scale (gamma)
bias – Per-channel bias (beta)
epsilon – Epsilon tensor for numerical stability (pass-by-value scalar)
attributes – Additional configuration
- Returns:
y: Normalized output tensor (same shape as x)
Layer normalization forward pass.
Normalizes the input across the last k feature dimensions, where k is inferred from the scale tensor shape. By default, all dimensions except the first (batch) dimension are normalized.
Common configurations:
Transformer: x=[B, S, D], scale=[D] → normalizes over D (k=1)
Vision: x=[N, C, H, W], scale=[1, C, H, W] → normalizes over C, H, W (k=3)
Formula:
mean = (1/m) * sum(x) over normalized dims, where m = product of normalized dims var = (1/m) * sum((x - mean)^2) over normalized dims xhat = (x - mean) / sqrt(var + epsilon) y = scale * xhat + bias
In training phase, mean and inverse variance are also returned as outputs.
See also
hipdnn_frontend::graph::LayernormAttributes
- Parameters:
x – Input tensor [N, D1, D2, …, Dk]
scale – Per-feature scale (gamma) tensor, matching the normalized dimensions. Can be full-rank with batch dims set to 1 (e.g. [1, C, H, W]) or reduced-rank with batch dims omitted (e.g. [C, H, W])
bias – Per-feature bias (beta) tensor (same shape as scale)
attributes – Configuration including epsilon and forward phase
- Returns:
Array of 3 output tensors:
[0] y: Normalized output (same shape as x)
[1] mean: Computed mean (nullptr in inference mode)
[2] invVariance: Computed inverse variance (nullptr in inference mode)
RMS normalization forward pass.
Normalizes the input using the root mean square across the channel dimension, without mean subtraction. Unlike layer normalization, RMSNorm does not center the activations.
Formula:
where C = number of channels.rms[n,h,w] = sqrt((1/C) * sum_c x[n,c,h,w]^2 + epsilon) y[n,c,h,w] = scale[c] * (x[n,c,h,w] / rms[n,h,w]) + bias[c]
In training phase, the inverse RMS is also returned as an output for use in the backward pass.
See also
hipdnn_frontend::graph::RMSNormAttributes, hipdnn_frontend::graph::LayernormAttributes
- Parameters:
x – Input tensor [N, C, H, W, …] (minimum 2 dimensions)
scale – Per-channel scale (gamma) tensor [1, C, 1, 1, …]
attributes – Configuration including epsilon, forward phase, and optional bias [1, C, 1, 1, …]
- Returns:
Array of 2 output tensors:
[0] y: Normalized output (same shape as x)
[1] invRms: Inverse RMS values (nullptr in inference mode)
Block-scale dequantization.
Dequantizes a blocked low-precision tensor using per-block scale factors. Supports MX blocked data-types (mxfp8, mxbfp8, mxfp6, mxfp4).
See also
hipdnn_frontend::graph::BlockScaleDequantizeAttributes
- Parameters:
x – Input blocked tensor to dequantize
scale – Scale tensor for block dequantization
attributes – Configuration: block_size, is_negative_scale
- Returns:
y: Dequantized output tensor
Block-scale quantization.
Quantizes an input tensor into a blocked low-precision representation with per-block scale factors. Supports MX blocked data-types (mxfp8, mxbfp8, mxfp6, mxfp4).
See also
hipdnn_frontend::graph::BlockScaleQuantizeAttributes
- Parameters:
x – Input tensor to quantize
attributes – Configuration: block_size, axis, transpose
- Returns:
[y, scale]: Quantized output tensor and computed scale tensor
Unary element-wise operation.
Applies an element-wise function to a single input tensor. The operation is specified by PointwiseAttributes::set_mode().
See also
hipdnn_frontend::graph::PointwiseAttributes, hipdnn_frontend::PointwiseMode
- Parameters:
in0 – Input tensor (arbitrary shape)
attributes – Configuration specifying the pointwise mode and any mode-specific parameters (e.g., relu_lower_clip, elu_alpha)
- Returns:
out0: Output tensor (same shape as in0)
Binary element-wise operation.
Applies an element-wise function to two input tensors. Inputs support broadcasting.
See also
hipdnn_frontend::graph::PointwiseAttributes, hipdnn_frontend::PointwiseMode
- Parameters:
in0 – First input tensor
in1 – Second input tensor (broadcastable to in0 shape)
attributes – Configuration specifying the pointwise mode
- Returns:
out0: Output tensor (broadcast shape of in0 and in1)
Ternary element-wise operation.
Applies an element-wise function to three input tensors. Currently only BINARY_SELECT uses this overload:
out[i] = in0[i] ? in1[i] : in2[i]See also
hipdnn_frontend::graph::PointwiseAttributes, hipdnn_frontend::PointwiseMode
- Parameters:
in0 – Condition tensor (selector mask)
in1 – Value selected where in0 is non-zero
in2 – Value selected where in0 is zero
attributes – Configuration specifying the pointwise mode
- Returns:
out0: Output tensor
Reduction operation.
Reduces an input tensor along one or more dimensions using the specified reduction mode. Creates a new output tensor managed by the graph.
See also
ReductionAttributes, ReductionMode
- Parameters:
x – Input tensor (arbitrary shape)
attributes – Configuration specifying the reduction mode
- Returns:
y: Output tensor (graph-managed, shape inferred during build)
Reduction operation with explicit output tensor.
Reduces an input tensor along one or more dimensions using the specified reduction mode. The caller provides the output tensor, allowing explicit control over output shape for partial reductions.
See also
ReductionAttributes, ReductionMode
- Parameters:
x – Input tensor (arbitrary shape)
y – Output tensor (caller-provided, reduced shape)
attributes – Configuration specifying the reduction mode
- Returns:
y: The provided output tensor
Matrix multiplication.
Computes the matrix product of two tensors with optional batch dimensions.
Formula:
C[..., i, j] = sum_k( A[..., i, k] * B[..., k, j] )
Batch dimensions are broadcast when they differ (one must be divisible by the other).
See also
hipdnn_frontend::graph::MatmulAttributes
- Parameters:
a – Left input matrix […, M, K]
b – Right input matrix […, K, N]
attributes – Additional configuration
- Returns:
c: Output matrix […, M, N]
Add a custom operation to the graph.
Custom ops let users coordinate directly with plugins without requiring hipDNN to understand the operation. hipDNN transports the tensor I/O topology and an opaque byte payload, and the target plugin interprets the payload.
See also
hipdnn_frontend::graph::CustomOpAttributes
Note
This operation requires a matching custom plugin to find an engine. It will fail engine selection unless a plugin is loaded that explicitly handles the specified
custom_op_id.- Parameters:
inputs – Input tensors (variable length)
numOutputs – Number of output tensors to create
attributes – Custom op configuration including opaque payload
- Returns:
Vector of output tensors
Scaled dot-product attention forward pass.
Computes scaled dot-product attention:
Attention(Q, K, V) = softmax(Q * K^T / sqrt(d_k)) * V
Supports optional causal masking, attention bias, dropout, paged attention, and FP8 quantization via descale/scale tensors.
See also
hipdnn_frontend::graph::SdpaAttributes
- Parameters:
q – Query tensor [B, H, S_q, D]
k – Key tensor [B, H, S_kv, D]
v – Value tensor [B, H, S_kv, D]
attributes – Configuration: masking, dropout, attention scale, paged attention, and other SDPA options
- Returns:
[o, stats]: Output tensor [B, H, S_q, D] and optional softmax statistics (nullptr if generate_stats is not set)
Scaled dot-product attention backward pass.
Computes gradients dQ, dK, dV for the backward pass of SDPA:
Attention(Q, K, V) = softmax(Q * K^T / sqrt(d_k)) * V
Requires softmax statistics (logsumexp) from the forward pass, which are generated when the forward pass is configured with
set_generate_stats(true).See also
hipdnn_frontend::graph::SdpaBackwardAttributes, hipdnn_frontend::graph::SdpaAttributes
- Parameters:
q – Query tensor from forward pass [B, H, S_q, D]
k – Key tensor from forward pass [B, H, S_kv, D]
v – Value tensor from forward pass [B, H, S_kv, D]
o – Output tensor from forward pass [B, H, S_q, D]
dO – Upstream gradient tensor [B, H, S_q, D]
stats – Softmax statistics (logsumexp) from forward pass [B, H, S_q, 1]
attributes – Configuration: masking, dropout, attention scale
- Returns:
Array of 3 output tensors:
[0] dQ: Gradient w.r.t. query [B, H, S_q, D]
[1] dK: Gradient w.r.t. key [B, H, S_kv, D]
[2] dV: Gradient w.r.t. value [B, H, S_kv, D]
Convolution forward pass.
Computes a cross-correlation (or convolution) of the input with filters.
Example for 2D (using NCHW notation for illustration):
y[n,k,oh,ow] = sum_c,r,s x[n, c, oh*stride_h + r*dilation_h - pad_h, ow*stride_w + s*dilation_w - pad_w] * w[k, c, r, s] output_dim = floor((input + pad_before + pad_after - dilation * (kernel - 1) - 1) / stride) + 1
See also
hipdnn_frontend::graph::ConvFpropAttributes
- Parameters:
x – Input activation tensor (batch, channels, spatial dimensions)
w – Filter/weight tensor (output channels, input channels, filter spatial dims)
attributes – Convolution parameters: padding, stride, dilation, convolution mode
- Returns:
y: Output activation tensor
Convolution data gradient (backward data)
Computes the gradient of the loss with respect to the convolution input, given the output gradient and the filter weights. Used during backpropagation.
Example for 2D (using NCHW notation for illustration):
dx[n,c,h,w] = sum_k,r,s dy[n, k, p, q] * w[k, c, r, s] where p = (h + pad_h - r*dilation_h) / stride_h (integer, in [0, H_out)) q = (w + pad_w - s*dilation_w) / stride_w (integer, in [0, W_out))
See also
hipdnn_frontend::graph::ConvDgradAttributes
- Parameters:
dy – Upstream gradient (loss gradient w.r.t. conv output)
w – Filter/weight tensor
attributes – Convolution parameters: padding, stride, dilation (must match forward pass)
- Returns:
dx: Gradient w.r.t. input (same shape as forward input)
Convolution weight gradient (backward weights)
Computes the gradient of the loss with respect to the filter weights, given the output gradient and the original input. Used during backpropagation.
Example for 2D (using NCHW notation for illustration):
dw[k,c,r,s] = sum_n,p,q dy[n, k, p, q] * x[n, c, h, w] where h = p*stride_h - pad_h + r*dilation_h w = q*stride_w - pad_w + s*dilation_w
See also
hipdnn_frontend::graph::ConvWgradAttributes
- Parameters:
dy – Upstream gradient (loss gradient w.r.t. conv output)
x – Original input activation tensor
attributes – Convolution parameters: padding, stride, dilation (must match forward pass)
- Returns:
dw: Gradient w.r.t. filter weights (same shape as forward weights)
Public Static Functions
Create a new tensor with similar properties to an existing tensor.
Creates a new TensorAttributes object by copying properties from the provided tensor, but clears the UID and optionally assigns a new name. This is useful for creating tensors with similar dimensions and data types but representing different data.
// Create a tensor similar to x but with a different UID auto y = Graph::tensor_like(x, "output"); y->set_uid(2);
- Parameters:
tensor – The tensor to copy properties from
name – Optional name for the new tensor
- Returns:
Shared pointer to the newly created TensorAttributes
-
static inline std::shared_ptr<TensorAttributes> tensor(const TensorAttributes &tensor)#
Create a new tensor from existing tensor attributes.
Creates a new TensorAttributes object as a copy of the provided tensor, preserving all properties including UID. This is the standard way to create a tensor for use in graph operations.
// Create a tensor from attributes auto x = Graph::tensor(TensorAttributes() .set_dim({1, 64, 28, 28}) .set_stride({50176, 784, 28, 1}) .set_data_type(DataType::HALF) .set_uid(0));
See also
execute() for passing device memory pointers at execution time
See also
tensor_like() for creating a tensor with cleared UID and custom name
Note
This creates a tensor descriptor (shape, type, strides) only. No device memory is allocated. Device pointers are provided at execution time via the variant pack.
- Parameters:
tensor – The tensor attributes to copy
- Returns:
Shared pointer to the newly created TensorAttributes
Protected Functions
-
inline hipdnnBackendDescriptor_t get_raw_graph_descriptor() const#
-
inline Error build_operation_graph_via_descriptors(hipdnnHandle_t handle)#
Builds the operation graph using the backend descriptor C API. Each node creates its operation descriptor(s) via virtual dispatch, then the GraphDescriptor is assembled and finalized.
NOTE: This method is intentionally not yet exposed publicly. It will replace the FlatBuffer-based build_operation_graph() once all operation types are implemented.
Private Functions
-
inline Error applyKnobSettingsToEngineConfig(const std::vector<KnobSetting> &validatedSettings)#
Apply validated knob settings to the engine config descriptor, using either the descriptor-based or FlatBuffer serialization path depending on the HIPDNN_USE_DESCRIPTOR_API feature flag.
-
inline void assignUnsetTensorUids()#
-
inline Error initializeEngineConfig(hipdnnBackendDescriptor_t engineHeuristicDesc)#
-
inline Error initializeEngineConfig(int64_t engineId)#
Initialize engine config for a specific engine ID.
Note
This method does NOT finalize the engine config. The caller must finalize after setting any knobs on the config.
- Parameters:
engineId – The engine to configure
-
inline std::unordered_map<std::shared_ptr<TensorAttributes>, size_t> buildTensorToOriginNodeMap() const#
-
inline void reorderNodesTopologically(const std::vector<size_t> &topologicalOrder)#
-
inline std::pair<std::unordered_set<std::shared_ptr<TensorAttributes>>, std::unordered_set<std::shared_ptr<TensorAttributes>>> getGraphInputTensorAttributesAndRemainder() const#
-
inline flatbuffers::DetachedBuffer buildFlatbufferOperationGraphConst() const#
Private Members
-
std::unique_ptr<detail::ScopedHipdnnBackendDescriptor> _graphDesc#
-
std::unique_ptr<detail::ScopedHipdnnBackendDescriptor> _engineConfigDesc#
-
std::unique_ptr<detail::ScopedHipdnnBackendDescriptor> _executionPlanDesc#
-
std::optional<int64_t> _preferredEngineId#
Private Static Functions
-
static inline std::optional<int64_t> getDefaultEngineId()#
-
static inline bool useDescriptorApi()#
-
static inline std::shared_ptr<TensorAttributes> outputTensor(const std::string &name)#
-
static inline int64_t getUnusedTensorUid(int64_t ¤tTensorId, std::unordered_set<int64_t> &usedIds)#
-
inline Error validate()#
-
class Graph : public hipdnn_frontend::graph::INode#
-
namespace graph#