Supported ONNX Operators

Supported ONNX Operators#

2024-12-04

13 min read time

Applies to Linux

MIGraphX supports operators up to Opset 19. Latest information of ONNX operators can be found in the ONNX GitHub repository.

MIGraphX supports the following ONNX data types: BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FLOAT8, FLOAT16, FLOAT32, and DOUBLE

Note

FP8 support is only for E4M3FNUZ, see Float stored in 8 bits in the ONNX documentation.

See below for the support matrix of ONNX operators in MIGraphX.

Note

The listed supported types are taken from the ONNX specification. An operator might support other additional datatypes.

Operator Support Matrix#

Operator

Supported

Supported Types

Limitations

Abs

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Acos

✅

FP8, FP16, FP32, FP64

Acosh

✅

FP8, FP16, FP32, FP64

Add

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

And

✅

BOOL

ArgMax

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

ArgMin

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Asin

✅

FP8, FP16, FP32, FP64

Asinh

✅

FP8, FP16, FP32, FP64

Atan

✅

FP8, FP16, FP32, FP64

Atanh

✅

FP8, FP16, FP32, FP64

AveragePool

✅

FP8, FP16, FP32, FP64

BatchNormalization

✅

FP8, FP16, FP32, FP64

BiasGelu

✅

FP8, FP16, FP32, FP64

Bernoulli

❌

BitShift

❌

BitwiseAnd

❌

BitwiseNot

❌

BitwiseOr

❌

BitwiseXor

❌

BlackmanWindow

❌

Cast

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP16, FP32, FP64

saturate not supported

CastLike

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP16, FP32, FP64

saturate not supported

Ceil

✅

FP8, FP16, FP32, FP64

Celu

✅

FP32

CenterCropPad

❌

Clip

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Col2Im

❌

Compress

❌

Concat

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

ConcatFromSequence

❌

Constant

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

ConstantOfShape

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP16, FP32, FP64

dynamic shape is not supported

Conv

✅

FP8, FP16, FP32, FP64

ConvInteger

✅

INT8

ConvTranspose

✅

FP8, FP16, FP32, FP64

auto padding not supported, output_padding math differs

Cos

✅

FP8, FP16, FP32, FP64

Cosh

✅

FP8, FP16, FP32, FP64

CumSum

✅

UINT32, UINT64, INT32, INT64, FP8, FP16, FP32, FP64

axis dynamic shape is not supported

DFT

❌

DeformConv

❌

DepthToSpace

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

DequantizeLinear

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Det

❌

Div

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Dropout

✅

Any

Changed to identity

DynamicQuantizeLinear

✅

FP8, FP16, FP32, FP64

uint8 quantization only, dynamic shape is not supported

Einsum

✅

Any

more than 1 diagonal per input is not supported e.g. iijj->ij

batch diagonal where batches are not the leading dims is not supported e.g. ii...->i...

Elu

✅

FP8, FP16, FP32, FP64

Equal

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Erf

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Exp

✅

FP8, FP16, FP32, FP64

Expand

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

dynamic shape is not supported

EyeLike

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

FastGelu

✅

FP8, FP16, FP32

Flatten

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Floor

✅

FP8, FP16, FP32, FP64

Gather

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

GatherElements

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

GatherND

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Gelu

✅

FP8, FP16, FP32, FP64

Gemm

✅

UINT32, UINT64, INT32, INT64, FP8, FP16, FP32, FP64

GlobalAveragePool

✅

FP8, FP16, FP32, FP64

GlobalLpPool

✅

FP8, FP16, FP32, FP64

GlobalMaxPool

✅

FP8, FP16, FP32, FP64

Greater

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

GreaterOrEqual

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

GridSample

✅

UINT32, UINT64, INT32, INT64, FP16, FP32, FP64

5-D inputs not supported

GroupNormalization

✅

FP8, FP16, FP32, FP64

stash_type not supported

GRU

✅

FP8, FP16, FP32, FP64

Affine, ThresholdRelu, ScaledTanh, HardSigmoid, Softsign, Softplus optional activation functions are not enabled

HammingWindow

❌

HannWindow

❌

HardSigmoid

✅

FP8, FP16, FP32, FP64

HardSwish

✅

FP8, FP16, FP32, FP64

Hardmax

✅

FP8, FP16, FP32, FP64

Identity

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

identity, sequence datatypes are not supported

If

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

identity, sequence datatypes are not supported

InstanceNormalization

✅

FP16, FP32, FP64

IsInf

✅

FP8, FP16, FP32, FP64

IsNaN

✅

FP8, FP16, FP32, FP64

LayerNormalization

✅

FP8, FP16, FP32, FP64

stash_type not supported

LeakyRelu

✅

FP8, FP16, FP32, FP64

Less

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

LessOrEqual

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Log

✅

FP8, FP16, FP32, FP64

LogSoftmax

✅

FP8, FP16, FP32, FP64

Loop

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

identity, sequence datatypes are not supported, max_iteration has upper-bound

LRN

✅

FP8, FP16, FP32, FP64

LSTM

✅

FP32, FP16

Affine, ThresholdRelu, ScaledTanh, HardSigmoid, Softsign, Softplus optional activation functions are not enabled

LpNormalization

✅

FP8, FP16, FP32, FP64

LpPool

✅

FP32, FP16, FP8, INT8

lpnorm not supported pooling mode on GPU (MIOpen limitation)

MatMul

✅

UINT32, UINT64, INT32, INT64, FP8, FP16, FP32, FP64

MatMulInteger

✅

UINT8, INT8

dynamic shape is not supported

Max

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

MaxPool

✅

FP32, FP16, FP8, INT8

storage_order not supported, dialtion is partially supported on GPU (MIOpen limitation), indices 2nd ouput not supported

MaxRoiPool

❌

MaxUnpool

❌

Mean

✅

FP8, FP16, FP32, FP64

MeanVarian ceNormalization

✅

FP8, FP16, FP32, FP64

MelWeightMatrix

❌

Min

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Mish

❌

Mod

✅

UINT8, UINT16, UINT32, UINT64, INT16, INT64, FP8, FP16, FP32, FP64

int8, int32 issue on GPU

Mul

✅

UINT8, UINT16, UINT32, UINT64, INT16, INT64, FP8, FP16, FP32, FP64

Multinomial

✅

FP8, FP16, FP32, FP64

Neg

✅

INT8, INT32, INT64, FP8, FP16, FP32, FP64

NegativeLogLikelihoodLoss

❌

NonMaxSuppression

✅

FP8, FP16, FP32, FP64

fixed output size unless use_dyn_output set

NonZero

✅

FP8, FP16, FP32, FP64

fixed output size unless use_dyn_output set

Not

✅

BOOL

OneHot

✅

UINT8, UINT16, UINT32, UINT64, INT16, INT64, FP8, FP16, FP32, FP64

dynamic shape is not supported

Optional

❌

OptionalGetElement

❌

OptionalHasElement

❌

Or

✅

BOOL

Pad

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

edge, warp padding not supported, pads must be constant

Pow

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

PRelu

✅

UINT32, UINT64, INT32, INT64, FP8, FP16, FP32, FP64

QLinearAdd

✅

UINT8, INT8

QLinearAveragePool

✅

UINT8, INT8

QLinearConcat

✅

UINT8, INT8

QLinearConv

✅

UINT8, INT8

QLinearGlobalAveragePool

✅

UINT8, INT8

QLinearLeakyRelu

✅

UINT8, INT8

QLinearMatMul

✅

UINT8, INT8

non-scalar inputs are not supported

QLinearMul

✅

UINT8, INT8

QLinearSigmoid

✅

UINT8, INT8

QuantizeLinear

✅

FP8, FP16, FP32, INT32

saturate is not supported

RandomNormal

✅

FP16, FP32, FP64

RandomNormalLike

✅

FP16, FP32, FP64

RandomUniform

✅

FP16, FP32, FP64

RandomUniformLike

✅

FP16, FP32, FP64

Range

✅

FP16, FP32, FP64, INT16, INT32, INT64

start, end, delta dynamic shape is not supported

Reciprocal

✅

FP8, FP16, FP32, FP64

ReduceL1

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

ReduceL2

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

ReduceLogSum

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

ReduceLogSumExp

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

ReduceMax

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

ReduceMean

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

ReduceMin

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

ReduceProd

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

ReduceSum

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

ReduceSumSquare

✅

FP16, FP32, FP64, UINT32, UINT64, INT32, INT64

axes dynamic shape is not supported

Relu

✅

FP16, FP32, FP64, INT8, INT16, INT32, INT64

Reshape

✅

FP32, FP16, INT32, INT64, FP8, INT8, BOOL

allowzero not supported, dynamic shape is not supported

Resize

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

cubic, half_pixel_symmetric, tf_crop_and_resize not supported, linear mode not supported for non-constant inputs, exclude_outside 1 is not supported, antialias, extrapolation_value, keep_aspect_ratio_policy not supported

ReverseSequence

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

variable sequence_lens is not supported

RNN

✅

FP32, FP16

Affine, ThresholdRelu, ScaledTanh, HardSigmoid, Softsign, Softplus optional activation functions are not enabled

RoiAlign

✅

FP8, FP16, FP32, FP64

Round

✅

FP8, FP16, FP32, FP64

STFT

❌

Scan

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

identity, sequence datatypes are not supported, Number of iterations has upper-bound Version 8 not supported

Scatter (deprecated)

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

ScatterElements

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

ScatterND

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Selu

✅

FP8, FP16, FP32, FP64

SequenceAt

❌

SequenceConstruct

❌

SequenceEmpty

❌

SequenceErase

❌

SequenceInsert

❌

SequenceLength

❌

SequenceMap

❌

Shape

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Shrink

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Sigmoid

✅

FP8, FP16, FP32, FP64

Sign

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Sin

✅

FP8, FP16, FP32, FP64

Sinh

✅

FP8, FP16, FP32, FP64

Size

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Slice

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

variable inputs are not supported

Softmax

✅

FP8, FP16, FP32, FP64

SoftmaxCrossEntropyLoss

❌

Softplus

✅

FP8, FP16, FP32, FP64

Softsign

✅

FP8, FP16, FP32, FP64

SpaceToDepth

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Split

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

dynamic shape is not supported

SplitToSequence

❌

Sqrt

✅

FP8, FP16, FP32, FP64

Squeeze

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

variable axes is not supported

StringNormalizer

❌

Sub

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Sum

✅

FP8, FP16, FP32, FP64

Tan

✅

FP8, FP16, FP32, FP64

Tanh

✅

FP8, FP16, FP32, FP64

TfIdfVectorizer

❌

ThresholdedRelu

✅

FP8, FP16, FP32, FP64

Tile

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

dynamic shape is not supported

TopK

✅

UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

dynamic k is not supported, sorted is not supported

Transpose

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Trilu

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

dynamic k is not supported

Unique

✅

Any

only axis = 0 is supported

Unsqueeze

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

variable axes is not supported

Upsample (deprecated)

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

Where

✅

BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP8, FP16, FP32, FP64

mixed static and dynamic shape inputs are not supported

Xor

✅

BOOL
Contents