Layernorm

Layernorm#

MIOpen: Layernorm
Layernorm

Enumerations

enum  miopenNormMode_t {
  MIOPEN_ELEMENTWISE_AFFINE = 0 ,
  MIOPEN_WEIGHT_BIAS
}
 

Functions

miopenStatus_t miopenLayerNormForward (miopenHandle_t handle, miopenNormMode_t mode, const miopenTensorDescriptor_t xDesc, const void *x, const miopenTensorDescriptor_t weightDesc, const void *weight, const miopenTensorDescriptor_t biasDesc, const void *bias, const float epsilon, const int32_t normalized_dim, const miopenTensorDescriptor_t yDesc, void *y, const miopenTensorDescriptor_t meanDesc, void *mean, const miopenTensorDescriptor_t rstdDesc, void *rstd)
 Execute a layernorm forward layer. More...
 

Detailed Description

Enumeration Type Documentation

◆ miopenNormMode_t

LayerNorm mode

Enumerator
MIOPEN_ELEMENTWISE_AFFINE 

initialized to ones for weights and zeros for biases

MIOPEN_WEIGHT_BIAS 

learnable weights and biases of the module of shape normalized_shape

Examples
/home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-miopen/checkouts/docs-6.1.2/include/miopen/miopen.h.

Function Documentation

◆ miopenLayerNormForward()

miopenStatus_t miopenLayerNormForward ( miopenHandle_t  handle,
miopenNormMode_t  mode,
const miopenTensorDescriptor_t  xDesc,
const void *  x,
const miopenTensorDescriptor_t  weightDesc,
const void *  weight,
const miopenTensorDescriptor_t  biasDesc,
const void *  bias,
const float  epsilon,
const int32_t  normalized_dim,
const miopenTensorDescriptor_t  yDesc,
void *  y,
const miopenTensorDescriptor_t  meanDesc,
void *  mean,
const miopenTensorDescriptor_t  rstdDesc,
void *  rstd 
)

Execute a layernorm forward layer.

Parameters
handleMIOpen handle (input)
modeLayerNorm mode (input)
xDescTensor descriptor for data input tensor x (input)
xData tensor x (input)
weightDescTensor descriptor for data input tensor weight (input)
weightData tensor weight (input)
biasDescTensor descriptor for data input tensor bias (input)
biasData tensor bias (input)
epsilonValue to stablize inverse variance calculation (input)
normalized_dimNomalized dimensions in the input array (input)
yDescTensor descriptor for output data tensor y (input)
yData tensor y (output)
meanDescTensor descriptor for output data tensor mean (input)
meanData tensor mean (output)
rstdDescTensor descriptor for output data tensor rstd (input)
rstdData tensor rstd (output)
Returns
miopenStatus_t
Examples
/home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-miopen/checkouts/docs-6.1.2/include/miopen/miopen.h.