Activation Layers

Activation Layers#

Activation types and layers.

miopenActivationMode_t#

enum miopenActivationMode_t#

Activation layer modes

Values:

enumerator miopenActivationPASTHRU#

No activation, pass through the data

enumerator miopenActivationLOGISTIC#

Sigmoid function: \(1 / (1 + e^{-x})\)

enumerator miopenActivationTANH#

Tanh activation \( \beta * tanh( \alpha * x) \)

enumerator miopenActivationRELU#

Rectified Linear Unit \( max(0, x) \)

enumerator miopenActivationSOFTRELU#

\(log(1 + e^x)\)

enumerator miopenActivationABS#

Absolute value \(abs(x)\)

enumerator miopenActivationPOWER#

Scaled and shifted power \((\alpha + \beta * x)^{gamma}\)

enumerator miopenActivationCLIPPEDRELU#

Clipped Rectified Linear Unit \( min(\alpha, max(0,x)) \)

enumerator miopenActivationLEAKYRELU#

Leaky Rectified Linear Unit \( \alpha * x | x <= 0; x | x > 0 \)

enumerator miopenActivationELU#

Exponential Rectified Linear Unit \( \alpha * (e^{x} - 1) | x <= 0; x | x > 0 \)

miopenCreateActivationDescriptor#

miopenStatus_t miopenCreateActivationDescriptor(miopenActivationDescriptor_t *activDesc)#

Creates the Activation descriptor object.

Parameters:

activDesc – Pointer to an activation tensor descriptor type

Returns:

miopenStatus_t

miopenSetActivationDescriptor#

miopenStatus_t miopenSetActivationDescriptor(const miopenActivationDescriptor_t activDesc, miopenActivationMode_t mode, double activAlpha, double activBeta, double activGamma)#

Sets the activation layer descriptor details.

Sets all of the descriptor details for the activation layer

Parameters:

  • activDesc – Pointer to a activation layer descriptor (output)

  • mode – Activation mode enum (input)

  • activAlpha – Alpha value for some activation modes (input)

  • activBeta – Beta value for some activation modes (input)

  • activGamma – Gamma value for some activation modes (input)

Returns:

miopenStatus_t

miopenGetActivationDescriptor#

miopenStatus_t miopenGetActivationDescriptor(const miopenActivationDescriptor_t activDesc, miopenActivationMode_t *mode, double *activAlpha, double *activBeta, double *activGamma)#

Gets the activation layer descriptor details.

Retrieves all of the descriptor details for the activation layer.

Parameters:

  • activDesc – Pointer to a activation layer descriptor (input)

  • mode – Activation mode enum (output)

  • activAlpha – Alpha value for some activation modes (output)

  • activBeta – Beta value for some activation modes (output)

  • activGamma – Gamma value for some activation modes (output)

Returns:

miopenStatus_t

miopenActivationForward#

miopenStatus_t miopenActivationForward(miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t yDesc, void *y)#

Execute an activation forward layer.

Parameters:

  • handle – MIOpen handle (input)

  • activDesc – Descriptor for activation layer (input)

  • alpha – Floating point scaling factor, allocated on the host (input)

  • xDesc – Tensor descriptor for data input tensor x (input)

  • x – Data tensor x (input)

  • beta – Floating point shift factor, allocated on the host (input)

  • yDesc – Tensor descriptor for output data tensor y (input)

  • y – Data tensor y (output)

Returns:

miopenStatus_t

miopenActivationBackward#

miopenStatus_t miopenActivationBackward(miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t yDesc, const void *y, const miopenTensorDescriptor_t dyDesc, const void *dy, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t dxDesc, void *dx)#

Execute a activation backwards layer.

Parameters:

  • handle – MIOpen handle (input)

  • activDesc – Descriptor for activation layer (input)

  • alpha – Floating point scaling factor, allocated on the host (input)

  • yDesc – Tensor descriptor for input data tensor y (input)

  • y – Data tensor y (input)

  • dyDesc – Tensor descriptor for input data tensor dy (input)

  • dy – Data delta tensor dy (input)

  • xDesc – Tensor descriptor for data input tensor x (input)

  • x – Data tensor x (input)

  • beta – Floating point shift factor, allocated on the host (input)

  • dxDesc – Tensor descriptor for data output tensor dx (input)

  • dx – Output data delta tensor dx (output)

Returns:

miopenStatus_t

miopenDestroyActivationDescriptor#

miopenStatus_t miopenDestroyActivationDescriptor(miopenActivationDescriptor_t activDesc)#

Destroys the activation descriptor object.

Parameters:

activDesc – Activation tensor descriptor type (input)

Returns:

miopenStatus_t

Contents