Activation Layers

Activation Layers#

Activation types and layers.

miopenActivationMode_t#

enum miopenActivationMode_t#

Activation layer modes

Values:

enumerator miopenActivationPASTHRU#

No activation, pass through the data

enumerator miopenActivationLOGISTIC#

Sigmoid function: 1/(1+e−x)

enumerator miopenActivationTANH#

Tanh activation β∗tanh(α∗x)

enumerator miopenActivationRELU#

Rectified Linear Unit max(0,x)

enumerator miopenActivationSOFTRELU#

log(1+ex)

enumerator miopenActivationABS#

Absolute value abs(x)

enumerator miopenActivationPOWER#

Scaled and shifted power (α+β∗x)gamma

enumerator miopenActivationCLIPPEDRELU#

Clipped Rectified Linear Unit min(α,max(0,x))

enumerator miopenActivationLEAKYRELU#

Leaky Rectified Linear Unit α∗x|x<=0;x|x>0

enumerator miopenActivationELU#

Exponential Rectified Linear Unit α∗(ex−1)|x<=0;x|x>0

miopenCreateActivationDescriptor#

miopenStatus_t miopenCreateActivationDescriptor(miopenActivationDescriptor_t *activDesc)#

Creates the Activation descriptor object.

Parameters:

activDesc – Pointer to an activation tensor descriptor type

Returns:

miopenStatus_t

miopenSetActivationDescriptor#

miopenStatus_t miopenSetActivationDescriptor(const miopenActivationDescriptor_t activDesc, miopenActivationMode_t mode, double activAlpha, double activBeta, double activGamma)#

Sets the activation layer descriptor details.

Sets all of the descriptor details for the activation layer

Parameters:

  • activDesc – Pointer to a activation layer descriptor (output)

  • mode – Activation mode enum (input)

  • activAlpha – Alpha value for some activation modes (input)

  • activBeta – Beta value for some activation modes (input)

  • activGamma – Gamma value for some activation modes (input)

Returns:

miopenStatus_t

miopenGetActivationDescriptor#

miopenStatus_t miopenGetActivationDescriptor(const miopenActivationDescriptor_t activDesc, miopenActivationMode_t *mode, double *activAlpha, double *activBeta, double *activGamma)#

Gets the activation layer descriptor details.

Retrieves all of the descriptor details for the activation layer.

Parameters:

  • activDesc – Pointer to a activation layer descriptor (input)

  • mode – Activation mode enum (output)

  • activAlpha – Alpha value for some activation modes (output)

  • activBeta – Beta value for some activation modes (output)

  • activGamma – Gamma value for some activation modes (output)

Returns:

miopenStatus_t

miopenActivationForward#

miopenStatus_t miopenActivationForward(miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t yDesc, void *y)#

Execute an activation forward layer.

Parameters:

  • handle – MIOpen handle (input)

  • activDesc – Descriptor for activation layer (input)

  • alpha – Floating point scaling factor, allocated on the host (input)

  • xDesc – Tensor descriptor for data input tensor x (input)

  • x – Data tensor x (input)

  • beta – Floating point shift factor, allocated on the host (input)

  • yDesc – Tensor descriptor for output data tensor y (input)

  • y – Data tensor y (output)

Returns:

miopenStatus_t

miopenActivationBackward#

miopenStatus_t miopenActivationBackward(miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t yDesc, const void *y, const miopenTensorDescriptor_t dyDesc, const void *dy, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t dxDesc, void *dx)#

Execute a activation backwards layer.

Parameters:

  • handle – MIOpen handle (input)

  • activDesc – Descriptor for activation layer (input)

  • alpha – Floating point scaling factor, allocated on the host (input)

  • yDesc – Tensor descriptor for input data tensor y (input)

  • y – Data tensor y (input)

  • dyDesc – Tensor descriptor for input data tensor dy (input)

  • dy – Data delta tensor dy (input)

  • xDesc – Tensor descriptor for data input tensor x (input)

  • x – Data tensor x (input)

  • beta – Floating point shift factor, allocated on the host (input)

  • dxDesc – Tensor descriptor for data output tensor dx (input)

  • dx – Output data delta tensor dx (output)

Returns:

miopenStatus_t

miopenDestroyActivationDescriptor#

miopenStatus_t miopenDestroyActivationDescriptor(miopenActivationDescriptor_t activDesc)#

Destroys the activation descriptor object.

Parameters:

activDesc – Activation tensor descriptor type (input)

Returns:

miopenStatus_t

Contents