Dropout Layer

The dropout layer API documentation

miopenRNGType_t

enum miopenRNGType_t

random number generator type

Values:

enumerator MIOPEN_RNG_PSEUDO_XORWOW

XORWOW pseudorandom generator

miopenCreateDropoutDescriptor

miopenStatus_t miopenCreateDropoutDescriptor(miopenDropoutDescriptor_t *dropoutDesc)

Creates the dropout descriptor object.

Parameters:

dropoutDesc – Pointer to a dropout descriptor type

Returns:

miopenStatus_t

miopenGetDropoutDescriptor

miopenStatus_t miopenGetDropoutDescriptor(miopenDropoutDescriptor_t dropoutDesc, miopenHandle_t handle, float *dropout, void **states, unsigned long long *seed, bool *use_mask, bool *state_evo, miopenRNGType_t *rng_mode)

Get the details of the dropout descriptor.

Interface for querying the dropout descriptor

Parameters:

• dropoutDesc – Dropout layer descriptor (input)

• handle – MIOpen handle (input)

• dropout – The probability by which the input is set to 0 in the dropout layer (Output)

• states – Pointer to memory that holds random number generator states (Output)

• seed – Seed used to initialize random number generator states (Output)

• use_mask – Boolean flag indicating whether to use a saved mask (an existing or user-defined dropout layout) in reserveSpace (Output)

• state_evo – Boolean flag indicating whether to adopt state evolution strategy to update the PRNG states by the end of each implementation (Output placeholder, currently not enabled)

• rng_mode – Random number generator used to generate parallel random number sequences (Output)

Returns:

miopenStatus_t

miopenRestoreDropoutDescriptor

miopenStatus_t miopenRestoreDropoutDescriptor(miopenDropoutDescriptor_t dropoutDesc, miopenHandle_t handle, float dropout, void *states, size_t stateSizeInBytes, unsigned long long seed, bool use_mask, bool state_evo, miopenRNGType_t rng_mode)

Restore the dropout descriptor to a saved state.

This function restores the state of dropout descriptor using the address of a state buffer with previously saved PRNG state pattern, without launching the expensive PRNG initialization process.

Interface for restoring the dropout descriptor

Parameters:

• dropoutDesc – Dropout layer descriptor (input/Output)

• handle – MIOpen handle (input)

• dropout – The probability by which the input is set to 0 in the dropout layer (input)

• states – Pointer to memory that holds random number generator states (input)

• stateSizeInBytes – Number of bytes holding random generator states (input)

• seed – Seed used to initialize random number generator states (input)

• use_mask – Boolean flag indicating whether to use a saved mask (an existing or user-defined dropout layout) in reserveSpace (input)

• state_evo – Boolean flag indicating whether to adopt state evolution strategy to update the PRNG states by the end of each implementation (input placeholder, currently not enabled)

• rng_mode – Random number generator used to generate parallel random number sequences (input)

Returns:

miopenStatus_t

miopenDestroyDropoutDescriptor

miopenStatus_t miopenDestroyDropoutDescriptor(miopenDropoutDescriptor_t dropoutDesc)

Destroys the dropout descriptor object.

Parameters:

dropoutDesc – Dropout descriptor type (input)

Returns:

miopenStatus_t

miopenSetDropoutDescriptor

miopenStatus_t miopenSetDropoutDescriptor(miopenDropoutDescriptor_t dropoutDesc, miopenHandle_t handle, float dropout, void *states, size_t stateSizeInBytes, unsigned long long seed, bool use_mask, bool state_evo, miopenRNGType_t rng_mode)

Initialize the dropout descriptor.

Interface for setting up the dropout descriptor

Parameters:

• dropoutDesc – Dropout layer descriptor (input/Output)

• handle – MIOpen handle (input)

• dropout – The probability by which the input is set to 0 in the dropout layer (input)

• states – Pointer to memory that holds random number generator states (input)

• stateSizeInBytes – Number of bytes provided for random generator states (input)

• seed – Seed used to initialize random number generator states (input)

• use_mask – Boolean flag indicating whether to use a saved mask (an existing or user-defined dropout layout) in reserveSpace (input)

• state_evo – Boolean flag indicating whether to adopt state evolution strategy to update the PRNG states by the end of each implementation (input placeholder, currently not enabled)

• rng_mode – Random number generator used to generate parallel random number sequences (input)

Returns:

miopenStatus_t

miopenDropoutGetReserveSpaceSize

miopenStatus_t miopenDropoutGetReserveSpaceSize(const miopenTensorDescriptor_t xDesc, size_t *reserveSpaceSizeInBytes)

Query the amount of memory required to run dropout.

This function calculates the amount of memory required to run dropout.

Parameters:

• xDesc – Tensor descriptor for data tensor x (input)

• reserveSpaceSizeInBytes – Number of bytes of reservespace required for executing dropout (Output)

Returns:

miopenStatus_t

miopenDropoutGetStatesSize

miopenStatus_t miopenDropoutGetStatesSize(miopenHandle_t handle, size_t *stateSizeInBytes)

Query the amount of memory required to store the states of the random number generators.

This function calculates the amount of memory required to store the states of the random number generators used by miopenDropoutForward.

Parameters:

• handle – MIOpen handle (input)

• stateSizeInBytes – Number of bytes required to store random generator states (Output)

Returns:

miopenStatus_t

miopenDropoutForward

miopenStatus_t miopenDropoutForward(miopenHandle_t handle, const miopenDropoutDescriptor_t dropoutDesc, const miopenTensorDescriptor_t noise_shape, const miopenTensorDescriptor_t xDesc, const void *x, const miopenTensorDescriptor_t yDesc, void *y, void *reserveSpace, size_t reserveSpaceSizeInBytes)

Execute forward dropout operation.

Interface for executing the forward pass on a Dropout.

Parameters:

• handle – MIOpen handle (input)

• dropoutDesc – Dropout layer descriptor (input)

• noise_shape – Tensor descriptor for noise shape (input placeholder, currently not enabled)

• xDesc – Tensor descriptor for data tensor x (input)

• x – Data tensor x (input)

• yDesc – Tensor descriptor for data tensor y (input)

• y – Data tensor y (Output)

• reserveSpace – Pointer to memory allocated for executing forward dropout, expecting reserveSpace unchanged before next call of miopenDropoutBackward (Output)

• reserveSpaceSizeInBytes – Number of bytes of reservespace required for executing forward dropout (input)

Returns:

miopenStatus_t

Return value description:

• miopenStatusSuccess - No errors.

• miopenStatusBadParm - Incorrect parameter detected. Check if the following conditions are met:

  • Input/Output dimension/element size/datatype does not match.

  • Tensor dimension is not in the range of 1D to 5D.

  • Noise shape is unsupported.

  • Dropout rate is not in the range of (0, 1].

  • Insufficient state size for parallel PRNG.

  • Insufficient reservespace size.

  • Memory required by dropout forward configs exceeds GPU memory range.

miopenDropoutBackward

miopenStatus_t miopenDropoutBackward(miopenHandle_t handle, const miopenDropoutDescriptor_t dropoutDesc, const miopenTensorDescriptor_t noise_shape, const miopenTensorDescriptor_t dyDesc, const void *dy, const miopenTensorDescriptor_t dxDesc, void *dx, void *reserveSpace, size_t reserveSpaceSizeInBytes)

Execute backward dropout operation.

Interface for executing the backward pass on a Dropout.

Parameters:

• handle – MIOpen handle (input)

• dropoutDesc – Dropout layer descriptor (input)

• noise_shape – Tensor descriptor for noise shape (input placeholder, currently not enabled)

• dyDesc – Tensor descriptor for data delta tensor dy (input)

• dy – Data delta tensor dy (input)

• dxDesc – Tensor descriptor for data delta tensor dx (input)

• dx – Data delta tensor dx (Output)

• reserveSpace – Pointer to memory allocated for executing backward dropout, expecting reserveSpace unchanged after previous call of miopenDropoutForward (input)

• reserveSpaceSizeInBytes – Number of bytes of reservespace required for executing backward dropout (input)

Returns:

miopenStatus_t

Return value description:

• miopenStatusSuccess - No errors.

• miopenStatusBadParm - Incorrect parameter detected. Check if the following conditions are met:

  • Input/Output dimension/element size/datatype does not match.

  • Tensor dimension is not in the range of 1D to 5D.

  • Dropout rate is not in the range of (0, 1].

  • Insufficient reservespace size.

  • Memory required by dropout backward configs exceeds GPU memory range.