hipBLASLt datatypes reference

Contents

hipBLASLt datatypes reference#

hipblasLtEpilogue_t#

enum hipblasLtEpilogue_t#

Specify the enum type to set the postprocessing options for the epilogue.

Values:

enumerator HIPBLASLT_EPILOGUE_DEFAULT#

No special postprocessing, just scale and quantize the results if necessary.

enumerator HIPBLASLT_EPILOGUE_RELU#

Apply ReLU point-wise transform to the results:(x:=max(x, 0))

enumerator HIPBLASLT_EPILOGUE_BIAS#

Apply (broadcast) bias from the bias vector. Bias vector length must match matrix D rows, and it must be packed (such as stride between vector elements is 1). Bias vector is broadcast to all columns and added before applying the final postprocessing.

enumerator HIPBLASLT_EPILOGUE_RELU_BIAS#

Apply bias and then ReLU transform.

enumerator HIPBLASLT_EPILOGUE_GELU#

Apply GELU point-wise transform to the results (x:=GELU(x)).

enumerator HIPBLASLT_EPILOGUE_GELU_BIAS#

Apply Bias and then GELU transform.

enumerator HIPBLASLT_EPILOGUE_GELU_AUX#

Output GEMM results before applying GELU transform.

enumerator HIPBLASLT_EPILOGUE_GELU_AUX_BIAS#

Output GEMM results after applying bias but before applying GELU transform.

enumerator HIPBLASLT_EPILOGUE_DGELU#

Apply gradient GELU transform. Requires additional aux input.

enumerator HIPBLASLT_EPILOGUE_DGELU_BGRAD#

Apply gradient GELU transform and bias gradient to the results. Requires additional aux input.

enumerator HIPBLASLT_EPILOGUE_BGRADA#

Apply bias gradient to A and output gemm result.

enumerator HIPBLASLT_EPILOGUE_BGRADB#

Apply bias gradient to B and output gemm result.

hipblasLtPointerMode_t#

enum hipblasLtPointerMode_t#

Pointer mode to use for alpha.

Values:

enumerator HIPBLASLT_POINTER_MODE_HOST#
enumerator HIPBLASLT_POINTER_MODE_DEVICE#

targets host memory

enumerator HIPBLASLT_POINTER_MODE_ALPHA_DEVICE_VECTOR_BETA_HOST#

targets device memory

hipblasLtHandle_t#

typedef void *hipblasLtHandle_t#

Handle to the hipBLASLt library context queue.

The hipblasLtHandle_t type is a pointer type to an opaque structure holding the hipBLASLt library context. Use the following functions to manipulate this library context:

hipblasLtCreate(): To initialize the hipBLASLt library context and return a handle to an opaque structure holding the hipBLASLt library context. hipblasLtDestroy(): To destroy a previously created hipBLASLt library context descriptor and release the resources.

hipblasLtMatmulAlgo_t#

struct hipblasLtMatmulAlgo_t#

Description of the matrix multiplication algorithm.

This is an opaque structure holding the description of the matrix multiplication algorithm. This structure can be trivially serialized and later restored for use with the same version of hipBLASLt library to save on selecting the right configuration again.

hipblasLtMatmulDesc_t#

typedef hipblasLtMatmulDescOpaque_t *hipblasLtMatmulDesc_t#

Descriptor of the matrix multiplication operation.

This is a pointer to an opaque structure holding the description of the matrix multiplication operation hipblasLtMatmul(). Use the following functions to manipulate this descriptor: hipblasLtMatmulDescCreate(): To create one instance of the descriptor. hipblasLtMatmulDescDestroy(): To destroy a previously created descriptor and release the resources.

hipblasLtMatmulDescAttributes_t#

enum hipblasLtMatmulDescAttributes_t#

Specify the attributes that define the specifics of the matrix multiply operation.

Values:

enumerator HIPBLASLT_MATMUL_DESC_TRANSA#

Specifies the type of transformation operation that should be performed on matrix A. Default value is HIPBLAS_OP_N (for example, non-transpose operation). See hipblasOperation_t. Data Type:int32_t

enumerator HIPBLASLT_MATMUL_DESC_TRANSB#

Specifies the type of transformation operation that should be performed on matrix B. Default value is HIPBLAS_OP_N (for example, non-transpose operation). See hipblasOperation_t. Data Type:int32_t

enumerator HIPBLASLT_MATMUL_DESC_EPILOGUE#

Epilogue function. See hipblasLtEpilogue_t. Default value is: HIPBLASLT_EPILOGUE_DEFAULT. Data Type: uint32_t

enumerator HIPBLASLT_MATMUL_DESC_BIAS_POINTER#

Bias or Bias gradient vector pointer in the device memory. Data Type:void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_BIAS_DATA_TYPE#

Type of the bias vector in the device memory. Can be set same as D matrix type or Scale type. Bias case: see HIPBLASLT_EPILOGUE_BIAS. Data Type:int32_t based on hipDataType

enumerator HIPBLASLT_MATMUL_DESC_A_SCALE_POINTER#

Device pointer to the scale factor value that converts data in matrix A to the compute data type range. The scaling factor must have the same type as the compute type. If not specified, or set to NULL, the scaling factor is assumed to be 1. If set for an unsupported matrix data, scale, and compute type combination, calling hipblasLtMatmul() will return HIPBLAS_INVALID_VALUE. Default value: NULL Data Type: void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_B_SCALE_POINTER#

Equivalent to HIPBLASLT_MATMUL_DESC_A_SCALE_POINTER for matrix B. Default value: NULL Type: void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_C_SCALE_POINTER#

Equivalent to HIPBLASLT_MATMUL_DESC_A_SCALE_POINTER for matrix C. Default value: NULL Type: void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_D_SCALE_POINTER#

Equivalent to HIPBLASLT_MATMUL_DESC_A_SCALE_POINTER for matrix D. Default value: NULL Type: void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_EPILOGUE_AUX_SCALE_POINTER#

Equivalent to HIPBLASLT_MATMUL_DESC_A_SCALE_POINTER for matrix AUX. Default value: NULL Type: void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_EPILOGUE_AUX_POINTER#

Epilogue auxiliary buffer pointer in the device memory. Data Type:void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_EPILOGUE_AUX_LD#

The leading dimension of the epilogue auxiliary buffer pointer in the device memory. Data Type:int64_t

enumerator HIPBLASLT_MATMUL_DESC_EPILOGUE_AUX_BATCH_STRIDE#

The batch stride of the epilogue auxiliary buffer pointer in the device memory. Data Type:int64_t

enumerator HIPBLASLT_MATMUL_DESC_POINTER_MODE#

Specifies alpha and beta are passed by reference, whether they are scalars on the host or on the device, or device vectors. Default value is: HIPBLASLT_POINTER_MODE_HOST (i.e., on the host). Data Type: int32_t based on hipblasLtPointerMode_t

enumerator HIPBLASLT_MATMUL_DESC_AMAX_D_POINTER#

Device pointer to the memory location that on completion will be set to the maximum of absolute values in the output matrix. Data Type:void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_COMPUTE_INPUT_TYPE_A_EXT#

Compute input A types. Defines the data type used for the input A of matrix multiply.

enumerator HIPBLASLT_MATMUL_DESC_COMPUTE_INPUT_TYPE_B_EXT#

Compute input B types. Defines the data type used for the input B of matrix multiply.

enumerator HIPBLASLT_MATMUL_DESC_A_SCALE_POINTER_VEC_EXT#

Equivalent to HIPBLASLT_MATMUL_DESC_A_SCALE_POINTER but in vector. Default value: NULL Type: void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_B_SCALE_POINTER_VEC_EXT#

Equivalent to HIPBLASLT_MATMUL_DESC_B_SCALE_POINTER but in vector. Default value: NULL Type: void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_MAX#

hipblasLtMatmulHeuristicResult_t#

struct hipblasLtMatmulHeuristicResult_t#

Description of the matrix multiplication algorithm.

This is a descriptor that holds the configured matrix multiplication algorithm descriptor and its runtime properties. This structure can be trivially serialized and later restored for use with the same version of hipBLASLt library to save on selecting the right configuration again.

Param algo:

hipblasLtMatmulAlgo_t struct

Param workspaceSize:

Actual size of workspace memory required

Param state:

Result status. Other fields are valid only if, after call to hipblasLtMatmulAlgoGetHeuristic(), this member is set to HIPBLAS_STATUS_SUCCESS

hipblasLtMatmulPreference_t#

typedef hipblasLtMatmulPreferenceOpaque_t *hipblasLtMatmulPreference_t#

Descriptor of the matrix multiplication preference.

This is a pointer to an opaque structure holding the description of the preferences for hipblasLtMatmulAlgoGetHeuristic() configuration. Use the following functions to manipulate this descriptor: hipblasLtMatmulPreferenceCreate(): To create one instance of the descriptor. hipblasLtMatmulPreferenceDestroy(): To destroy a previously created descriptor and release the resources.

hipblasLtMatmulPreferenceAttributes_t#

enum hipblasLtMatmulPreferenceAttributes_t#

It is an enumerated type used to apply algorithm search preferences while fine-tuning the heuristic function.

Values:

enumerator HIPBLASLT_MATMUL_PREF_SEARCH_MODE#

Search mode. Data Type: uint32_t

enumerator HIPBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES#

Maximum allowed workspace memory. Default is 0 (no workspace memory allowed). Data Type: uint64_t

enumerator HIPBLASLT_MATMUL_PREF_MAX#

hipblasLtMatrixLayout_t#

typedef hipblasLtMatrixLayoutOpaque_t *hipblasLtMatrixLayout_t#

Descriptor of the matrix layout.

This is a pointer to an opaque structure holding the description of a matrix layout. Use the following functions to manipulate this descriptor: hipblasLtMatrixLayoutCreate(): To create one instance of the descriptor. hipblasLtMatrixLayoutDestroy(): To destroy a previously created descriptor and release the resources.

hipblasLtMatrixLayoutAttribute_t#

enum hipblasLtMatrixLayoutAttribute_t#

Specify the attributes that define the details of the matrix.

Values:

enumerator HIPBLASLT_MATRIX_LAYOUT_BATCH_COUNT#

Number of batch of this matrix. Default value is 1. Data Type: int32_t

enumerator HIPBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET#

Stride (in elements) to the next matrix for the strided batch operation. Default value is 0. Data Type: int64_t

enumerator HIPBLASLT_MATRIX_LAYOUT_TYPE#

Data type, see hipDataType.

uint32_t

enumerator HIPBLASLT_MATRIX_LAYOUT_ORDER#

Memory order of the data, see hipblasLtOrder_t.

int32_t, default: HIPBLASLT_ORDER_COL

enumerator HIPBLASLT_MATRIX_LAYOUT_ROWS#

Number of rows.

Usually only values that can be expressed as int32_t are supported.

uint64_t

enumerator HIPBLASLT_MATRIX_LAYOUT_COLS#

Number of columns.

Usually only values that can be expressed as int32_t are supported.

uint64_t

enumerator HIPBLASLT_MATRIX_LAYOUT_LD#

Matrix leading dimension.

For HIPBLASLT_ORDER_COL this is stride (in elements) of matrix column, for more details and documentation for other memory orders see documentation for hipblasLtOrder_t values.

Currently only non-negative values are supported, must be large enough so that matrix memory locations are not overlapping (e.g. greater or equal to HIPBLASLT_MATRIX_LAYOUT_ROWS in case of HIPBLASLT_ORDER_COL).

int64_t;

hipblasLtMatrixTransformDesc_t#

typedef hipblasLtMatrixTransformDescOpaque_t *hipblasLtMatrixTransformDesc_t#

Opaque descriptor for hipblasLtMatrixTransform() operation details.

The hipblasLtMatrixTransformDesc_t is a pointer to an opaque structure holding the description of a matrix transformation operation. hipblasLtMatrixTransformDescCreate(): To create one instance of the descriptor. hipblasLtMatrixTransformDescDestroy(): To destroy a previously created descriptor and release the resources.

Contents