hipBLASLt Datatypes Reference

hipblasLtEpilogue_t

enum hipblasLtEpilogue_t

Specify the enum type to set the postprocessing options for the epilogue.

Values:

enumerator HIPBLASLT_EPILOGUE_DEFAULT

No special postprocessing, just scale and quantize the results if necessary.

enumerator HIPBLASLT_EPILOGUE_RELU

Apply ReLU point-wise transform to the results:(x:=max(x, 0))

enumerator HIPBLASLT_EPILOGUE_BIAS

Apply (broadcast) bias from the bias vector. Bias vector length must match matrix D rows, and it must be packed (such as stride between vector elements is 1). Bias vector is broadcast to all columns and added before applying the final postprocessing.

enumerator HIPBLASLT_EPILOGUE_RELU_BIAS

Apply bias and then ReLU transform.

enumerator HIPBLASLT_EPILOGUE_GELU

Apply GELU point-wise transform to the results (x:=GELU(x)).

enumerator HIPBLASLT_EPILOGUE_GELU_BIAS

Apply Bias and then GELU transform.

enumerator HIPBLASLT_EPILOGUE_GELU_AUX

Output GEMM results before applying GELU transform.

enumerator HIPBLASLT_EPILOGUE_GELU_AUX_BIAS

Output GEMM results after applying bias but before applying GELU transform.

enumerator HIPBLASLT_EPILOGUE_DGELU

Apply gradient GELU transform. Requires additional aux input.

enumerator HIPBLASLT_EPILOGUE_DGELU_BGRAD

Apply gradient GELU transform and bias gradient to the results. Requires additional aux input.

enumerator HIPBLASLT_EPILOGUE_BGRADA

Apply bias gradient to A and output gemm result.

enumerator HIPBLASLT_EPILOGUE_BGRADB

Apply bias gradient to B and output gemm result.

hipblasLtHandle_t

typedef void *hipblasLtHandle_t

Handle to the hipBLASLt library context queue.

The hipblasLtHandle_t type is a pointer type to an opaque structure holding the hipBLASLt library context. Use the following functions to manipulate this library context:

hipblasLtCreate(): To initialize the hipBLASLt library context and return a handle to an opaque structure holding the hipBLASLt library context. hipblasLtDestroy(): To destroy a previously created hipBLASLt library context descriptor and release the resources.

hipblasLtMatmulAlgo_t

struct hipblasLtMatmulAlgo_t

Description of the matrix multiplication algorithm.

This is an opaque structure holding the description of the matrix multiplication algorithm. This structure can be trivially serialized and later restored for use with the same version of hipBLASLt library to save on selecting the right configuration again.

hipblasLtMatmulDesc_t

typedef hipblasLtMatmulDescOpaque_t *hipblasLtMatmulDesc_t

Descriptor of the matrix multiplication operation.

This is a pointer to an opaque structure holding the description of the matrix multiplication operation hipblasLtMatmul(). Use the following functions to manipulate this descriptor: hipblasLtMatmulDescCreate(): To create one instance of the descriptor. hipblasLtMatmulDescDestroy(): To destroy a previously created descriptor and release the resources.

hipblasLtMatmulDescAttributes_t

enum hipblasLtMatmulDescAttributes_t

Specify the attributes that define the specifics of the matrix multiply operation.

Values:

enumerator HIPBLASLT_MATMUL_DESC_TRANSA

Specifies the type of transformation operation that should be performed on matrix A. Default value is HIPBLAS_OP_N (for example, non-transpose operation). See hipblasOperation_t. Data Type:int32_t

enumerator HIPBLASLT_MATMUL_DESC_TRANSB

Specifies the type of transformation operation that should be performed on matrix B. Default value is HIPBLAS_OP_N (for example, non-transpose operation). See hipblasOperation_t. Data Type:int32_t

enumerator HIPBLASLT_MATMUL_DESC_EPILOGUE

Epilogue function. See hipblasLtEpilogue_t. Default value is: HIPBLASLT_EPILOGUE_DEFAULT. Data Type: uint32_t

enumerator HIPBLASLT_MATMUL_DESC_BIAS_POINTER

Bias or Bias gradient vector pointer in the device memory. Data Type:void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_BIAS_DATA_TYPE

Type of the bias vector in the device memory. Can be set same as D matrix type or Scale type. Bias case: see HIPBLASLT_EPILOGUE_BIAS. Data Type:int32_t based on hipblasDatatype_t

enumerator HIPBLASLT_MATMUL_DESC_EPILOGUE_AUX_POINTER

Epilogue auxiliary buffer pointer in the device memory. Data Type:void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_EPILOGUE_AUX_LD

The leading dimension of the epilogue auxiliary buffer pointer in the device memory. Data Type:int64_t

enumerator HIPBLASLT_MATMUL_DESC_EPILOGUE_AUX_BATCH_STRIDE

The batch stride of the epilogue auxiliary buffer pointer in the device memory. Data Type:int64_t

enumerator HIPBLASLT_MATMUL_DESC_D_SCALE_VECTOR_POINTER

D scale vector length must match matrix D rows. It must match Scale data type. D scale vector is broadcast to all columns and multipied after final postprocssion. Data Type: void* /const void*

enumerator HIPBLASLT_MATMUL_DESC_MAX

hipblasLtMatmulHeuristicResult_t

struct hipblasLtMatmulHeuristicResult_t

Description of the matrix multiplication algorithm.

This is a descriptor that holds the configured matrix multiplication algorithm descriptor and its runtime properties. This structure can be trivially serialized and later restored for use with the same version of hipBLASLt library to save on selecting the right configuration again.

hipblasLtMatmulPreference_t

typedef hipblasLtMatmulPreferenceOpaque_t *hipblasLtMatmulPreference_t

Descriptor of the matrix multiplication preference.

This is a pointer to an opaque structure holding the description of the preferences for hipblasLtMatmulAlgoGetHeuristic() configuration. Use the following functions to manipulate this descriptor: hipblasLtMatmulPreferenceCreate(): To create one instance of the descriptor. hipblasLtMatmulPreferenceDestroy(): To destroy a previously created descriptor and release the resources.

hipblasLtMatmulPreferenceAttributes_t

enum hipblasLtMatmulPreferenceAttributes_t

It is an enumerated type used to apply algorithm search preferences while fine-tuning the heuristic function.

Values:

enumerator HIPBLASLT_MATMUL_PREF_SEARCH_MODE

Search mode. Data Type: uint32_t

enumerator HIPBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES

Maximum allowed workspace memory. Default is 0 (no workspace memory allowed). Data Type: uint64_t

enumerator HIPBLASLT_MATMUL_PREF_MAX

hipblasLtMatrixLayout_t

typedef hipblasLtMatrixLayoutOpaque_t *hipblasLtMatrixLayout_t

Descriptor of the matrix layout.

This is a pointer to an opaque structure holding the description of a matrix layout. Use the following functions to manipulate this descriptor: hipblasLtMatrixLayoutCreate(): To create one instance of the descriptor. hipblasLtMatrixLayoutDestroy(): To destroy a previously created descriptor and release the resources.

hipblasLtMatrixLayoutAttribute_t

enum hipblasLtMatrixLayoutAttribute_t

Specify the attributes that define the details of the matrix.

Values:

enumerator HIPBLASLT_MATRIX_LAYOUT_BATCH_COUNT

Number of batch of this matrix. Default value is 1. Data Type: int32_t

enumerator HIPBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET

Stride (in elements) to the next matrix for the strided batch operation. Default value is 0. Data Type: int64_t