API Reference Guide

3. API Reference Guide#

3.1. Introduction#

hipSPARSELt is a library that contains basic linear algebra subroutines for sparse matrices written in HIP for GPU devices. It is designed to be used from C and C++ code. The functionality of hipSPARSELt is organized in the following categories:

Library Management Functions describe functions that provide the library handle.
Matrix Descriptor Functions describe fuctions that used to define sparse and dense matrix.
Matmul Descriptor Functions describe fuctions that used to define how to do the matrix multiply.
Matmul Algorithm Functions describe functions that provide algortithms for doing the matrix multiply.
Matmul Functions describe operations that provide multiply of sparse matrices.
Helper Functions describe available helper functions that are required for subsequent library calls.

The code is open and hosted here: ROCmSoftwarePlatform/hipSPARSELt

hipSPARSELt SPARSE library supports AMD’s sparse matrix core technique on AMD RDNA/CDNA GPUs that speeds up structured sparsity computation and improves AI framework and Inference engine specifically. This is also a marshalling library, with multiple supported backends. It sits between the application and a ‘worker’ SPARSE library, marshalling inputs into the backend library and marshalling results back to the application. hipSPARSELt exports an interface that does not require the client to change, regardless of the chosen backend. Currently, hipSPARSELt support rocSPARSELt (included in hipSPARSELt) and cuSPARSELt as backends.

3.2. hipSPARSELt API#

3.2.1. Device and Stream Management#

hipSetDevice() and hipGetDevice() are HIP device management APIs. They are NOT part of the hipSPARSELt API.

3.2.2. Asynchronous Execution#

All hipSPARSELt library functions, unless otherwise stated, are non blocking and executed asynchronously with respect to the host. They may return before the actual computation has finished. To force synchronization, hipDeviceSynchronize() or hipStreamSynchronize() can be used. This will ensure that all previously executed hipSPARSELt functions on the device / this particular stream have completed.

3.2.3. HIP Device Management#

Before a HIP kernel invocation, users need to call hipSetDevice() to set a device, e.g. device 1. If users do not explicitly call it, the system by default sets it as device 0. Unless users explicitly call hipSetDevice() to set to another device, their HIP kernels are always launched on device 0.

The above is a HIP (and CUDA) device management approach and has nothing to do with hipSPARSELt. hipSPARSELt honors the approach above and assumes users have already set the device before a hipSPARSELt routine call.

Once users set the device, they create a handle with hipsparseLtInit().

Subsequent hipSPARSELt routines take this handle as an input parameter. hipSPARSELt ONLY queries (by hipGetDevice()) the user’s device; hipSPARSELt does NOT set the device for users. If hipSPARSELt does not see a valid device, it returns an error message. It is the users’ responsibility to provide a valid device to hipSPARSELt and ensure the device safety.

Users CANNOT switch devices between hipsparseLtInit() and hipsparseLtDestroy(). If users want to change device, they must destroy the current handle and create another hipSPARSELt handle.

3.2.4. HIP Stream Management#

HIP kernels are always launched in a queue (also known as stream).

If users do not explicitly specify a stream, the system provides a default stream, maintained by the system. Users cannot create or destroy the default stream. However, users can freely create new streams (with hipStreamCreate()) and bind it to the hipSPARSELt operations, such as hipsparseLtSpMMAPrune() and hipsparseLtMatmul(). HIP kernels are invoked in hipSPARSELt routines. If users create a stream, they are responsible for destroying it.

3.2.5. Multiple Streams and Multiple Devices#

If the system under test has multiple HIP devices, users can run multiple hipSPARSELt handles concurrently, but can NOT run a single hipSPARSELt handle on different discrete devices. Each handle is associated with a particular singular device, and a new handle should be created for each additional device.

3.2.6. Storage Formats#

3.2.6.1. Structured sparsity storage format#

The Structured sparsity storage format represents a \(m \times n\) matrix by

m	number of rows (integer).
n	number of columns (integer).
sparsity	50%, ratio of `nnz` elemetns in every 2:1 (int) or 4:2 (others) element along the row. 4:2 means every 4 continuous elements will only have 2 `nnz` elements.
compresed_matrix	matrix of `nnz` elements containing the data
metadata	matrix of `nnz` elements containing the element indices in every 4:2 or 2:1 array along the row. contents or structure of metadata is dependent on the chosen solution by backend implementation.

Consider the following \(4 \times 4\) matrix and the structured sparsity structures, with \(m = 4, n = 4\):

\[\begin{split}A = \begin{pmatrix} 1.0 & 2.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 3.0 & 4.0 \\ 0.0 & 6.0 & 7.0 & 0.0 \\ 0.0 & 6.0 & 0.0 & 8.0 \\ \end{pmatrix}\end{split}\]

where

\[\begin{split}Compresed A = \begin{pmatrix} 1.0 & 2.0 \\ 3.0 & 4.0 \\ 6.0 & 7.0 \\ 6.0 & 8.0 \\ \end{pmatrix} metadata = \begin{pmatrix} 0 & 1 \\ 2 & 3 \\ 1 & 2 \\ 1 & 3 \\ \end{pmatrix}\end{split}\]

4. Using hipSPARSELt API#

4.1. Types#

4.1.1. hipsparseLtHandle_t#

struct hipsparseLtHandle_t#

Handle to the hipSPARSELt library context queue.

The hipSPARSELt handle is a structure holding the hipSPARSELt library context. It must be initialized using hipsparseLtInit and the returned handle must be passed to all subsequent library function calls. It should be destroyed at the end using hipsparseLtDestroy.

4.1.2. hipsparseLtMatDescriptor_t#

struct hipsparseLtMatDescriptor_t#

Descriptor of the matrix.

The hipSPARSELt matrix descriptor is a structure holding all properties of a matrix. It must be initialized using hipsparseLtDenseDescriptorInit and the returned descriptor must be passed to all subsequent library calls that involve the matrix. It should be destroyed at the end using hipsparseLtMatDescriptorDestroy.

4.1.3. hipsparseLtMatmulDescriptor_t#

struct hipsparseLtMatmulDescriptor_t#

Descriptor of the matrix multiplication operation.

The hipSPARSELt matrix multiplication descriptor is a structure holding the description of the matrix multiplication operation. It is initialized with hipsparseLtMatmulDescriptorInit function.

4.1.4. hipsparseLtMatmulAlgSelection_t#

struct hipsparseLtMatmulAlgSelection_t#

Descriptor of the matrix multiplication algorithm.

It is initialized with hipsparseLtMatmulAlgSelectionInit function.

4.1.5. hipsparseLtMatmulPlan_t#

struct hipsparseLtMatmulPlan_t#

Descriptor of the matrix multiplication execution plan.

The hipSPARSELt matrix multiplication execution plan descriptor is a structure holding all the information necessary to execute the hipsparseLtMatmul operation. It is initialized and destroyed with hipsparseLtMatmulPlanInit and hipsparseLtMatmulPlanDestroy functions respectively.

4.1.6. hipsparseLtDatatype_t#

enum hipsparseLtDatatype_t#

List of hipsparselt data types.

Indicates the precision width of data stored in a hipsparselt type. Should use hipDatatype_t instead in the furture.

Values:

enumerator HIPSPARSELT_R_16F#: 16 bit floating point, real

enumerator HIPSPARSELT_R_32F#: 32 bit floating point, real

enumerator HIPSPARSELT_R_8I#: 8 bit signed integer, real

enumerator HIPSPARSELT_R_16BF#: 16 bit bfloat, real

enumerator HIPSPARSELT_R_8F#: 8 bit floating point, real

enumerator HIPSPARSELT_R_8BF#: 8 bit bfloat, real

4.1.7. hipsparseLtSparsity_t#

enum hipsparseLtSparsity_t#

Specify the sparsity of the structured matrix.

The enumerator specifies the sparsity ratio of the structured matrix as sparsity = nnz / total elements The sparsity property is used in the hipsparseLtStructuredDescriptorInit function.

Values:

enumerator HIPSPARSELT_SPARSITY_50_PERCENT#: 50% sparsity ratio - 1:2 for tf32 and float, 2:4 for half, bfloat16, int

4.1.8. hipsparseLtMatDescAttribute_t#

enum hipsparseLtMatDescAttribute_t#

Specify the additional attributes of a matrix descriptor.

The hipsparseLtMatDescAttribute_t is used in the hipsparseLtMatDescSetAttribute and hipsparseLtMatDescGetAttribute functions

Values:

enumerator HIPSPARSELT_MAT_NUM_BATCHES#: number of matrices in a batch. READ/WRITE

enumerator HIPSPARSELT_MAT_BATCH_STRIDE#: stride between consecutive matrices in a batch expressed in terms of matrix elements. READ/WRITE

4.1.9. hipsparseLtComputetype_t#

enum hipsparseLtComputetype_t#

Specify the compute precision modes of the matrix.

Values:

enumerator HIPSPARSELT_COMPUTE_16F#: 16-bit floating-point precision. CUDA backend only.

enumerator HIPSPARSELT_COMPUTE_32I#: 32-bit integer precision

enumerator HIPSPARSELT_COMPUTE_32F#: 32-bit floating-point precision. HIP backend only.

enumerator HIPSPARSELT_COMPUTE_TF32#: 32-bit floating point value are rounded to TF32 before the computation. CUDA backend only.

enumerator HIPSPARSELT_COMPUTE_TF32_FAST#: 32-bit floating point value are truncated to TF32 before the computation. CUDA backend only.

4.1.10. hipsparseLtMatmulDescAttribute_t#

enum hipsparseLtMatmulDescAttribute_t#

Specify the additional attributes of a matrix multiplication descriptor.

The hipsparseLtMatmulDescAttribute_t is used in the hipsparseLtMatmulDescSetAttribute and hipsparseLtMatmulDescGetAttribute functions

Values:

enumerator HIPSPARSELT_MATMUL_ACTIVATION_RELU#: ReLU activation function.

enumerator HIPSPARSELT_MATMUL_ACTIVATION_RELU_UPPERBOUND#: Upper bound of the ReLU activation function.

enumerator HIPSPARSELT_MATMUL_ACTIVATION_RELU_THRESHOLD#: Lower threshold of the ReLU activation function.

enumerator HIPSPARSELT_MATMUL_ACTIVATION_GELU#: GeLU activation function.

enumerator HIPSPARSELT_MATMUL_ACTIVATION_GELU_SCALING#: Scaling coefficient for the GeLU activation function. It implies gelu is endable

enumerator HIPSPARSELT_MATMUL_ALPHA_VECTOR_SCALING#: Enable/Disable alpha vector (per-channel) scaling

enumerator HIPSPARSELT_MATMUL_BETA_VECTOR_SCALING#: Enable/Disable beta vector (per-channel) scaling

enumerator HIPSPARSELT_MATMUL_BIAS_STRIDE#: Bias pointer. The bias vector size must equal to the number of rows of the output matrix (D).

enumerator HIPSPARSELT_MATMUL_BIAS_POINTER#: Bias stride between consecutive bias vectors. 0 means broadcast the first bias vector.

enumerator HIPSPARSELT_MATMUL_ACTIVATION_ABS#: ABS activation function. HIP backend only

enumerator HIPSPARSELT_MATMUL_ACTIVATION_LEAKYRELU#: LeakyReLU activation function. HIP backend only

enumerator HIPSPARSELT_MATMUL_ACTIVATION_LEAKYRELU_ALPHA#: Alpha value of the LeakyReLU activation function. HIP backend only

enumerator HIPSPARSELT_MATMUL_ACTIVATION_SIGMOID#: Sigmoid activation function. HIP backend only

enumerator HIPSPARSELT_MATMUL_ACTIVATION_TANH#: Tanh activation function. HIP backend only

enumerator HIPSPARSELT_MATMUL_ACTIVATION_TANH_ALPHA#: Alpha value of the Tanh activation function. HIP backend only

enumerator HIPSPARSELT_MATMUL_ACTIVATION_TANH_BETA#: Beta value of the Tanh activation function. HIP backend only

enumerator HIPSPARSELT_MATMUL_BIAS_TYPE#: Precision of Bias. HIP backend only, When Input’s datatype is FP16 - Bias type can be FP16 or FP32. (default FP16) When Input’s datatype is BF16 - Bias type can be BF16 or FP32. (default BF16) In other cases - Bias type is FP32.

4.1.11. hipsparseLtMatmulAlg_t#

enum hipsparseLtMatmulAlg_t#

Specify the algorithm for matrix-matrix multiplication.

The hipsparseLtMatmulAlg_t is used in the hipsparseLtMatmulAlgSelectionInit function.

Values:

enumerator HIPSPARSELT_MATMUL_ALG_DEFAULT#

4.1.12. hipsparseLtPruneAlg_t#

enum hipsparseLtMatmulAlgAttribute_t#

Specify the matrix multiplication algorithm attributes.

The hipsparseLtMatmulAlgAttribute_t is used in the hipsparseLtMatmulAlgGetAttribute and hipsparseLtMatmulAlgSetAttribute functions.

Values:

enumerator HIPSPARSELT_MATMUL_ALG_CONFIG_ID#

enumerator HIPSPARSELT_MATMUL_ALG_CONFIG_MAX_ID#

enumerator HIPSPARSELT_MATMUL_SEARCH_ITERATIONS#

enumerator HIPSPARSELT_MATMUL_SPLIT_K#

enumerator HIPSPARSELT_MATMUL_SPLIT_K_MODE#

enumerator HIPSPARSELT_MATMUL_SPLIT_K_BUFFERS#

4.1.13. hipsparseLtMatmulAlgAttribute_t#

enum hipsparseLtPruneAlg_t#

Specify the pruning algorithm to apply to the structured matrix before the compression.

The hipsparseLtPruneAlg_t is used in the hipsparseLtSpMMAPrune and hipsparseLtSpMMAPrune2 function.

Values:

enumerator HIPSPARSELT_PRUNE_SPMMA_TILE#: Zero out eight elements in a 4x4 tile, nonzero elements have the maximum L1-norm value in all combinations in the tile. Exactly two elements in each row and column.

enumerator HIPSPARSELT_PRUNE_SPMMA_STRIP#: Zero out two elements in a 1x4 strip, nonzero elements have the maximum L1-norm value in all combinations in the strip.

4.1.14. hipsparseLtSplitKMode_t#

enum hipsparseLtSplitKMode_t#

Specify the split k mode value.

The hipsparseLtSplitKMode_t is used by HIPSPARSELT_MATMUL_SPLIT_K_MODE attribute in hipsparseLtMatmulAlgAttribute_t.

Values:

enumerator HIPSPARSELT_SPLIT_K_MODE_ONE_KERNEL#: Use the same SP-MM kernel to do the final reduction

enumerator HIPSPARSELT_SPLIT_K_MODE_TWO_KERNELS#: Use another kernel to do the final reduction

4.2. Exported Sparse Functions#

4.2.1. Library Management Functions#

Function name

hipsparseLtInit()

hipsparseLtDestroy()

hipsparseLtGetVersion()

hipsparseLtGetProperty()

4.2.2. Matrix Descriptor Functions#

Function name	int (CUDA)	single	half	bfloat16	int8
`hipsparseLtDenseDescriptorInit()`	x	x	x	x	x
`hipsparseLtStructuredDescriptorInit()`	x	x	x	x	x

Function name

hipsparseLtMatDescriptorDestroy()

hipsparseLtMatDescSetAttribute()

hipsparseLtMatDescGetAttribute()

4.2.3. Matmul Descriptor Functions#

Function name	TF32 (CUDA)	TF32 Fast (CUDA)	single (HIP)	half (CUDA)	int
`hipsparseLtMatmulDescriptorInit()`	x	x	x	x	x

Function name

hipsparseLtMatmulDescriptorInit()

hipsparseLtMatmulDescSetAttribute()

hipsparseLtMatmulDescGetAttribute()

4.2.4. Matmul Algorithm Functions#

Function name

hipsparseLtMatmulAlgSelectionInit()

hipsparseLtMatmulAlgSetAttribute()

hipsparseLtMatmulAlgGetAttribute()

4.2.5. Matmul Functions#

Function name

hipsparseLtMatmulGetWorkspace()

hipsparseLtMatmulPlanInit()

hipsparseLtMatmulPlanDestroy()

hipsparseLtMatmul()

hipsparseLtMatmulSearch()

4.2.6. Helper Functions#

Function name

hipsparseLtSpMMAPrune()

hipsparseLtSpMMAPruneCheck()

hipsparseLtSpMMAPrune2()

hipsparseLtSpMMAPruneCheck2()

hipsparseLtSpMMACompressedSize()

hipsparseLtSpMMACompress()

hipsparseLtSpMMACompressedSize2()

hipsparseLtSpMMACompress2()

4.3. Asynchronous API#

Except a functions having memory allocation inside preventing asynchronicity, all hipSPARSELt functions are configured to operate in non-blocking fashion with respect to CPU, meaning these library functions return immediately.

4.4. Library Management Functions#

This library management describe functions that provide the library handle.

4.4.1. hipsparseLtInit()#

hipsparseStatus_t hipsparseLtInit(hipsparseLtHandle_t *handle)#

Create a hipsparselt handle.

hipsparseLtInit creates the hipSPARSELt library context. It must be initialized before any other hipSPARSELt API function is invoked and must be passed to all subsequent library function calls. The handle should be destroyed at the end using hipsparseLtDestroy_handle().

Parameters:

handle – [out] hipsparselt library handle

Return values:

HIPSPARSE_STATUS_SUCCESS – the initialization succeeded.
HIPSPARSE_STATUS_INVALID_VALUE – handle is invalid.

4.4.2. hipsparseLtDestroy()#

hipsparseStatus_t hipsparseLtDestroy(const hipsparseLtHandle_t *handle)#

Destroy a hipsparselt handle.

hipsparseLtDestroy destroys the hipSPARSELt library context and releases all resources used by the hipSPARSELt library.

Parameters:

handle – [in] hipsparselt library handle

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_NOT_INITIALIZED – handle is invalid.

4.4.3. hipsparseLtGetVersion()#

hipsparseStatus_t hipsparseLtGetVersion(const hipsparseLtHandle_t *handle, int *version)#

Retrive the version number of the hipSPARSELt library.

hipsparseLtGetVersion return the version number of the hipSPARSELt library.

Parameters:

handle – [in] hipsparselt library handle.
version – [out] the version number of the library.

Return values:

HIPSPARSE_STATUS_SUCCESS –
HIPSPARSE_STATUS_INVALID_VALUE – handle is invalid.

4.4.4. hipsparseLtGetProperty()#

hipsparseStatus_t hipsparseLtGetProperty(hipLibraryPropertyType propertyType, int *value)#

Retrive the value of the requested property.

hipsparseLtGetProperty return the value of the requested property.

Parameters:

propertyType – [in] property type. hipLibraryPropertyType (defined in library_types.h)
value – [out] value of the requested property.

Return values:

HIPSPARSE_STATUS_SUCCESS –

4.5. Matrix Descriptor Functions#

The matrix descriptor describe fuctions that used to define sparse and dense matrix

4.5.1. hipsparseLtDenseDescriptorInit()#

hipsparseStatus_t hipsparseLtDenseDescriptorInit(const hipsparseLtHandle_t *handle, hipsparseLtMatDescriptor_t *matDescr, int64_t rows, int64_t cols, int64_t ld, uint32_t alignment, hipsparseLtDatatype_t valueType, hipsparseOrder_t order)#

Create a descriptor for dense matrix.

hipsparseLtDenseDescriptorInit creates a matrix descriptor It initializes It should be destroyed at the end using hipsparseLtMatDescriptorDestroy().

Parameters:

handle – [in] the hipsparselt handle
matDescr – [out] the pointer to the dense matrix descriptor
rows – [in] number of rows
cols – [in] number of columns
ld – [in] leading dimension
alignment – [in] memory alignment in bytes (not used by HIP backend)
valueType – [in] data type of the matrix. see hipsparseLtDatatype_t
order – [in] memory layout. HIPSPARSE_ORDER_COL or HIPSPARSE_ORDER_ROW. (HIP backend only support HIPSPARSE_ORDER_COL.)

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle, descr , rows , cols , ld is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – rows , cols , ld , alignment , valueType or order is not supported.

4.5.2. hipsparseLtStructuredDescriptorInit()#

hipsparseStatus_t hipsparseLtStructuredDescriptorInit(const hipsparseLtHandle_t *handle, hipsparseLtMatDescriptor_t *matDescr, int64_t rows, int64_t cols, int64_t ld, uint32_t alignment, hipsparseLtDatatype_t valueType, hipsparseOrder_t order, hipsparseLtSparsity_t sparsity)#

Create a descriptor for structured matrix.

hipsparseLtStructuredDescriptorInit creates a matrix descriptor It initializes It should be destroyed at the end using hipsparseLtMatDescriptorDestroy().

Parameters:

handle – [in] the hipsparselt handle
matDescr – [out] the pointer to the dense matrix descriptor
rows – [in] number of rows
cols – [in] number of columns
ld – [in] leading dimension
alignment – [in] memory alignment in bytes (not used by HIP backend)
valueType – [in] data type of the matrix. see hipsparseLtDatatype_t
order – [in] memory layout. HIPSPARSE_ORDER_COL or HIPSPARSE_ORDER_ROW. (HIP backend only support HIPSPARSE_ORDER_COL.)
sparsity – [in] matrix sparsity ratio. see hipsparseLtSparsity_t

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle, descr , rows , cols , ld is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – rows , cols , ld , alignment , valueType or order is not supported.

4.5.3. hipsparseLtMatDescriptorDestroy()#

hipsparseStatus_t hipsparseLtMatDescriptorDestroy(const hipsparseLtMatDescriptor_t *matDescr)#

Destroy a matrix descriptor.

hipsparseLtMatDescriptorDestroy destroys a matrix descriptor and releases all resources used by the descriptor

Parameters:

matDescr – [in] the matrix descriptor

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – descr is invalid.

4.5.4. hipsparseLtMatDescSetAttribute()#

hipsparseStatus_t hipsparseLtMatDescSetAttribute(const hipsparseLtHandle_t *handle, hipsparseLtMatDescriptor_t *matDescr, hipsparseLtMatDescAttribute_t matAttribute, const void *data, size_t dataSize)#

Specify the matrix attribute of a matrix descriptor.

hipsparseLtMatDescSetAttribute sets the value of the specified attribute belonging to matrix descr such as number of batches and their stride.

Parameters:

handle – [in] the hipsparselt handle
matDescr – [inout] the matrix descriptor
matAttribute – [in] HIPSPARSELT_MAT_NUM_BATCHES, HIPSPARSELT_MAT_BATCH_STRIDE.
data – [in] pointer to the value to which the specified attribute will be set.
dataSize – [in] size in bytes of the attribute value used for verification.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , matmulDescr , data or dataSize is invalid.

4.5.5. hipsparseLtMatDescGetAttribute()#

hipsparseStatus_t hipsparseLtMatDescGetAttribute(const hipsparseLtHandle_t *handle, const hipsparseLtMatDescriptor_t *matDescr, hipsparseLtMatDescAttribute_t matAttribute, void *data, size_t dataSize)#

Get the matrix type of a matrix descriptor.

hipsparseLtMatDescGetAttribute returns the matrix attribute of a matrix descriptor

Parameters:

handle – [in] the hipsparselt handle
matDescr – [in] the matrix descriptor
matAttribute – [in] HIPSPARSELT_MAT_NUM_BATCHES, HIPSPARSELT_MAT_BATCH_STRIDE.
data – [inout] the memory address containing the attribute value retrieved by this function
dataSize – [in] size in bytes of the attribute value used for verification.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , descr , data or dataSize is invalid.

4.6. Matmul Descriptor Functions#

This matmul descriptor describe fuctions that used to define how to do the matrix multiply.

4.6.1. hipsparseLtMatmulDescriptorInit()#

hipsparseStatus_t hipsparseLtMatmulDescriptorInit(const hipsparseLtHandle_t *handle, hipsparseLtMatmulDescriptor_t *matmulDescr, hipsparseOperation_t opA, hipsparseOperation_t opB, const hipsparseLtMatDescriptor_t *matA, const hipsparseLtMatDescriptor_t *matB, const hipsparseLtMatDescriptor_t *matC, const hipsparseLtMatDescriptor_t *matD, hipsparseLtComputetype_t computeType)#

Initializes the matrix multiplication descriptor.

hipsparseLtMatmulDescriptorInit creates a matrix multiplication descriptor.

Parameters:

handle – [in] the hipsparselt handle
matmulDescr – [inout] the matrix multiplication descriptor
opA – [in] hipsparse operation for Matrix A. HIPSPARSE_OPERATION_NON_TRANSPOSE or HIPSPARSE_OPERATION_TRANSPOSE
opB – [in] hipsparse operation for Matrix B. HIPSPARSE_OPERATION_NON_TRANSPOSE or HIPSPARSE_OPERATION_TRANSPOSE
matA – [in] the matrix descriptor (HIP backend only structured (sparse) matrix matrix)
matB – [in] the matrix descriptor (HIP backend only support dense matrix)
matC – [in] the matrix descriptor (dense matrix)
matD – [in] the matrix descriptor (dense matrix)
computeType – [in] size in bytes of the attribute value used for verification.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , matmulDescr , opA , opB , matA , matB , matC , matD or computeType ,is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – opA , opB or computeType is invalid.

4.6.2. hipsparseLtMatmulDescSetAttribute()#

hipsparseStatus_t hipsparseLtMatmulDescSetAttribute(const hipsparseLtHandle_t *handle, hipsparseLtMatmulDescriptor_t *matmulDescr, hipsparseLtMatmulDescAttribute_t matmulAttribute, const void *data, size_t dataSize)#

Specify the matrix attribute of a matrix descriptor.

hipsparseLtMatmulDescSetAttribute sets the value of the specified attribute belonging to matrix descr such as number of batches and their stride.

Parameters:

handle – [in] the hipsparselt handle
matmulDescr – [inout] the matrix multiplication descriptor
matmulAttribute – [in] see hipsparseLtMatmulDescAttribute_t
data – [in] pointer to the value to which the specified attribute will be set.
dataSize – [in] size in bytes of the attribute value used for verification.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , matDescr , data or dataSize is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – hipsparseLtMatmulDescAttribute_t is not supported.

4.6.3. hipsparseLtMatmulDescGetAttribute()#

hipsparseStatus_t hipsparseLtMatmulDescGetAttribute(const hipsparseLtHandle_t *handle, const hipsparseLtMatmulDescriptor_t *matmulDescr, hipsparseLtMatmulDescAttribute_t matmulAttribute, void *data, size_t dataSize)#

Get the matrix type of a matrix descriptor.

hipsparseLtMatmulDescGetAttribute returns the matrix attribute of a matrix descriptor

Parameters:

handle – [in] the hipsparselt handle
matmulDescr – [in] the matrix multiplication descriptor
matmulAttribute – [in] see hipsparseLtMatmulDescAttribute_t
data – [inout] the memory address containing the attribute value retrieved by this function
dataSize – [in] size in bytes of the attribute value used for verification.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , matDescr , data or dataSize is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – hipsparseLtMatmulDescAttribute_t is not supported.

4.7. Matmul Algorithm Functions#

This matmul algorithm describe functions that provide algortithms for doing the matrix multiply.

4.7.1. hipsparseLtMatmulAlgSelectionInit()#

hipsparseStatus_t hipsparseLtMatmulAlgSelectionInit(const hipsparseLtHandle_t *handle, hipsparseLtMatmulAlgSelection_t *algSelection, const hipsparseLtMatmulDescriptor_t *matmulDescr, hipsparseLtMatmulAlg_t alg)#

Initializes the algorithm selection descriptor.

hipsparseLtMatmulAlgSelectionInit creates a algorithm selection descriptor.

Parameters:

handle – [in] the hipsparselt handle
algSelection – [out] the pointer to the algorithm selection descriptor
matmulDescr – [in] the matrix multiplication descriptor
alg – [in] the algorithm used to do the matrix multiplication.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , matmulDescr or algSelection is invalid.

4.7.2. hipsparseLtMatmulAlgSetAttribute()#

hipsparseStatus_t hipsparseLtMatmulAlgSetAttribute(const hipsparseLtHandle_t *handle, hipsparseLtMatmulAlgSelection_t *algSelection, hipsparseLtMatmulAlgAttribute_t attribute, const void *data, size_t dataSize)#

Specify the algorithm attribute of a algorithm selection descriptor.

hipsparseLtMatmulAlgSetAttribute sets the value of the specified attribute belonging to algorithm selection descriptor.

Parameters:

handle – [in] the hipsparselt handle
algSelection – [inout] the algorithm selection descriptor
attribute – [in] attributes are specify in hipsparseLtMatmulAlgAttribute_t
data – [in] pointer to the value to which the specified attribute will be set.
dataSize – [in] size in bytes of the attribute value used for verification.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , algSelection , attribute , data or dataSize is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – attribute is not supported.

4.7.3. hipsparseLtMatmulAlgGetAttribute()#

hipsparseStatus_t hipsparseLtMatmulAlgGetAttribute(const hipsparseLtHandle_t *handle, const hipsparseLtMatmulAlgSelection_t *algSelection, hipsparseLtMatmulAlgAttribute_t attribute, void *data, size_t dataSize)#

Get the specific algorithm attribute from algorithm selection descriptor.

hipsparseLtMatmulAlgGetAttribute returns the value of the queried attribute belonging to algorithm selection descriptor.

Parameters:

handle – [in] the hipsparselt handle
algSelection – [in] the algorithm selection descriptor
attribute – [in] attributes are specify in hipsparseLtMatmulAlgAttribute_t
data – [inout] the memory address containing the attribute value retrieved by this function
dataSize – [in] size in bytes of the attribute value used for verification.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , algSelection , attribute , data or dataSize is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – attribute is not supported.

4.8. Matmul Functions#

This matmul describe operations that provide multiply of sparse matrices.

4.8.1. hipsparseLtMatmulGetWorkspace()#

hipsparseStatus_t hipsparseLtMatmulGetWorkspace(const hipsparseLtHandle_t *handle, const hipsparseLtMatmulPlan_t *plan, size_t *workspaceSize)#

Determines the required workspace size.

hipsparseLtMatmulGetWorkspace determines the required workspace size associated to the selected algorithm.

Parameters:

handle – [in] hipsparselt library handle
plan – [in] the matrix multiplication plan descriptor.
workspaceSize – [out] Workspace size in bytes

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , algSelection or workspaceSize is invalid.

4.8.2. hipsparseLtMatmulPlanInit()#

hipsparseStatus_t hipsparseLtMatmulPlanInit(const hipsparseLtHandle_t *handle, hipsparseLtMatmulPlan_t *plan, const hipsparseLtMatmulDescriptor_t *matmulDescr, const hipsparseLtMatmulAlgSelection_t *algSelection)#

Initializes the matrix multiplication plan descriptor.

hipsparseLtMatmulPlanInit creates a matrix multiplication plan descriptor. It should be destroyed at the end using hipsparseLtMatmulPlanDestroy.

Parameters:

handle – [in] hipsparselt library handle
plan – [out] the matrix multiplication plan descriptor
matmulDescr – [in] the matrix multiplication descriptor
algSelection – [in] the algorithm selection descriptor

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , plan , matmulDescr , algSelection or workspaceSize is invalid. HIPSPARSELT_MAT_NUM_BATCHES from matrix A to D are inconisistent

4.8.3. hipsparseLtMatmulPlanDestroy()#

hipsparseStatus_t hipsparseLtMatmulPlanDestroy(const hipsparseLtMatmulPlan_t *plan)#

Destroy a matrix multiplication plan descriptor.

hipsparseLtMatmulPlanDestroy releases the resources used by an instance of the matrix multiplication plan. This function is the last call with a specific plan instance.

Parameters:

plan – [in] the matrix multiplication plan descriptor

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – plan is invalid.

4.8.4. hipsparseLtMatmul()#

hipsparseStatus_t hipsparseLtMatmul(const hipsparseLtHandle_t *handle, const hipsparseLtMatmulPlan_t *plan, const void *alpha, const void *d_A, const void *d_B, const void *beta, const void *d_C, void *d_D, void *workspace, hipStream_t *streams, int32_t numStreams)#

Sparse matrix dense matrix multiplication.

hipsparseLtMatmul computes the matrix multiplication of matrices A and B to produce the output matrix D, according to the following operation:

\[ D := Activation(\alpha \cdot op(A) \cdot op(B) + \beta \cdot C + bias) * scale \]

Note

This function is non blocking and executed asynchronously with respect to the host. It may return before the actual computation has finished.

Note

Currently, only supports the case where D has the same shape of C.

Parameters:

handle – [in] hipsparselt library handle
plan – [in] Matrix multiplication plan
alpha – [in] scalar \(\alpha\). (float)
d_A – [in] Pointer to the structured matrix A
d_B – [in] Pointer to the dense matrix B
beta – [in] scalar \(\beta\). (float)
d_C – [in] Pointer to the dense matrix C
d_D – [out] Pointer to the dense matrix D
workspace – [in] Pointor to the worksapce
streams – [in] Pointer to HIP stream array for the computation
numStreams – [in] Number of HIP streams in streams

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_NOT_INITIALIZED – handle or plan is invalid.
HIPSPARSE_STATUS_INVALID_VALUE – handle, plan alpha, d_A, d_B, beta, d_C , d_D , workspace streams or numStreams is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – the problme is not supported.

Data Type Support

Input	Output	Compute Type	Backend
HIPSPARSELT_R_16F	HIPSPARSELT_R_16F	HIPSPARSELT_COMPUTE_32F	HIP
HIPSPARSELT_R_16BF	HIPSPARSELT_R_16BF	HIPSPARSELT_COMPUTE_32F	HIP
HIPSPARSELT_R_8I	HIPSPARSELT_R_8I	HIPSPARSELT_COMPUTE_32I	HIP / CUDA
HIPSPARSELT_R_8I	HIPSPARSELT_R_16F	HIPSPARSELT_COMPUTE_32I	HIP / CUDA
HIPSPARSELT_R_16F	HIPSPARSELT_R_16F	HIPSPARSELT_COMPUTE_16F	CUDA
HIPSPARSELT_R_16BF	HIPSPARSELT_R_16BF	HIPSPARSELT_COMPUTE_16F	CUDA
HIPSPARSELT_R_32F	HIPSPARSELT_R_32F	HIPSPARSELT_COMPUTE_TF32	CUDA
HIPSPARSELT_R_32F	HIPSPARSELT_R_32F	HIPSPARSELT_COMPUTE_TF32_FAST	CUDA

4.8.5. hipsparseLtMatmulSearch()#

hipsparseStatus_t hipsparseLtMatmulSearch(const hipsparseLtHandle_t *handle, hipsparseLtMatmulPlan_t *plan, const void *alpha, const void *d_A, const void *d_B, const void *beta, const void *d_C, void *d_D, void *workspace, hipStream_t *streams, int32_t numStreams)#

Sparse matrix dense matrix multiplication.

hipsparseLtMatmulSearch evaluates all available algorithms for the matrix multiplication and automatically updates the plan by selecting the fastest one. The functionality is intended to be used for auto-tuning purposes when the same operation is repeated multiple times over different inputs.

Note

This function’s behavior is the same of hipsparseLtMatmul

Note

d_C and d_D must be two different memory buffers, otherwise the output will be incorrect.

Note

This function is NOT asynchronous with respect to streams[0] (blocking call)

Note

The number of iterations for the evaluation can be set by using hipsparseLtMatmulAlgSetAttribute() with HIPSPARSELT_MATMUL_SEARCH_ITERATIONS.

Note

The selected algorithm id can be retrieved by using

Parameters:

handle – [in] hipsparselt library handle
plan – [in] Matrix multiplication plan
alpha – [in] scalar \(\alpha\). (float)
d_A – [in] Pointer to the structured matrix A
d_B – [in] Pointer to the dense matrix B
beta – [in] scalar \(\beta\). (float)
d_C – [in] Pointer to the dense matrix C
d_D – [out] Pointer to the dense matrix D
workspace – [in] Pointor to the worksapce
streams – [in] Pointer to HIP stream array for the computation
numStreams – [in] Number of HIP streams in streams

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_NOT_INITIALIZED – handle or plan is invalid.
HIPSPARSE_STATUS_INVALID_VALUE – handle, plan alpha, d_A, d_B, beta, d_C , d_D , workspace streams or numStreams is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – the problme is not supported.

4.9. Helper Functions#

This module holds available helper functions that are required for subsequent library calls

4.9.1. hipsparseLtSpMMAPrune()#

hipsparseStatus_t hipsparseLtSpMMAPrune(const hipsparseLtHandle_t *handle, const hipsparseLtMatmulDescriptor_t *matmulDescr, const void *d_in, void *d_out, hipsparseLtPruneAlg_t pruneAlg, hipStream_t stream)#

Purnes a dense matrix.

hipsparseLtSpMMAPrune prunes a dense matrix d_in according to the specified algorithm pruneAlg, HIPSPARSELT_PRUNE_SPMMA_TILE or HIPSPARSELT_PRUNE_SPMMA_STRIP.

Note

The function requires no extra storage.

Note

This function supports asynchronous execution with respect to stream.

Parameters:

handle – [in] hipsparselt library handle
matmulDescr – [in] matrix multiplication descriptor.
d_in – [in] pointer to the dense matrix.
d_out – [out] pointer to the pruned matrix.
pruneAlg – [in] pruning algorithm.
stream – [in] HIP stream for the computation.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , matmulDescr , d_in or d_out is invalid.

4.9.2. hipsparseLtSpMMAPruneCheck()#

hipsparseStatus_t hipsparseLtSpMMAPruneCheck(const hipsparseLtHandle_t *handle, const hipsparseLtMatmulDescriptor_t *matmulDescr, const void *d_in, int *d_valid, hipStream_t stream)#

checks the correctness of the pruning structure for a given matrix.

hipsparseLtSpMMAPruneCheck checks the correctness of the pruning structure for a given matrix. Contents in the given matrix must be sparsity 2:4.

Parameters:

handle – [in] hipsparselt library handle
matmulDescr – [in] matrix multiplication descriptor.
d_in – [in] pointer to the matrix to check.
d_valid – [out] validation results (0 correct, 1 wrong).
stream – [in] HIP stream for the computation.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , matmulDescr , d_in or d_valid is invalid.

4.9.3. hipsparseLtSpMMAPrune2()#

hipsparseStatus_t hipsparseLtSpMMAPrune2(const hipsparseLtHandle_t *handle, const hipsparseLtMatDescriptor_t *sparseMatDescr, int isSparseA, hipsparseOperation_t op, const void *d_in, void *d_out, hipsparseLtPruneAlg_t pruneAlg, hipStream_t stream)#

Purnes a dense matrix.

hipsparseLtSpMMAPrune2 prunes a dense matrix d_in according to the specified algorithm pruneAlg, HIPSPARSELT_PRUNE_SPMMA_TILE or HIPSPARSELT_PRUNE_SPMMA_STRIP.

Note

The function requires no extra storage.

Note

This function supports asynchronous execution with respect to stream.

Parameters:

handle – [in] hipsparselt library handle
sparseMatDescr – [in] structured(sparse) matrix descriptor.
isSparseA – [in] specify if the structured (sparse) matrix is in the first position (matA or matB) (Currently, only support matA)
op – [in] operation that will be applied to the structured (sparse) matrix in the multiplication
d_in – [in] pointer to the dense matrix.
d_out – [out] pointer to the pruned matrix.
pruneAlg – [in] pruning algorithm.
stream – [in] HIP stream for the computation.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , sparseMatDescr , op , d_in or d_out is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – the problem is not support

4.9.4. hipsparseLtSpMMAPruneCheck2()#

hipsparseStatus_t hipsparseLtSpMMAPruneCheck2(const hipsparseLtHandle_t *handle, const hipsparseLtMatDescriptor_t *sparseMatDescr, int isSparseA, hipsparseOperation_t op, const void *d_in, int *d_valid, hipStream_t stream)#

checks the correctness of the pruning structure for a given matrix.

hipsparseLtSpMMAPruneCheck2 checks the correctness of the pruning structure for a given matrix. Contents in the given matrix must be sparsity 2:4.

Parameters:

handle – [in] hipsparselt library handle
sparseMatDescr – [in] structured(sparse) matrix descriptor.
isSparseA – [in] specify if the structured (sparse) matrix is in the first position (matA or matB) (HIP backend only support matA)
op – [in] operation that will be applied to the structured (sparse) matrix in the multiplication
d_in – [in] pointer to the matrix to check.
d_valid – [out] validation results (0 correct, 1 wrong).
stream – [in] HIP stream for the computation.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , sparseMatDescr , op , d_in or d_valid is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – the problem is not support

4.9.5. hipsparseLtSpMMACompressedSize()#

hipsparseStatus_t hipsparseLtSpMMACompressedSize(const hipsparseLtHandle_t *handle, const hipsparseLtMatmulPlan_t *plan, size_t *compressedSize, size_t *compressBufferSize)#

provide the size of the compressed matrix.

HIPSPARSE_STATUS_INVALID_VALUE provides the size of the compressed matrix to be allocated before calling hipsparseLtSpMMACompress() or hipsparseLtSpMMACompress2().

Parameters:

handle – [in] hipsparselt library handle
plan – [in] matrix multiplication plan descriptor.
compressedSize – [out] size in bytes of the compressed matrix.
compressBufferSize – [out] size in bytes for the buffer needed for the matrix compression.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , plan , compressedSize or compressBufferSize is invalid.

4.9.6. hipsparseLtSpMMACompress()#

hipsparseStatus_t hipsparseLtSpMMACompress(const hipsparseLtHandle_t *handle, const hipsparseLtMatmulPlan_t *plan, const void *d_dense, void *d_compressed, void *d_compressBuffer, hipStream_t stream)#

compresses a dense matrix to structured matrix.

hipsparseLtSpMMACompress compresses a dense matrix d_dense. The compressed matrix is intended to be used as the first/second operand A/B in the hipsparseLtMatmul() function.

Parameters:

handle – [in] handle to the hipsparselt library context queue.
plan – [in] matrix multiplication plan descriptor.
d_dense – [in] pointer to the dense matrix.
d_compressed – [out] compressed matrix and metadata.
d_compressBuffer – [out] temporary buffer for the compression.
stream – [in] HIP stream for the computation.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , plan , d_dense or d_compressed is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – the problem is not support

4.9.7. hipsparseLtSpMMACompressedSize2()#

hipsparseStatus_t hipsparseLtSpMMACompressedSize2(const hipsparseLtHandle_t *handle, const hipsparseLtMatDescriptor_t *sparseMatDescr, size_t *compressedSize, size_t *compressBufferSize)#

provide the size of the compressed matrix.

hipsparseLtSpMMACompressedSize2 provides the size of the compressed matrix to be allocated before calling hipsparseLtSpMMACompress or hipsparseLtSpMMACompress2

Parameters:

handle – [in] hipsparselt library handle
sparseMatDescr – [in] structured(sparse) matrix descriptor.
compressedSize – [out] size in bytes of the compressed matrix.
compressBufferSize – [out] size in bytes for the buffer needed for the matrix compression.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_NOT_INITIALIZED – handle , sparseMatDescr , compressedSize or compressBufferSize is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – the problem is not support

4.9.8. hipsparseLtSpMMACompress2()#

hipsparseStatus_t hipsparseLtSpMMACompress2(const hipsparseLtHandle_t *handle, const hipsparseLtMatDescriptor_t *sparseMatDescr, int isSparseA, hipsparseOperation_t op, const void *d_dense, void *d_compressed, void *d_compressBuffer, hipStream_t stream)#

compresses a dense matrix to structured matrix.

hipsparseLtSpMMACompress2 compresses a dense matrix d_dense. The compressed matrix is intended to be used as the first/second operand A/B in the hipsparseLtMatmul() function.

Parameters:

handle – [in] handle to the hipsparselt library context queue.
sparseMatDescr – [in] structured(sparse) matrix descriptor.
isSparseA – [in] specify if the structured (sparse) matrix is in the first position (matA or matB) (HIP backend only support matA)
op – [in] operation that will be applied to the structured (sparse) matrix in the multiplication
d_dense – [in] pointer to the dense matrix.
d_compressed – [out] compressed matrix and metadata
d_compressBuffer – [out] temporary buffer for the compression.
stream – [in] HIP stream for the computation.

Return values:

HIPSPARSE_STATUS_SUCCESS – the operation completed successfully.
HIPSPARSE_STATUS_INVALID_VALUE – handle , sparseMatDescr , op , d_dense or d_compressed is invalid.
HIPSPARSE_STATUS_NOT_SUPPORTED – the problem is not support