Stream Management

Stream Management#

HIP Runtime API Reference: Stream Management

Collaboration diagram for Stream Management:

Typedefs
typedef void(*	hipStreamCallback_t) (hipStream_t stream, hipError_t status, void *userData)

Functions
hipError_t	hipStreamCreate (hipStream_t *stream)
	Create an asynchronous stream. More...

hipError_t	hipStreamCreateWithFlags (hipStream_t *stream, unsigned int flags)
	Create an asynchronous stream. More...

hipError_t	hipStreamCreateWithPriority (hipStream_t *stream, unsigned int flags, int priority)
	Create an asynchronous stream with the specified priority. More...

hipError_t	hipDeviceGetStreamPriorityRange (int leastPriority, int greatestPriority)
	Returns numerical values that correspond to the least and greatest stream priority. More...

hipError_t	hipStreamDestroy (hipStream_t stream)
	Destroys the specified stream. More...

hipError_t	hipStreamQuery (hipStream_t stream)
	Return hipSuccess if all of the operations in the specified `stream` have completed, or hipErrorNotReady if not. More...

hipError_t	hipStreamSynchronize (hipStream_t stream)
	Wait for all commands in stream to complete. More...

hipError_t	hipStreamWaitEvent (hipStream_t stream, hipEvent_t event, unsigned int flags)
	Make the specified compute stream wait for an event. More...

hipError_t	hipStreamGetFlags (hipStream_t stream, unsigned int *flags)
	Return flags associated with this stream. More...

hipError_t	hipStreamGetPriority (hipStream_t stream, int *priority)
	Query the priority of a stream. More...

hipError_t	hipExtStreamCreateWithCUMask (hipStream_t stream, uint32_t cuMaskSize, const uint32_t cuMask)
	Create an asynchronous stream with the specified CU mask. More...

hipError_t	hipExtStreamGetCUMask (hipStream_t stream, uint32_t cuMaskSize, uint32_t *cuMask)
	Get CU mask associated with an asynchronous stream. More...

hipError_t	hipStreamAddCallback (hipStream_t stream, hipStreamCallback_t callback, void *userData, unsigned int flags)
	Adds a callback to be called on the host after all currently enqueued items in the stream have completed. For each hipStreamAddCallback call, a callback will be executed exactly once. The callback will block later work in the stream until it is finished. More...

hipError_t	hipStreamWaitValue32 (hipStream_t stream, void *ptr, uint32_t value, unsigned int flags, uint32_t mask __dparm(0xFFFFFFFF))
	Enqueues a wait command to the stream.[BETA]. More...

hipError_t	hipStreamWaitValue64 (hipStream_t stream, void *ptr, uint64_t value, unsigned int flags, uint64_t mask __dparm(0xFFFFFFFFFFFFFFFF))
	Enqueues a wait command to the stream.[BETA]. More...

hipError_t	hipStreamWriteValue32 (hipStream_t stream, void *ptr, uint32_t value, unsigned int flags)
	Enqueues a write command to the stream.[BETA]. More...

hipError_t	hipStreamWriteValue64 (hipStream_t stream, void *ptr, uint64_t value, unsigned int flags)
	Enqueues a write command to the stream.[BETA]. More...

hipError_t	hipMallocAsync (void **dev_ptr, size_t size, hipStream_t stream)
	Allocates memory with stream ordered semantics. More...

hipError_t	hipFreeAsync (void *dev_ptr, hipStream_t stream)
	Frees memory with stream ordered semantics. More...

hipError_t	hipMemPoolTrimTo (hipMemPool_t mem_pool, size_t min_bytes_to_hold)
	Releases freed memory back to the OS. More...

hipError_t	hipMemPoolSetAttribute (hipMemPool_t mem_pool, hipMemPoolAttr attr, void *value)
	Sets attributes of a memory pool. More...

hipError_t	hipMemPoolGetAttribute (hipMemPool_t mem_pool, hipMemPoolAttr attr, void *value)
	Gets attributes of a memory pool. More...

hipError_t	hipMemPoolSetAccess (hipMemPool_t mem_pool, const hipMemAccessDesc *desc_list, size_t count)
	Controls visibility of the specified pool between devices. More...

hipError_t	hipMemPoolGetAccess (hipMemAccessFlags flags, hipMemPool_t mem_pool, hipMemLocation location)
	Returns the accessibility of a pool from a device. More...

hipError_t	hipMemPoolCreate (hipMemPool_t mem_pool, const hipMemPoolProps pool_props)
	Creates a memory pool. More...

hipError_t	hipMemPoolDestroy (hipMemPool_t mem_pool)
	Destroys the specified memory pool. More...

hipError_t	hipMallocFromPoolAsync (void **dev_ptr, size_t size, hipMemPool_t mem_pool, hipStream_t stream)
	Allocates memory from a specified pool with stream ordered semantics. More...

hipError_t	hipMemPoolExportToShareableHandle (void *shared_handle, hipMemPool_t mem_pool, hipMemAllocationHandleType handle_type, unsigned int flags)
	Exports a memory pool to the requested handle type. More...

hipError_t	hipMemPoolImportFromShareableHandle (hipMemPool_t mem_pool, void shared_handle, hipMemAllocationHandleType handle_type, unsigned int flags)
	Imports a memory pool from a shared handle. More...

hipError_t	hipMemPoolExportPointer (hipMemPoolPtrExportData export_data, void dev_ptr)
	Export data to share a memory pool allocation between processes. More...

hipError_t	hipMemPoolImportPointer (void *dev_ptr, hipMemPool_t mem_pool, hipMemPoolPtrExportData export_data)
	Import a memory pool allocation from another process. More...

Detailed Description

This section describes the stream management functions of HIP runtime API. The following Stream APIs are not (yet) supported in HIP:

hipStreamAttachMemAsync is a nop

This section describes Stream Memory Wait and Write functions of HIP runtime API.

This section describes Stream Ordered Memory Allocator functions of HIP runtime API.

The asynchronous allocator allows the user to allocate and free in stream order. All asynchronous accesses of the allocation must happen between the stream executions of the allocation and the free. If the memory is accessed outside of the promised stream order, a use before allocation / use after free error will cause undefined behavior.

The allocator is free to reallocate the memory as long as it can guarantee that compliant memory accesses will not overlap temporally. The allocator may refer to internal stream ordering as well as inter-stream dependencies (such as HIP events and null stream dependencies) when establishing the temporal guarantee. The allocator may also insert inter-stream dependencies to establish the temporal guarantee. Whether or not a device supports the integrated stream ordered memory allocator may be queried by calling hipDeviceGetAttribute with the device attribute hipDeviceAttributeMemoryPoolsSupported

Typedef Documentation

◆ hipStreamCallback_t

typedef void(* hipStreamCallback_t) (hipStream_t stream, hipError_t status, void *userData)

Stream CallBack struct

Function Documentation

◆ hipDeviceGetStreamPriorityRange()

hipError_t hipDeviceGetStreamPriorityRange	(	int *	leastPriority,
		int *	greatestPriority
	)

Returns numerical values that correspond to the least and greatest stream priority.

Parameters

[in,out]	leastPriority	pointer in which value corresponding to least priority is returned.
[in,out]	greatestPriority	pointer in which value corresponding to greatest priority is returned.

Returns in *leastPriority and *greatestPriority the numerical values that correspond to the least and greatest stream priority respectively. Stream priorities follow a convention where lower numbers imply greater priorities. The range of meaningful stream priorities is given by [*greatestPriority, *leastPriority]. If the user attempts to create a stream with a priority value that is outside the the meaningful range as specified by this API, the priority is automatically clamped to within the valid range.

◆ hipExtStreamCreateWithCUMask()

hipError_t hipExtStreamCreateWithCUMask	(	hipStream_t *	stream,
		uint32_t	cuMaskSize,
		const uint32_t *	cuMask
	)

Create an asynchronous stream with the specified CU mask.

Parameters

[in,out]	stream	Pointer to new stream
[in]	cuMaskSize	Size of CU mask bit array passed in.
[in]	cuMask	Bit-vector representing the CU mask. Each active bit represents using one CU. The first 32 bits represent the first 32 CUs, and so on. If its size is greater than physical CU number (i.e., multiProcessorCount member of hipDeviceProp_t), the extra elements are ignored. It is user's responsibility to make sure the input is meaningful.

Returns: hipSuccess, hipErrorInvalidHandle, hipErrorInvalidValue

Create a new asynchronous stream with the specified CU mask. stream returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, application must call hipStreamDestroy.

See also: hipStreamCreate, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

◆ hipExtStreamGetCUMask()

hipError_t hipExtStreamGetCUMask	(	hipStream_t	stream,
		uint32_t	cuMaskSize,
		uint32_t *	cuMask
	)

Get CU mask associated with an asynchronous stream.

Parameters

[in]	stream	stream to be queried
[in]	cuMaskSize	number of the block of memories (uint32_t *) allocated by user
[out]	cuMask	Pointer to a pre-allocated block of memories (uint32_t *) in which the stream's CU mask is returned. The CU mask is returned in a chunck of 32 bits where each active bit represents one active CU

Returns: hipSuccess, hipErrorInvalidHandle, hipErrorInvalidValue

See also: hipStreamCreate, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

◆ hipFreeAsync()

hipError_t hipFreeAsync	(	void *	dev_ptr,
		hipStream_t	stream
	)

Frees memory with stream ordered semantics.

Inserts a free operation into stream. The allocation must not be used after stream execution reaches the free. After this API returns, accessing the memory from any subsequent work launched on the GPU or querying its pointer attributes results in undefined behavior.

Note: During stream capture, this function results in the creation of a free node and must therefore be passed the address of a graph allocation.

Parameters

[in]	dev_ptr	Pointer to device memory to free
[in]	stream	The stream, where the destruciton will occur according to the execution order

Returns: hipSuccess, hipErrorInvalidValue, hipErrorNotSupported

See also: hipMallocFromPoolAsync, hipMallocAsync, hipMemPoolTrimTo, hipMemPoolGetAttribute, hipDeviceSetMemPool, hipMemPoolSetAttribute, hipMemPoolSetAccess, hipMemPoolGetAccess

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMallocAsync()

hipError_t hipMallocAsync	(	void **	dev_ptr,
		size_t	size,
		hipStream_t	stream
	)

Allocates memory with stream ordered semantics.

Inserts a memory allocation operation into stream. A pointer to the allocated memory is returned immediately in *dptr. The allocation must not be accessed until the the allocation operation completes. The allocation comes from the memory pool associated with the stream's device.

Note: The default memory pool of a device contains device memory from that device.; Basic stream ordering allows future work submitted into the same stream to use the allocation. Stream query, stream synchronize, and HIP events can be used to guarantee that the allocation operation completes before work submitted in a separate stream runs.; During stream capture, this function results in the creation of an allocation node. In this case, the allocation is owned by the graph instead of the memory pool. The memory pool's properties are used to set the node's creation parameters.

Parameters

[out]	dev_ptr	Returned device pointer of memory allocation
[in]	size	Number of bytes to allocate
[in]	stream	The stream establishing the stream ordering contract and the memory pool to allocate from

Returns: hipSuccess, hipErrorInvalidValue, hipErrorNotSupported, hipErrorOutOfMemory

See also: hipMallocFromPoolAsync, hipFreeAsync, hipMemPoolTrimTo, hipMemPoolGetAttribute, hipDeviceSetMemPool, hipMemPoolSetAttribute, hipMemPoolSetAccess, hipMemPoolGetAccess

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMallocFromPoolAsync()

hipError_t hipMallocFromPoolAsync	(	void **	dev_ptr,
		size_t	size,
		hipMemPool_t	mem_pool,
		hipStream_t	stream
	)

Allocates memory from a specified pool with stream ordered semantics.

Inserts an allocation operation into stream. A pointer to the allocated memory is returned immediately in dev_ptr. The allocation must not be accessed until the the allocation operation completes. The allocation comes from the specified memory pool.

Note: The specified memory pool may be from a device different than that of the specified stream.

Basic stream ordering allows future work submitted into the same stream to use the allocation. Stream query, stream synchronize, and HIP events can be used to guarantee that the allocation operation completes before work submitted in a separate stream runs.

Note: During stream capture, this function results in the creation of an allocation node. In this case, the allocation is owned by the graph instead of the memory pool. The memory pool's properties are used to set the node's creation parameters.

Parameters

[out]	dev_ptr	Returned device pointer
[in]	size	Number of bytes to allocate
[in]	mem_pool	The pool to allocate from
[in]	stream	The stream establishing the stream ordering semantic

Returns: hipSuccess, hipErrorInvalidValue, hipErrorNotSupported, hipErrorOutOfMemory

See also: hipMallocAsync, hipFreeAsync, hipMemPoolGetAttribute, hipMemPoolCreate hipMemPoolTrimTo, hipDeviceSetMemPool, hipMemPoolSetAttribute, hipMemPoolSetAccess, hipMemPoolGetAccess,

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolCreate()

hipError_t hipMemPoolCreate	(	hipMemPool_t *	mem_pool,
		const hipMemPoolProps *	pool_props
	)

Creates a memory pool.

Creates a HIP memory pool and returns the handle in mem_pool. The pool_props determines the properties of the pool such as the backing device and IPC capabilities.

By default, the memory pool will be accessible from the device it is allocated on.

Parameters

[out]	mem_pool	Contains createed memory pool
[in]	pool_props	Memory pool properties

Note: Specifying hipMemHandleTypeNone creates a memory pool that will not support IPC.

Returns: hipSuccess, hipErrorInvalidValue, hipErrorNotSupported

See also: hipMallocFromPoolAsync, hipMallocAsync, hipFreeAsync, hipMemPoolGetAttribute, hipMemPoolDestroy, hipMemPoolTrimTo, hipDeviceSetMemPool, hipMemPoolSetAttribute, hipMemPoolSetAccess, hipMemPoolGetAccess

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolDestroy()

hipError_t hipMemPoolDestroy ( hipMemPool_t mem_pool )

Destroys the specified memory pool.

If any pointers obtained from this pool haven't been freed or the pool has free operations that haven't completed when hipMemPoolDestroy is invoked, the function will return immediately and the resources associated with the pool will be released automatically once there are no more outstanding allocations.

Destroying the current mempool of a device sets the default mempool of that device as the current mempool for that device.

Parameters

[in] mem_pool Memory pool for destruction

Note: A device's default memory pool cannot be destroyed.

Returns: hipSuccess, hipErrorInvalidValue

See also: hipMallocFromPoolAsync, hipMallocAsync, hipFreeAsync, hipMemPoolGetAttribute, hipMemPoolCreate hipMemPoolTrimTo, hipDeviceSetMemPool, hipMemPoolSetAttribute, hipMemPoolSetAccess, hipMemPoolGetAccess

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolExportPointer()

hipError_t hipMemPoolExportPointer	(	hipMemPoolPtrExportData *	export_data,
		void *	dev_ptr
	)

Export data to share a memory pool allocation between processes.

Constructs export_data for sharing a specific allocation from an already shared memory pool. The recipient process can import the allocation with the hipMemPoolImportPointer api. The data is not a handle and may be shared through any IPC mechanism.

Parameters

[out]	export_data	Returned export data
[in]	dev_ptr	Pointer to memory being exported

Returns: hipSuccess, hipErrorInvalidValue, hipErrorOutOfMemory

See also: hipMemPoolImportPointer

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolExportToShareableHandle()

hipError_t hipMemPoolExportToShareableHandle	(	void *	shared_handle,
		hipMemPool_t	mem_pool,
		hipMemAllocationHandleType	handle_type,
		unsigned int	flags
	)

Exports a memory pool to the requested handle type.

Given an IPC capable mempool, create an OS handle to share the pool with another process. A recipient process can convert the shareable handle into a mempool with hipMemPoolImportFromShareableHandle. Individual pointers can then be shared with the hipMemPoolExportPointer and hipMemPoolImportPointer APIs. The implementation of what the shareable handle is and how it can be transferred is defined by the requested handle type.

Note: : To create an IPC capable mempool, create a mempool with a hipMemAllocationHandleType other than hipMemHandleTypeNone.

Parameters

[out]	shared_handle	Pointer to the location in which to store the requested handle
[in]	mem_pool	Pool to export
[in]	handle_type	The type of handle to create
[in]	flags	Must be 0

Returns: hipSuccess, hipErrorInvalidValue, hipErrorOutOfMemory

See also: hipMemPoolImportFromShareableHandle

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolGetAccess()

hipError_t hipMemPoolGetAccess	(	hipMemAccessFlags *	flags,
		hipMemPool_t	mem_pool,
		hipMemLocation *	location
	)

Returns the accessibility of a pool from a device.

Returns the accessibility of the pool's memory from the specified location.

Parameters

[out]	flags	Accessibility of the memory pool from the specified location/device
[in]	mem_pool	Memory pool being queried
[in]	location	Location/device for memory pool access

Returns: hipSuccess, hipErrorInvalidValue

See also: hipMallocFromPoolAsync, hipMallocAsync, hipFreeAsync, hipMemPoolGetAttribute, hipMemPoolTrimTo, hipDeviceSetMemPool, hipMemPoolSetAttribute, hipMemPoolSetAccess

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolGetAttribute()

hipError_t hipMemPoolGetAttribute	(	hipMemPool_t	mem_pool,
		hipMemPoolAttr	attr,
		void *	value
	)

Gets attributes of a memory pool.

Supported attributes are:

hipMemPoolAttrReleaseThreshold: (value type = cuuint64_t) Amount of reserved memory in bytes to hold onto before trying to release memory back to the OS. When more than the release threshold bytes of memory are held by the memory pool, the allocator will try to release memory back to the OS on the next call to stream, event or context synchronize. (default 0)
hipMemPoolReuseFollowEventDependencies: (value type = int) Allow hipMallocAsync to use memory asynchronously freed in another stream as long as a stream ordering dependency of the allocating stream on the free action exists. HIP events and null stream interactions can create the required stream ordered dependencies. (default enabled)
hipMemPoolReuseAllowOpportunistic: (value type = int) Allow reuse of already completed frees when there is no dependency between the free and allocation. (default enabled)
hipMemPoolReuseAllowInternalDependencies: (value type = int) Allow hipMallocAsync to insert new stream dependencies in order to establish the stream ordering required to reuse a piece of memory released by hipFreeAsync (default enabled).

Parameters

[in]	mem_pool	The memory pool to get attributes of
[in]	attr	The attribute to get
[in]	value	Retrieved value

Returns: hipSuccess, hipErrorInvalidValue

See also: hipMallocFromPoolAsync, hipMallocAsync, hipFreeAsync, hipMemPoolTrimTo, hipDeviceSetMemPool, hipMemPoolSetAttribute, hipMemPoolSetAccess, hipMemPoolGetAccess

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolImportFromShareableHandle()

hipError_t hipMemPoolImportFromShareableHandle	(	hipMemPool_t *	mem_pool,
		void *	shared_handle,
		hipMemAllocationHandleType	handle_type,
		unsigned int	flags
	)

Imports a memory pool from a shared handle.

Specific allocations can be imported from the imported pool with hipMemPoolImportPointer.

Note: Imported memory pools do not support creating new allocations. As such imported memory pools may not be used in hipDeviceSetMemPool or hipMallocFromPoolAsync calls.

Parameters

[out]	mem_pool	Returned memory pool
[in]	shared_handle	OS handle of the pool to open
[in]	handle_type	The type of handle being imported
[in]	flags	Must be 0

Returns: hipSuccess, hipErrorInvalidValue, hipErrorOutOfMemory

See also: hipMemPoolExportToShareableHandle

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolImportPointer()

hipError_t hipMemPoolImportPointer	(	void **	dev_ptr,
		hipMemPool_t	mem_pool,
		hipMemPoolPtrExportData *	export_data
	)

Import a memory pool allocation from another process.

Returns in dev_ptr a pointer to the imported memory. The imported memory must not be accessed before the allocation operation completes in the exporting process. The imported memory must be freed from all importing processes before being freed in the exporting process. The pointer may be freed with hipFree or hipFreeAsync. If hipFreeAsync is used, the free must be completed on the importing process before the free operation on the exporting process.

Note: The hipFreeAsync api may be used in the exporting process before the hipFreeAsync operation completes in its stream as long as the hipFreeAsync in the exporting process specifies a stream with a stream dependency on the importing process's hipFreeAsync.

Parameters

[out]	dev_ptr	Pointer to imported memory
[in]	mem_pool	Memory pool from which to import a pointer
[in]	export_data	Data specifying the memory to import

Returns: hipSuccess, hipErrorInvalidValue, hipErrorNotInitialized, hipErrorOutOfMemory

See also: hipMemPoolExportPointer

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolSetAccess()

hipError_t hipMemPoolSetAccess	(	hipMemPool_t	mem_pool,
		const hipMemAccessDesc *	desc_list,
		size_t	count
	)

Controls visibility of the specified pool between devices.

Parameters

[in]	mem_pool	Memory pool for acccess change
[in]	desc_list	Array of access descriptors. Each descriptor instructs the access to enable for a single gpu
[in]	count	Number of descriptors in the map array.

Returns: hipSuccess, hipErrorInvalidValue

See also: hipMallocFromPoolAsync, hipMallocAsync, hipFreeAsync, hipMemPoolGetAttribute, hipMemPoolTrimTo, hipDeviceSetMemPool, hipMemPoolSetAttribute, hipMemPoolGetAccess

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolSetAttribute()

hipError_t hipMemPoolSetAttribute	(	hipMemPool_t	mem_pool,
		hipMemPoolAttr	attr,
		void *	value
	)

Sets attributes of a memory pool.

Supported attributes are:

hipMemPoolAttrReleaseThreshold: (value type = cuuint64_t) Amount of reserved memory in bytes to hold onto before trying to release memory back to the OS. When more than the release threshold bytes of memory are held by the memory pool, the allocator will try to release memory back to the OS on the next call to stream, event or context synchronize. (default 0)
hipMemPoolReuseFollowEventDependencies: (value type = int) Allow hipMallocAsync to use memory asynchronously freed in another stream as long as a stream ordering dependency of the allocating stream on the free action exists. HIP events and null stream interactions can create the required stream ordered dependencies. (default enabled)
hipMemPoolReuseAllowOpportunistic: (value type = int) Allow reuse of already completed frees when there is no dependency between the free and allocation. (default enabled)
hipMemPoolReuseAllowInternalDependencies: (value type = int) Allow hipMallocAsync to insert new stream dependencies in order to establish the stream ordering required to reuse a piece of memory released by hipFreeAsync (default enabled).

Parameters

[in]	mem_pool	The memory pool to modify
[in]	attr	The attribute to modify
[in]	value	Pointer to the value to assign

Returns: hipSuccess, hipErrorInvalidValue

See also: hipMallocFromPoolAsync, hipMallocAsync, hipFreeAsync, hipMemPoolGetAttribute, hipMemPoolTrimTo, hipDeviceSetMemPool, hipMemPoolSetAccess, hipMemPoolGetAccess

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipMemPoolTrimTo()

hipError_t hipMemPoolTrimTo	(	hipMemPool_t	mem_pool,
		size_t	min_bytes_to_hold
	)

Releases freed memory back to the OS.

Releases memory back to the OS until the pool contains fewer than min_bytes_to_keep reserved bytes, or there is no more memory that the allocator can safely release. The allocator cannot release OS allocations that back outstanding asynchronous allocations. The OS allocations may happen at different granularity from the user allocations.

Note: : Allocations that have not been freed count as outstanding.; : Allocations that have been asynchronously freed but whose completion has not been observed on the host (eg. by a synchronize) can count as outstanding.

Parameters

[in]	mem_pool	The memory pool to trim allocations
[in]	min_bytes_to_hold	If the pool has less than min_bytes_to_hold reserved, then the TrimTo operation is a no-op. Otherwise the memory pool will contain at least min_bytes_to_hold bytes reserved after the operation.

Returns: hipSuccess, hipErrorInvalidValue

See also: hipMallocFromPoolAsync, hipMallocAsync, hipFreeAsync, hipMemPoolGetAttribute, hipDeviceSetMemPool, hipMemPoolSetAttribute, hipMemPoolSetAccess, hipMemPoolGetAccess

Warning: : This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

◆ hipStreamAddCallback()

hipError_t hipStreamAddCallback	(	hipStream_t	stream,
		hipStreamCallback_t	callback,
		void *	userData,
		unsigned int	flags
	)

Adds a callback to be called on the host after all currently enqueued items in the stream have completed. For each hipStreamAddCallback call, a callback will be executed exactly once. The callback will block later work in the stream until it is finished.

Parameters

[in]	stream	- Stream to add callback to
[in]	callback	- The function to call once preceding stream operations are complete
[in]	userData	- User specified data to be passed to the callback function
[in]	flags	- Reserved for future use, must be 0

Returns: hipSuccess, hipErrorInvalidHandle, hipErrorNotSupported

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamQuery, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy, hipStreamCreateWithPriority

◆ hipStreamCreate()

hipError_t hipStreamCreate ( hipStream_t * stream )

Create an asynchronous stream.

Parameters

[in,out] stream Valid pointer to hipStream_t. This function writes the memory with the newly created stream.

Returns: hipSuccess, hipErrorInvalidValue

Create a new asynchronous stream. stream returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, applicaiton must call hipStreamDestroy.

Returns: hipSuccess, hipErrorInvalidValue

See also: hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

◆ hipStreamCreateWithFlags()

hipError_t hipStreamCreateWithFlags	(	hipStream_t *	stream,
		unsigned int	flags
	)

Create an asynchronous stream.

Parameters

[in,out]	stream	Pointer to new stream
[in]	flags	to control stream creation.

Returns: hipSuccess, hipErrorInvalidValue

Create a new asynchronous stream. stream returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, applicaiton must call hipStreamDestroy. Flags controls behavior of the stream. See hipStreamDefault, hipStreamNonBlocking.

See also: hipStreamCreate, hipStreamCreateWithPriority, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

◆ hipStreamCreateWithPriority()

hipError_t hipStreamCreateWithPriority	(	hipStream_t *	stream,
		unsigned int	flags,
		int	priority
	)

Create an asynchronous stream with the specified priority.

Parameters

[in,out]	stream	Pointer to new stream
[in]	flags	to control stream creation.
[in]	priority	of the stream. Lower numbers represent higher priorities.

Returns: hipSuccess, hipErrorInvalidValue

Create a new asynchronous stream with the specified priority. stream returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, applicaiton must call hipStreamDestroy. Flags controls behavior of the stream. See hipStreamDefault, hipStreamNonBlocking.

See also: hipStreamCreate, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

◆ hipStreamDestroy()

hipError_t hipStreamDestroy ( hipStream_t stream )

Destroys the specified stream.

Parameters

[in,out] stream Valid pointer to hipStream_t. This function writes the memory with the newly created stream.

Returns: hipSuccess hipErrorInvalidHandle

Destroys the specified stream.

If commands are still executing on the specified stream, some may complete execution before the queue is deleted.

The queue may be destroyed while some commands are still inflight, or may wait for all commands queued to the stream before destroying it.

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamQuery, hipStreamWaitEvent, hipStreamSynchronize

◆ hipStreamGetFlags()

hipError_t hipStreamGetFlags	(	hipStream_t	stream,
		unsigned int *	flags
	)

Return flags associated with this stream.

Parameters

[in]	stream	stream to be queried
[in,out]	flags	Pointer to an unsigned integer in which the stream's flags are returned

Returns: hipSuccess, hipErrorInvalidValue, hipErrorInvalidHandle; hipSuccess hipErrorInvalidValue hipErrorInvalidHandle

Return flags associated with this stream in *flags.

See also: hipStreamCreateWithFlags

◆ hipStreamGetPriority()

hipError_t hipStreamGetPriority	(	hipStream_t	stream,
		int *	priority
	)

Query the priority of a stream.

Parameters

[in]	stream	stream to be queried
[in,out]	priority	Pointer to an unsigned integer in which the stream's priority is returned

Returns: hipSuccess, hipErrorInvalidValue, hipErrorInvalidHandle; hipSuccess hipErrorInvalidValue hipErrorInvalidHandle

Query the priority of a stream. The priority is returned in in priority.

See also: hipStreamCreateWithFlags

◆ hipStreamQuery()

hipError_t hipStreamQuery ( hipStream_t stream )

Return hipSuccess if all of the operations in the specified stream have completed, or hipErrorNotReady if not.

Parameters

[in] stream stream to query

Returns: hipSuccess, hipErrorNotReady, hipErrorInvalidHandle

This is thread-safe and returns a snapshot of the current state of the queue. However, if other host threads are sending work to the stream, the status may change immediately after the function is called. It is typically used for debug.

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamWaitEvent, hipStreamSynchronize, hipStreamDestroy

◆ hipStreamSynchronize()

hipError_t hipStreamSynchronize ( hipStream_t stream )

Wait for all commands in stream to complete.

Parameters

[in] stream stream identifier.

Returns: hipSuccess, hipErrorInvalidHandle

This command is host-synchronous : the host will block until the specified stream is empty.

This command follows standard null-stream semantics. Specifically, specifying the null stream will cause the command to wait for other streams on the same device to complete all pending operations.

This command honors the hipDeviceLaunchBlocking flag, which controls whether the wait is active or blocking.

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamWaitEvent, hipStreamDestroy

◆ hipStreamWaitEvent()

hipError_t hipStreamWaitEvent	(	hipStream_t	stream,
		hipEvent_t	event,
		unsigned int	flags
	)

Make the specified compute stream wait for an event.

Parameters

[in]	stream	stream to make wait.
[in]	event	event to wait on
[in]	flags	control operation [must be 0]

Returns: hipSuccess, hipErrorInvalidHandle

This function inserts a wait operation into the specified stream. All future work submitted to stream will wait until event reports completion before beginning execution.

This function only waits for commands in the current stream to complete. Notably,, this function does not impliciy wait for commands in the default stream to complete, even if the specified stream is created with hipStreamNonBlocking = 0.

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamSynchronize, hipStreamDestroy

◆ hipStreamWaitValue32()

hipError_t hipStreamWaitValue32	(	hipStream_t	stream,
		void *	ptr,
		uint32_t	value,
		unsigned int	flags,
		uint32_t mask	__dparm0xFFFFFFFF
	)

Enqueues a wait command to the stream.[BETA].

Parameters

[in]	stream	- Stream identifier
[in]	ptr	- Pointer to memory object allocated using 'hipMallocSignalMemory' flag
[in]	value	- Value to be used in compare operation
[in]	flags	- Defines the compare operation, supported values are hipStreamWaitValueGte hipStreamWaitValueEq, hipStreamWaitValueAnd and hipStreamWaitValueNor
[in]	mask	- Mask to be applied on value at memory before it is compared with value, default value is set to enable every bit

Returns: hipSuccess, hipErrorInvalidValue

Enqueues a wait command to the stream, all operations enqueued on this stream after this, will not execute until the defined wait condition is true.

hipStreamWaitValueGte: waits until *ptr&mask >= value hipStreamWaitValueEq : waits until *ptr&mask == value hipStreamWaitValueAnd: waits until ((*ptr&mask) & value) != 0 hipStreamWaitValueNor: waits until ~((*ptr&mask) | (value&mask)) != 0

Note: when using 'hipStreamWaitValueNor', mask is applied on both 'value' and '*ptr'.; Support for hipStreamWaitValue32 can be queried using 'hipDeviceGetAttribute()' and 'hipDeviceAttributeCanUseStreamWaitValue' flag.

@beta This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

See also: hipExtMallocWithFlags, hipFree, hipStreamWaitValue64, hipStreamWriteValue64, hipStreamWriteValue32, hipDeviceGetAttribute

◆ hipStreamWaitValue64()

hipError_t hipStreamWaitValue64	(	hipStream_t	stream,
		void *	ptr,
		uint64_t	value,
		unsigned int	flags,
		uint64_t mask	__dparm0xFFFFFFFFFFFFFFFF
	)

Enqueues a wait command to the stream.[BETA].

Parameters

[in]	stream	- Stream identifier
[in]	ptr	- Pointer to memory object allocated using 'hipMallocSignalMemory' flag
[in]	value	- Value to be used in compare operation
[in]	flags	- Defines the compare operation, supported values are hipStreamWaitValueGte hipStreamWaitValueEq, hipStreamWaitValueAnd and hipStreamWaitValueNor.
[in]	mask	- Mask to be applied on value at memory before it is compared with value default value is set to enable every bit

Returns: hipSuccess, hipErrorInvalidValue

Enqueues a wait command to the stream, all operations enqueued on this stream after this, will not execute until the defined wait condition is true.

hipStreamWaitValueGte: waits until *ptr&mask >= value hipStreamWaitValueEq : waits until *ptr&mask == value hipStreamWaitValueAnd: waits until ((*ptr&mask) & value) != 0 hipStreamWaitValueNor: waits until ~((*ptr&mask) | (value&mask)) != 0

Note: when using 'hipStreamWaitValueNor', mask is applied on both 'value' and '*ptr'.; Support for hipStreamWaitValue64 can be queried using 'hipDeviceGetAttribute()' and 'hipDeviceAttributeCanUseStreamWaitValue' flag.

@beta This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

See also: hipExtMallocWithFlags, hipFree, hipStreamWaitValue32, hipStreamWriteValue64, hipStreamWriteValue32, hipDeviceGetAttribute

◆ hipStreamWriteValue32()

hipError_t hipStreamWriteValue32	(	hipStream_t	stream,
		void *	ptr,
		uint32_t	value,
		unsigned int	flags
	)

Enqueues a write command to the stream.[BETA].

Parameters

[in]	stream	- Stream identifier
[in]	ptr	- Pointer to a GPU accessible memory object
[in]	value	- Value to be written
[in]	flags	- reserved, ignored for now, will be used in future releases

Returns: hipSuccess, hipErrorInvalidValue

Enqueues a write command to the stream, write operation is performed after all earlier commands on this stream have completed the execution.

@beta This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

See also: hipExtMallocWithFlags, hipFree, hipStreamWriteValue32, hipStreamWaitValue32, hipStreamWaitValue64

◆ hipStreamWriteValue64()

hipError_t hipStreamWriteValue64	(	hipStream_t	stream,
		void *	ptr,
		uint64_t	value,
		unsigned int	flags
	)

Enqueues a write command to the stream.[BETA].

Parameters

[in]	stream	- Stream identifier
[in]	ptr	- Pointer to a GPU accessible memory object
[in]	value	- Value to be written
[in]	flags	- reserved, ignored for now, will be used in future releases

Returns: hipSuccess, hipErrorInvalidValue

Enqueues a write command to the stream, write operation is performed after all earlier commands on this stream have completed the execution.

@beta This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

See also: hipExtMallocWithFlags, hipFree, hipStreamWriteValue32, hipStreamWaitValue32, hipStreamWaitValue64

Stream Management

Stream Management#

Typedefs

Functions

Detailed Description

Typedef Documentation

◆ hipStreamCallback_t

Function Documentation

◆ hipDeviceGetStreamPriorityRange()

◆ hipExtStreamCreateWithCUMask()

◆ hipExtStreamGetCUMask()

◆ hipFreeAsync()

◆ hipMallocAsync()

◆ hipMallocFromPoolAsync()

◆ hipMemPoolCreate()

◆ hipMemPoolDestroy()

◆ hipMemPoolExportPointer()

◆ hipMemPoolExportToShareableHandle()

◆ hipMemPoolGetAccess()

◆ hipMemPoolGetAttribute()

◆ hipMemPoolImportFromShareableHandle()

◆ hipMemPoolImportPointer()

◆ hipMemPoolSetAccess()

◆ hipMemPoolSetAttribute()

◆ hipMemPoolTrimTo()

◆ hipStreamAddCallback()

◆ hipStreamCreate()

◆ hipStreamCreateWithFlags()

◆ hipStreamCreateWithPriority()

◆ hipStreamDestroy()

◆ hipStreamGetFlags()

◆ hipStreamGetPriority()

◆ hipStreamQuery()

◆ hipStreamSynchronize()

◆ hipStreamWaitEvent()

◆ hipStreamWaitValue32()

◆ hipStreamWaitValue64()

◆ hipStreamWriteValue32()

◆ hipStreamWriteValue64()