CachingDeviceAllocator Struct Reference#
Classes | |
| struct | BlockDescriptor |
| class | TotalBytes |
Public Types | |
| typedef bool(* | Compare) (const BlockDescriptor &, const BlockDescriptor &) |
| BlockDescriptor comparator function interface. | |
| typedef std::multiset< BlockDescriptor, Compare > | CachedBlocks |
| Set type for cached blocks (ordered by size) | |
| typedef std::multiset< BlockDescriptor, Compare > | BusyBlocks |
| Set type for live blocks (ordered by ptr) | |
| typedef std::map< int, TotalBytes > | GpuCachedBytes |
| Map type of device ordinals to the number of cached bytes cached by each device. | |
Public Member Functions | |
| hipError_t | SetMaxCachedBytes (size_t max_cached_bytes) |
| hipError_t | DeviceAllocate (int device, void **d_ptr, size_t bytes, hipStream_t active_stream=0) |
| hipError_t | DeviceAllocate (void **d_ptr, size_t bytes, hipStream_t active_stream=0) |
| hipError_t | DeviceFree (int device, void *d_ptr) |
| hipError_t | DeviceFree (void *d_ptr) |
| hipError_t | FreeAllCached () |
| void | NearestPowerOf (unsigned int &power, size_t &rounded_bytes, unsigned int base, size_t value) |
| CachingDeviceAllocator (unsigned int bin_growth, unsigned int min_bin=1, unsigned int max_bin=INVALID_BIN, size_t max_cached_bytes=INVALID_SIZE, bool skip_cleanup=false, bool debug=false) | |
| Set of live device allocations currently in use. More... | |
| CachingDeviceAllocator (bool skip_cleanup=false, bool debug=false) | |
| Default constructor. More... | |
| hipError_t | SetMaxCachedBytes (size_t max_cached_bytes) |
| Sets the limit on the number bytes this allocator is allowed to cache per device. More... | |
| hipError_t | DeviceAllocate (int device, void **d_ptr, size_t bytes, hipStream_t active_stream=0) |
| Provides a suitable allocation of device memory for the given size on the specified device. More... | |
| hipError_t | DeviceAllocate (void **d_ptr, size_t bytes, hipStream_t active_stream=0) |
| Provides a suitable allocation of device memory for the given size on the current device. More... | |
| hipError_t | DeviceFree (int device, void *d_ptr) |
| Frees a live allocation of device memory on the specified device, returning it to the allocator. More... | |
| hipError_t | DeviceFree (void *d_ptr) |
| Frees a live allocation of device memory on the current device, returning it to the allocator. More... | |
| hipError_t | FreeAllCached () |
| Frees all cached device allocations on all devices. | |
| virtual | ~CachingDeviceAllocator () |
| Destructor. | |
Static Public Member Functions | |
| static unsigned int | IntPow (unsigned int base, unsigned int exp) |
Public Attributes | |
| std::mutex | mutex |
| unsigned int | bin_growth |
| Mutex for thread-safety. | |
| unsigned int | min_bin |
| Geometric growth factor for bin-sizes. | |
| unsigned int | max_bin |
| Minimum bin enumeration. | |
| size_t | min_bin_bytes |
| Maximum bin enumeration. | |
| size_t | max_bin_bytes |
| Minimum bin size. | |
| size_t | max_cached_bytes |
| Maximum bin size. | |
| const bool | skip_cleanup |
| Maximum aggregate cached bytes per device. | |
| bool | debug |
| Whether or not to skip a call to FreeAllCached() when destructor is called. (The CUDA runtime may have already shut down for statically declared allocators) | |
| GpuCachedBytes | cached_bytes |
| Whether or not to print (de)allocation events to stdout. | |
| CachedBlocks | cached_blocks |
| Map of device ordinal to aggregate cached bytes on that device. | |
| BusyBlocks | live_blocks |
| Set of cached device allocations available for reuse. | |
Static Public Attributes | |
| static const unsigned int | INVALID_BIN = (unsigned int) -1 |
| Out-of-bounds bin. | |
| static const size_t | INVALID_SIZE = (size_t) -1 |
| Invalid size. | |
| static const int | INVALID_DEVICE_ORDINAL = -1 |
| Invalid device ordinal. | |
Constructor & Destructor Documentation
◆ CachingDeviceAllocator() [1/2]
|
inline |
Set of live device allocations currently in use.
Constructor.
- Parameters
-
bin_growth Geometric growth factor for bin-sizes min_bin Minimum bin (default is bin_growth ^ 1) max_bin Maximum bin (default is no max bin) max_cached_bytes Maximum aggregate cached bytes per device (default is no limit) skip_cleanup Whether or not to skip a call to FreeAllCached()when the destructor is called (default is to deallocate)debug Whether or not to print (de)allocation events to stdout (default is no stderr output)
◆ CachingDeviceAllocator() [2/2]
|
inline |
Default constructor.
Configured with:
bin_growth= 8min_bin= 3max_bin= 7max_cached_bytes= (bin_growth^max_bin) * 3) - 1 = 6,291,455 bytes
which delineates five bin-sizes: 512B, 4KB, 32KB, 256KB, and 2MB and sets a maximum of 6,291,455 cached bytes per device
Member Function Documentation
◆ IntPow()
|
inlinestatic |
Integer pow function for unsigned base and exponent
◆ NearestPowerOf()
|
inline |
Round up to the nearest power-of
◆ SetMaxCachedBytes()
|
inline |
Sets the limit on the number bytes this allocator is allowed to cache per device.
Changing the ceiling of cached bytes does not cause any allocations (in-use or cached-in-reserve) to be freed. See FreeAllCached().
◆ DeviceAllocate() [1/2]
|
inline |
Provides a suitable allocation of device memory for the given size on the specified device.
Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.
- Parameters
-
[in] device Device on which to place the allocation [out] d_ptr Reference to pointer to the allocation [in] bytes Minimum number of bytes for the allocation [in] active_stream The stream to be associated with this allocation
◆ DeviceAllocate() [2/2]
|
inline |
Provides a suitable allocation of device memory for the given size on the current device.
Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.
- Parameters
-
[out] d_ptr Reference to pointer to the allocation [in] bytes Minimum number of bytes for the allocation [in] active_stream The stream to be associated with this allocation
◆ DeviceFree() [1/2]
|
inline |
Frees a live allocation of device memory on the specified device, returning it to the allocator.
Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.
◆ DeviceFree() [2/2]
|
inline |
Frees a live allocation of device memory on the current device, returning it to the allocator.
Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.
The documentation for this struct was generated from the following file:
- /home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-hipcub/checkouts/docs-5.6.0/hipcub/include/hipcub/backend/cub/util_allocator.hpp