CachingDeviceAllocator Struct Reference#
![](structhipcub_1_1CachingDeviceAllocator.png)
Classes | |
struct | BlockDescriptor |
class | TotalBytes |
Public Types | |
typedef bool(* | Compare) (const BlockDescriptor &, const BlockDescriptor &) |
BlockDescriptor comparator function interface. | |
typedef std::multiset< BlockDescriptor, Compare > | CachedBlocks |
Set type for cached blocks (ordered by size) | |
typedef std::multiset< BlockDescriptor, Compare > | BusyBlocks |
Set type for live blocks (ordered by ptr) | |
typedef std::map< int, TotalBytes > | GpuCachedBytes |
Map type of device ordinals to the number of cached bytes cached by each device. | |
Public Member Functions | |
hipError_t | SetMaxCachedBytes (size_t max_cached_bytes) |
hipError_t | DeviceAllocate (int device, void **d_ptr, size_t bytes, hipStream_t active_stream=0) |
hipError_t | DeviceAllocate (void **d_ptr, size_t bytes, hipStream_t active_stream=0) |
hipError_t | DeviceFree (int device, void *d_ptr) |
hipError_t | DeviceFree (void *d_ptr) |
hipError_t | FreeAllCached () |
void | NearestPowerOf (unsigned int &power, size_t &rounded_bytes, unsigned int base, size_t value) |
CachingDeviceAllocator (unsigned int bin_growth, unsigned int min_bin=1, unsigned int max_bin=INVALID_BIN, size_t max_cached_bytes=INVALID_SIZE, bool skip_cleanup=false, bool debug=false) | |
Set of live device allocations currently in use. More... | |
CachingDeviceAllocator (bool skip_cleanup=false, bool debug=false) | |
Default constructor. More... | |
hipError_t | SetMaxCachedBytes (size_t max_cached_bytes) |
Sets the limit on the number bytes this allocator is allowed to cache per device. More... | |
hipError_t | DeviceAllocate (int device, void **d_ptr, size_t bytes, hipStream_t active_stream=0) |
Provides a suitable allocation of device memory for the given size on the specified device. More... | |
hipError_t | DeviceAllocate (void **d_ptr, size_t bytes, hipStream_t active_stream=0) |
Provides a suitable allocation of device memory for the given size on the current device. More... | |
hipError_t | DeviceFree (int device, void *d_ptr) |
Frees a live allocation of device memory on the specified device, returning it to the allocator. More... | |
hipError_t | DeviceFree (void *d_ptr) |
Frees a live allocation of device memory on the current device, returning it to the allocator. More... | |
hipError_t | FreeAllCached () |
Frees all cached device allocations on all devices. | |
virtual | ~CachingDeviceAllocator () |
Destructor. | |
Static Public Member Functions | |
static unsigned int | IntPow (unsigned int base, unsigned int exp) |
Public Attributes | |
std::mutex | mutex |
unsigned int | bin_growth |
Mutex for thread-safety. | |
unsigned int | min_bin |
Geometric growth factor for bin-sizes. | |
unsigned int | max_bin |
Minimum bin enumeration. | |
size_t | min_bin_bytes |
Maximum bin enumeration. | |
size_t | max_bin_bytes |
Minimum bin size. | |
size_t | max_cached_bytes |
Maximum bin size. | |
const bool | skip_cleanup |
Maximum aggregate cached bytes per device. | |
bool | debug |
Whether or not to skip a call to FreeAllCached() when destructor is called. (The CUDA runtime may have already shut down for statically declared allocators) | |
GpuCachedBytes | cached_bytes |
Whether or not to print (de)allocation events to stdout. | |
CachedBlocks | cached_blocks |
Map of device ordinal to aggregate cached bytes on that device. | |
BusyBlocks | live_blocks |
Set of cached device allocations available for reuse. | |
Static Public Attributes | |
static const unsigned int | INVALID_BIN = (unsigned int) -1 |
Out-of-bounds bin. | |
static const size_t | INVALID_SIZE = (size_t) -1 |
Invalid size. | |
static const int | INVALID_DEVICE_ORDINAL = -1 |
Invalid device ordinal. | |
Constructor & Destructor Documentation
◆ CachingDeviceAllocator() [1/2]
|
inline |
Set of live device allocations currently in use.
Constructor.
- Parameters
-
bin_growth Geometric growth factor for bin-sizes min_bin Minimum bin (default is bin_growth ^ 1) max_bin Maximum bin (default is no max bin) max_cached_bytes Maximum aggregate cached bytes per device (default is no limit) skip_cleanup Whether or not to skip a call to FreeAllCached()
when the destructor is called (default is to deallocate)debug Whether or not to print (de)allocation events to stdout (default is no stderr output)
◆ CachingDeviceAllocator() [2/2]
|
inline |
Default constructor.
Configured with:
bin_growth
= 8min_bin
= 3max_bin
= 7max_cached_bytes
= (bin_growth
^max_bin
) * 3) - 1 = 6,291,455 bytes
which delineates five bin-sizes: 512B, 4KB, 32KB, 256KB, and 2MB and sets a maximum of 6,291,455 cached bytes per device
Member Function Documentation
◆ IntPow()
|
inlinestatic |
Integer pow function for unsigned base and exponent
◆ NearestPowerOf()
|
inline |
Round up to the nearest power-of
◆ SetMaxCachedBytes()
|
inline |
Sets the limit on the number bytes this allocator is allowed to cache per device.
Changing the ceiling of cached bytes does not cause any allocations (in-use or cached-in-reserve) to be freed. See FreeAllCached()
.
◆ DeviceAllocate() [1/2]
|
inline |
Provides a suitable allocation of device memory for the given size on the specified device.
Once freed, the allocation becomes available immediately for reuse within the active_stream
with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream
has completed.
- Parameters
-
[in] device Device on which to place the allocation [out] d_ptr Reference to pointer to the allocation [in] bytes Minimum number of bytes for the allocation [in] active_stream The stream to be associated with this allocation
◆ DeviceAllocate() [2/2]
|
inline |
Provides a suitable allocation of device memory for the given size on the current device.
Once freed, the allocation becomes available immediately for reuse within the active_stream
with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream
has completed.
- Parameters
-
[out] d_ptr Reference to pointer to the allocation [in] bytes Minimum number of bytes for the allocation [in] active_stream The stream to be associated with this allocation
◆ DeviceFree() [1/2]
|
inline |
Frees a live allocation of device memory on the specified device, returning it to the allocator.
Once freed, the allocation becomes available immediately for reuse within the active_stream
with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream
has completed.
◆ DeviceFree() [2/2]
|
inline |
Frees a live allocation of device memory on the current device, returning it to the allocator.
Once freed, the allocation becomes available immediately for reuse within the active_stream
with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream
has completed.
The documentation for this struct was generated from the following file:
- /home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-hipcub/checkouts/docs-5.5.1/hipcub/include/hipcub/backend/cub/util_allocator.hpp