CachingDeviceAllocator Struct Reference

CachingDeviceAllocator Struct Reference#

hipCUB: hipcub::CachingDeviceAllocator Struct Reference
Inheritance diagram for hipcub::CachingDeviceAllocator:

Classes

struct  BlockDescriptor
 
class  TotalBytes
 

Public Types

typedef bool(* Compare) (const BlockDescriptor &, const BlockDescriptor &)
 BlockDescriptor comparator function interface.
 
typedef std::multiset< BlockDescriptor, CompareCachedBlocks
 Set type for cached blocks (ordered by size)
 
typedef std::multiset< BlockDescriptor, CompareBusyBlocks
 Set type for live blocks (ordered by ptr)
 
typedef std::map< int, TotalBytesGpuCachedBytes
 Map type of device ordinals to the number of cached bytes cached by each device.
 

Public Member Functions

hipError_t SetMaxCachedBytes (size_t max_cached_bytes)
 
hipError_t DeviceAllocate (int device, void **d_ptr, size_t bytes, hipStream_t active_stream=0)
 
hipError_t DeviceAllocate (void **d_ptr, size_t bytes, hipStream_t active_stream=0)
 
hipError_t DeviceFree (int device, void *d_ptr)
 
hipError_t DeviceFree (void *d_ptr)
 
hipError_t FreeAllCached ()
 
void NearestPowerOf (unsigned int &power, size_t &rounded_bytes, unsigned int base, size_t value)
 
 CachingDeviceAllocator (unsigned int bin_growth, unsigned int min_bin=1, unsigned int max_bin=INVALID_BIN, size_t max_cached_bytes=INVALID_SIZE, bool skip_cleanup=false, bool debug=false)
 Set of live device allocations currently in use. More...
 
 CachingDeviceAllocator (bool skip_cleanup=false, bool debug=false)
 Default constructor. More...
 
hipError_t SetMaxCachedBytes (size_t max_cached_bytes)
 Sets the limit on the number bytes this allocator is allowed to cache per device. More...
 
hipError_t DeviceAllocate (int device, void **d_ptr, size_t bytes, hipStream_t active_stream=0)
 Provides a suitable allocation of device memory for the given size on the specified device. More...
 
hipError_t DeviceAllocate (void **d_ptr, size_t bytes, hipStream_t active_stream=0)
 Provides a suitable allocation of device memory for the given size on the current device. More...
 
hipError_t DeviceFree (int device, void *d_ptr)
 Frees a live allocation of device memory on the specified device, returning it to the allocator. More...
 
hipError_t DeviceFree (void *d_ptr)
 Frees a live allocation of device memory on the current device, returning it to the allocator. More...
 
hipError_t FreeAllCached ()
 Frees all cached device allocations on all devices.
 
virtual ~CachingDeviceAllocator ()
 Destructor.
 

Static Public Member Functions

static unsigned int IntPow (unsigned int base, unsigned int exp)
 

Public Attributes

std::mutex mutex
 
unsigned int bin_growth
 Mutex for thread-safety.
 
unsigned int min_bin
 Geometric growth factor for bin-sizes.
 
unsigned int max_bin
 Minimum bin enumeration.
 
size_t min_bin_bytes
 Maximum bin enumeration.
 
size_t max_bin_bytes
 Minimum bin size.
 
size_t max_cached_bytes
 Maximum bin size.
 
const bool skip_cleanup
 Maximum aggregate cached bytes per device.
 
bool debug
 Whether or not to skip a call to FreeAllCached() when destructor is called. (The CUDA runtime may have already shut down for statically declared allocators)
 
GpuCachedBytes cached_bytes
 Whether or not to print (de)allocation events to stdout.
 
CachedBlocks cached_blocks
 Map of device ordinal to aggregate cached bytes on that device.
 
BusyBlocks live_blocks
 Set of cached device allocations available for reuse.
 

Static Public Attributes

static const unsigned int INVALID_BIN = (unsigned int) -1
 Out-of-bounds bin.
 
static const size_t INVALID_SIZE = (size_t) -1
 Invalid size.
 
static const int INVALID_DEVICE_ORDINAL = -1
 Invalid device ordinal.
 

Constructor & Destructor Documentation

◆ CachingDeviceAllocator() [1/2]

hipcub::CachingDeviceAllocator::CachingDeviceAllocator ( unsigned int  bin_growth,
unsigned int  min_bin = 1,
unsigned int  max_bin = INVALID_BIN,
size_t  max_cached_bytes = INVALID_SIZE,
bool  skip_cleanup = false,
bool  debug = false 
)
inline

Set of live device allocations currently in use.

Constructor.

Parameters
bin_growthGeometric growth factor for bin-sizes
min_binMinimum bin (default is bin_growth ^ 1)
max_binMaximum bin (default is no max bin)
max_cached_bytesMaximum aggregate cached bytes per device (default is no limit)
skip_cleanupWhether or not to skip a call to FreeAllCached() when the destructor is called (default is to deallocate)
debugWhether or not to print (de)allocation events to stdout (default is no stderr output)

◆ CachingDeviceAllocator() [2/2]

hipcub::CachingDeviceAllocator::CachingDeviceAllocator ( bool  skip_cleanup = false,
bool  debug = false 
)
inline

Default constructor.

Configured with:

  • bin_growth = 8
  • min_bin = 3
  • max_bin = 7
  • max_cached_bytes = (bin_growth ^ max_bin) * 3) - 1 = 6,291,455 bytes

which delineates five bin-sizes: 512B, 4KB, 32KB, 256KB, and 2MB and sets a maximum of 6,291,455 cached bytes per device

Member Function Documentation

◆ IntPow()

static unsigned int hipcub::CachingDeviceAllocator::IntPow ( unsigned int  base,
unsigned int  exp 
)
inlinestatic

Integer pow function for unsigned base and exponent

◆ NearestPowerOf()

void hipcub::CachingDeviceAllocator::NearestPowerOf ( unsigned int &  power,
size_t &  rounded_bytes,
unsigned int  base,
size_t  value 
)
inline

Round up to the nearest power-of

◆ SetMaxCachedBytes()

hipError_t hipcub::CachingDeviceAllocator::SetMaxCachedBytes ( size_t  max_cached_bytes)
inline

Sets the limit on the number bytes this allocator is allowed to cache per device.

Changing the ceiling of cached bytes does not cause any allocations (in-use or cached-in-reserve) to be freed. See FreeAllCached().

◆ DeviceAllocate() [1/2]

hipError_t hipcub::CachingDeviceAllocator::DeviceAllocate ( int  device,
void **  d_ptr,
size_t  bytes,
hipStream_t  active_stream = 0 
)
inline

Provides a suitable allocation of device memory for the given size on the specified device.

Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.

Parameters
[in]deviceDevice on which to place the allocation
[out]d_ptrReference to pointer to the allocation
[in]bytesMinimum number of bytes for the allocation
[in]active_streamThe stream to be associated with this allocation

◆ DeviceAllocate() [2/2]

hipError_t hipcub::CachingDeviceAllocator::DeviceAllocate ( void **  d_ptr,
size_t  bytes,
hipStream_t  active_stream = 0 
)
inline

Provides a suitable allocation of device memory for the given size on the current device.

Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.

Parameters
[out]d_ptrReference to pointer to the allocation
[in]bytesMinimum number of bytes for the allocation
[in]active_streamThe stream to be associated with this allocation

◆ DeviceFree() [1/2]

hipError_t hipcub::CachingDeviceAllocator::DeviceFree ( int  device,
void *  d_ptr 
)
inline

Frees a live allocation of device memory on the specified device, returning it to the allocator.

Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.

◆ DeviceFree() [2/2]

hipError_t hipcub::CachingDeviceAllocator::DeviceFree ( void *  d_ptr)
inline

Frees a live allocation of device memory on the current device, returning it to the allocator.

Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.


The documentation for this struct was generated from the following file:
  • /home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-hipcub/checkouts/docs-5.4.3/hipcub/include/hipcub/backend/cub/util_allocator.hpp