Memory Resources

2026-10-07

1 min read time

Applies to Linux and Windows

This page covers the memory resource extended API, providing a standard C++ interface for heterogeneous, stream-ordered memory allocation in HIP.

The <cuda/memory_resource> header provides a standard C++ interface for heterogeneous, stream-ordered memory allocation tailored to the needs of HIP C++ developers.

<cuda/memory_resource> only defines the memory allocation interface; it does not provide allocator implementations. For HIP developers, hipMM provides implementations of the cuda::mr interfaces optimized for AMD GPUs.

We are still experimenting with the design, so for now the contents of <cuda/memory_resource> are only available if LIBCUDACXX_ENABLE_EXPERIMENTAL_MEMORY_RESOURCE is defined.

At a high level, the header provides:

API	Description	Since
cuda::get_property	Infrastructure to tag a user defined type with a given property	libhipcxx 2.7
cuda::mr::{async}_resource and cuda::mr::{async}_resource_with	Concepts that provide proper constraints for arbitrary memory resources.	libhipcxx 2.7
cuda::mr::{async}_resource_ref	A non-owning type-erased memory resource wrapper that enables consumers to specify properties of resources that they expect.	libhipcxx 2.7

These features are an evolution of std::pmr::memory_resource that was introduced in C++17. While std::pmr::memory_resource provides a polymorphic memory resource that can be adopted through inheritance, it is not properly suited for heterogeneous systems.

With the current design it ranges from cumbersome to impossible to verify whether a memory resource provides allocations that are e.g. accessible on device, or whether it can utilize other allocation mechanisms.

To better support asynchronous HIP stream-ordered allocations libhipcxx provides cuda::stream_ref as a wrapper around hipStream_t. The definition of cuda::stream_ref can be found in the <cuda/stream_ref> header.