CUDA Streams#
2026-10-26
4 min read time
-
static cuda_stream_view cuda_stream_default = {}#
Static cuda_stream_view of the default stream (stream 0), for convenience.
-
static const cuda_stream_view cuda_stream_legacy{cudaStreamLegacy}#
Static cuda_stream_view of cudaStreamLegacy, for convenience.
-
static const cuda_stream_view cuda_stream_per_thread{cudaStreamPerThread}#
Static cuda_stream_view of cudaStreamPerThread, for convenience.
- bool operator==(
- cuda_stream_view lhs,
- cuda_stream_view rhs
Equality comparison operator for streams.
- Parameters:
lhs – The first stream view to compare
rhs – The second stream view to compare
- Returns:
true if equal, false if unequal
- bool operator!=(
- cuda_stream_view lhs,
- cuda_stream_view rhs
Inequality comparison operator for streams.
- Parameters:
lhs – The first stream view to compare
rhs – The second stream view to compare
- Returns:
true if unequal, false if equal
- std::ostream &operator<<(
- std::ostream &os,
- cuda_stream_view stream
Output stream operator for printing / logging streams.
- Parameters:
os – The output ostream
stream – The cuda_stream_view to output
- Returns:
std::ostream& The output ostream
-
class cuda_stream#
- #include <cuda_stream.hpp>
Owning wrapper for a CUDA stream.
Provides RAII lifetime semantics for a CUDA stream.
Public Types
Public Functions
- cuda_stream( ) = default#
Move constructor (default)
A moved-from cuda_stream is invalid and it is Undefined Behavior to call methods that access the owned stream.
- cuda_stream &operator=( ) = default#
Move copy assignment operator (default)
A moved-from cuda_stream is invalid and it is Undefined Behavior to call methods that access the owned stream.
- Returns:
A reference to this cuda_stream
- cuda_stream(
- cuda_stream::flags flags = cuda_stream::flags::sync_default
Construct a new CUDA stream object.
- Parameters:
flags – Stream creation flags.
- Throws:
rmm::cuda_error – if stream creation fails
- bool is_valid(
Returns true if the owned stream is non-null.
- Returns:
true If the owned stream has not been explicitly moved and is therefore non-null.
- Returns:
false If the owned stream has been explicitly moved and is therefore null.
- cudaStream_t value(
Get the value of the wrapped CUDA stream.
- Returns:
cudaStream_t The wrapped CUDA stream.
- explicit operator cudaStream_t(
Explicit conversion to cudaStream_t.
- cuda_stream_view view(
Creates an immutable, non-owning view of the wrapped CUDA stream.
- Returns:
rmm::cuda_stream_view The view of the CUDA stream
- operator cuda_stream_view(
Implicit conversion to cuda_stream_view.
- Returns:
A view of the owned stream
- operator cuda::stream_ref(
Implicit conversion to cuda::stream_ref.
- Returns:
A stream_ref for the owned stream
- void synchronize(
Synchronize the owned CUDA stream.
Calls
cudaStreamSynchronize().- Throws:
rmm::cuda_error – if stream synchronization fails
- void synchronize_no_throw(
Synchronize the owned CUDA stream. Does not throw if there is an error.
Calls
cudaStreamSynchronize()and asserts if there is an error.
-
class cuda_stream_pool#
- #include <cuda_stream_pool.hpp>
A pool of CUDA streams.
Provides efficient access to collection of CUDA stream objects.
Successive calls may return a
cuda_stream_viewof identical streams. For example, a possible implementation is to maintain a circular buffer ofcuda_streamobjects.Public Functions
- explicit cuda_stream_pool(
- std::size_t pool_size = default_size,
- cuda_stream::flags flags = cuda_stream::flags::sync_default
Construct a new CUDA stream pool object of the given non-zero size.
- Throws:
logic_error – if
pool_sizeis zero- Parameters:
pool_size – The number of streams in the pool
flags – Flags used when creating streams in the pool.
- rmm::cuda_stream_view get_stream(
Get a
cuda_stream_viewof a stream in the pool.This function is thread safe with respect to other calls to the same function.
- Returns:
- rmm::cuda_stream_view get_stream(
- std::size_t stream_id
Get a
cuda_stream_viewof the stream associated withstream_id. Equivalent values ofstream_idreturn a stream_view to the same underlying stream.This function is thread safe with respect to other calls to the same function.
Note
stream_idis wrapped around the pool size, therefore any size_t value is allowed.- Parameters:
stream_id – Unique identifier for the desired stream
- Returns:
- std::size_t get_pool_size(
Get the number of streams in the pool.
This function is thread safe with respect to other calls to the same function.
- Returns:
the number of streams in the pool
Public Static Attributes
-
static std::size_t default_size = {16}#
Default stream pool size.
-
class cuda_stream_view#
- #include <cuda_stream_view.hpp>
Strongly-typed non-owning wrapper for CUDA streams with default constructor.
This wrapper is simply a “view”: it does not own the lifetime of the stream it wraps.
Public Functions
- cuda_stream_view(
- cuda_stream_view const&
@default_copy_constructor
- cuda_stream_view( ) = default#
@default_move_constructor
- cuda_stream_view &operator=(
- cuda_stream_view const&
@default_copy_assignment{cuda_stream_view}
- cuda_stream_view &operator=( ) = default#
@default_move_assignment{cuda_stream_view}
- cuda_stream_view(
- cudaStream_t stream
Constructor from a cudaStream_t.
- Parameters:
stream – The underlying stream for this view
- cuda_stream_view(
- cuda::stream_ref stream
Implicit conversion from stream_ref.
- Parameters:
stream – The underlying stream for this view
- cudaStream_t value(
Get the wrapped stream.
- Returns:
cudaStream_t The underlying stream referenced by this cuda_stream_view
- operator cudaStream_t(
Implicit conversion to cudaStream_t.
- Returns:
cudaStream_t The underlying stream referenced by this cuda_stream_view
- operator cuda::stream_ref(
Implicit conversion to stream_ref.
- Returns:
stream_ref The underlying stream referenced by this cuda_stream_view
- bool is_per_thread_default(
@briefreturn{true if the wrapped stream is the CUDA per-thread default stream}
- bool is_default(
@briefreturn{true if the wrapped stream is explicitly the CUDA legacy default stream}
- void synchronize(
Synchronize the viewed CUDA stream.
Calls
cudaStreamSynchronize().- Throws:
rmm::cuda_error – if stream synchronization fails
- void synchronize_no_throw(
Synchronize the viewed CUDA stream. Does not throw if there is an error.
Calls
cudaStreamSynchronize()and asserts if there is an error.