What is hip-ucxx?

Contents

What is hip-ucxx?#

2026-06-11

3 min read time

Applies to Linux

UCXX is a C++ wrapper library around UCX (Unified Communication X), providing a modern object-oriented API for high-performance inter-process and inter-node communication. UCX itself is a low-level C library for point-to-point networking that supports a wide range of transports (InfiniBand, RoCE, shared memory, TCP, and more). UCXX adds C++ abstractions on top of the UCX C library, making it significantly easier to build communication-intensive applications.

hip-ucxx is part of the AMD ROCm Data Science toolkit (ROCm-DS) and serves as the communication layer for distributed GPU workloads in the ROCm-DS ecosystem. hip-ucxx is the AMD ROCm-native port of the RAPIDS UCXX project. It has been adapted for the ROCm/HIP stack while preserving the directory structure, file naming, and API naming to minimize porting friction for developers migrating their workloads to AMD platforms.

Key highlights in hip-ucxx v0.1.0 include:

  • Full integration with the ROCm-DS ecosystem. hip-ucxx serves as the communication layer for distributed GPU computing across ROCm-DS components.

  • Multiple transport methods:

    • Tag matching: Tagged send/receive for message-based communication with sender-receiver coordination.

    • Stream: Ordered byte-stream communication over an endpoint.

    • Active messages (AM): Enables execution of user-defined callbacks on the receiver side upon message arrival.

    • Remote memory access (RMA/RDMA): Direct memory read/write operations across processes without involving the remote CPU.

  • GPU-direct communication enables GPU-to-GPU direct data transfers without staging through host memory, reducing latency and increasing throughput.

  • Python async API built on asyncio and a drop-in Dask Distributed backend (distributed-ucxx) that enables Dask to use UCX for inter-worker communication in distributed GPU computing pipelines.

  • Supports hipMM device buffers for efficient GPU memory management during transfers.

Modules#

The following table summarizes the C++ modules available in hip-ucxx:

Module

Headers

Description

Core

cpp/include/ucxx/context.h, worker.h, address.h, config.h, component.h

UCP context, worker, and address management. Configuration handling, base component class, and logging.

Communication

cpp/include/ucxx/endpoint.h, listener.h

UCP endpoint and listener management for establishing and accepting connections.

Request

cpp/include/ucxx/request_tag.h, request_stream.h, request_am.h, request_mem.h

Non-blocking send/receive operations via tag, stream, active message, and remote memory access APIs. Also includes flush and endpoint close requests.

Memory

cpp/include/ucxx/buffer.h, memory_handle.h, remote_key.h

Buffer types (host and hipMM device buffers), memory handle registration for RMA, and remote key management.

Async

cpp/include/ucxx/future.h, notifier.h, delayed_submission.h

Future-based asynchronous completion tracking, request notification, delayed request submission, and worker progress thread management.

Utilities

cpp/include/ucxx/typedefs.h, header.h, exception.h

Type definitions, multi-buffer header metadata, exception handling, and miscellaneous helper utilities (socket address, file descriptor, UCX wrappers).
Contents