Use the Python API

Use the Python API#

The hipFile Python bindings let you perform GPU-accelerated file I/O from Python without writing C code. This page walks through each step of a typical workflow: opening the driver, registering buffers, reading and writing files, handling errors, and querying version and driver properties.

For a complete worked example that copies a file through GPU memory and verifies the result with SHA-256 hashes, see Perform GPU I/O with the Python bindings. For full API signatures and class details, see Python API reference.

Import hipFile#

Import the top-level hipfile package. All public classes, functions, enums, and exceptions are re-exported from the package root:

from hipfile import (
    Driver,
    FileHandle,
    Buffer,
    HipFileException,
    OpError,
    FileHandleType,
    get_version,
    driver_get_properties,
)

Open and close the driver#

The Driver class manages the hipFile driver lifecycle. Use it as a context manager so the driver closes automatically when the block exits:

with Driver() as drv:
    print(f"Driver reference count: {drv.use_count()}")
    # ... perform I/O inside this block ...

The driver is reference-counted internally. Each call to open() increments the count, and each call to close() decrements it. Driver.use_count() returns the current count as an integer.

You can also call open() and close() explicitly:

drv = Driver()
drv.open()
# ... perform I/O ...
drv.close()

Create and register a GPU buffer#

hipFile operates on GPU memory. After allocating device memory (for example, through ctypes bindings to hipMalloc), wrap the pointer in a Buffer and register it with the driver.

Buffer.from_ctypes_void_p accepts a ctypes.c_void_p, the buffer length in bytes, and registration flags (pass 0 for default behavior):

from hipfile.hipMalloc import hipMalloc, hipFree

size = 1024 * 1024  # 1 MiB
gpu_ptr = hipMalloc(size)

with Buffer.from_ctypes_void_p(gpu_ptr, size, 0) as buf:
    # buf is registered; use it for reads and writes
    ...

hipFree(gpu_ptr)

When used as a context manager, Buffer calls register() on entry and deregister() on exit. You can also call these methods directly:

buf = Buffer.from_ctypes_void_p(gpu_ptr, size, 0)
buf.register()
# ... use buf ...
buf.deregister()

If the ctypes.c_void_p is null, from_ctypes_void_p raises ValueError.

Open a file handle#

FileHandle wraps os.open together with hipFile handle registration. Pass a filesystem path, the os.open flags, and optionally a file mode and handle type:

import os

with FileHandle("/path/to/input.bin", os.O_RDONLY | os.O_DIRECT) as fh:
    bytes_read = fh.read(buf, size, file_offset=0, buffer_offset=0)

For writing, include os.O_WRONLY or os.O_RDWR with os.O_CREAT and os.O_DIRECT:

with FileHandle(
    "/path/to/output.bin",
    os.O_RDWR | os.O_DIRECT | os.O_CREAT | os.O_TRUNC,
) as fh_out:
    bytes_written = fh_out.write(buf, size, file_offset=0, buffer_offset=0)

The default file creation mode is 0o644. The default handle type is FileHandleType.OPAQUE_FD.

Read and write data#

FileHandle.read and FileHandle.write each take four positional arguments:

buffer

A registered Buffer instance.

size

Number of bytes to transfer.

file_offset

Byte offset within the file.

buffer_offset

Byte offset within the GPU buffer.

Both methods return the number of bytes transferred:

bytes_read = fh.read(buf, size, file_offset=0, buffer_offset=0)
bytes_written = fh.write(buf, size, file_offset=0, buffer_offset=0)

If a system-level I/O error occurs, an OSError is raised. If a hipFile or HIP driver error occurs, a HipFileException is raised.

Handle errors#

HipFileException wraps both the hipFileOpError_t code and the underlying hipError_t from the HIP runtime:

try:
    fh.read(buf, size, 0, 0)
except HipFileException as e:
    print(f"hipFile error code: {e.hipfile_err}")
    print(f"HIP runtime error code: {e.hip_err}")
    print(f"Description: {e}")
hipfile_err

The hipFileOpError_t integer code describing the failure.

hip_err

The hipError_t integer code from the HIP runtime. This value is only meaningful when hipfile_err equals OpError.HIP_DRIVER_ERROR.

The string representation of the exception includes the error description returned by the C library.

Classify errors with OpError#

The OpError enum mirrors every value from hipFileOpError_t. Use it to match specific error conditions:

from hipfile import OpError

try:
    fh.read(buf, size, 0, 0)
except HipFileException as e:
    if e.hipfile_err == OpError.DIO_NOT_SET:
        print("O_DIRECT was not set on the file descriptor.")
    elif e.hipfile_err == OpError.HIP_DRIVER_ERROR:
        print(f"HIP runtime error: {e.hip_err}")

Query version and driver properties#

get_version() returns a (major, minor, patch) tuple:

major, minor, patch = get_version()
print(f"hipFile version: {major}.{minor}.{patch}")

driver_get_properties() returns a dictionary of driver property names mapped to their integer values:

props = driver_get_properties()
for name, value in props.items():
    print(f"{name}: {value}")

Both functions raise HipFileException if the underlying C call fails.

Contents