Batch I/O#
batch-roundtrip.cpp reads a file into a registered GPU buffer using batch IO operations, writes the buffer back out in a second batch phase, and verifies that the output matches the input. 4KiB IOs are used and the batch queue is refilled by submitting additional batch IO operations as IO completions are received.
When to use this pattern#
Use batch I/O when you need to:
Amortize submission overhead across many small transfers instead of having per-request overhead.
Keep the storage device busy with a bounded number of outstanding requests.
Process completions as they arrive, in whatever order the device returns them.
Prerequisites#
Verify you have:
A working hipFile installation. See Install hipFile.
An AMD GPU with ROCm and hipFile support.
A file system that supports
O_DIRECT, for example, ext4 or XFS.The
examples_commonhelper library shipped with hipFile underexamples/common/.
Step-by-step walkthrough#
The walkthrough follows a complete file round trip, from creating the input file and registering resources to verifying the output and cleaning up. Reads and writes run in separate phases, each using a bounded window of 4 KiB requests that is refilled as completions arrive.
Parse arguments and seed the input file
The example takes the payload size and the batch capacity on the command line:
payload_size = parse_integral<size_t>(argv[3]); batch_capacity = parse_integral<unsigned>(argv[4]);
batch_capacitymust be between1and128.seed_read_file()then writespayload_sizebytes where byteiisi & 0xFFtoread_pathwith POSIXwrite(), replacing any prior file contents.Allocate and register one GPU buffer
const size_t buffer_size = align_up(payload_size, BATCH_IO_SIZE); hip_err = hipMalloc(&device_buffer, buffer_size); hipfile_err = hipFileBufRegister(device_buffer, buffer_size, 0);
The buffer holds the whole file rounded up to a 4 KiB boundary, so every request transfers a full
BATCH_IO_SIZEblock without a partial tail. All requests in both phases share this one registered buffer and address disjoint regions of it throughdevPtr_offset.Open input and output files
The
open_file()helper fromexamples_commoncallsopen()withO_DIRECTand thenhipFileHandleRegister(). Each phase uses one file handle for all of its requests.Create the batch context
hipfile_err = hipFileBatchIOSetUp(&batch_handle, batch_capacity);
batch_capacityis the maximum number of requests that can be in flight on this handle at one time. The same handle is reused for the read phase and the write phase. Each phase drains completely before the next one starts.Describe a request
configure_request()fills in onehipFileIOParams_tfor chunkchunk_index:request.operation.mode = hipFileBatch; request.operation.u.batch.devPtr_base = device_buffer; request.operation.u.batch.file_offset = static_cast<int64_t>(operation_offset); request.operation.u.batch.devPtr_offset = static_cast<int64_t>(operation_offset); request.operation.u.batch.size = BATCH_IO_SIZE; request.operation.fh = file_handle; request.operation.opcode = opcode; request.operation.cookie = request.cookie.get();
modemust behipFileBatch. Thecookieis an opaque pointer that hipFile hands back in the completion event. The example passes a pointer to theBatchCookieholding the chunk index and the byte count that chunk should transfer. Because the cookie is the only link between an event and its request, it must stay valid until that request completes.Reads at the tail of the file expect fewer than
BATCH_IO_SIZEbytes, soexpected_bytesis clamped to the bytes remaining in the payload. Writes always expect a full block because the padded buffer is written out in full and truncated afterwards.Fill the window and submit
run_batch_phase()starts with up tobatch_capacityconfigured requests and submits every request that isn’t already in flight:hipfile_err = hipFileBatchIOSubmit(batch_handle, submissions.size(), submissions.data(), /*flags=*/0);
hipFileBatchIOSubmit()takes an array of parameters, so one call can enqueue many requests. The number of in-flight requests plus the number being submitted must not exceed the capacity passed tohipFileBatchIOSetUp().Wait for completions
unsigned nr = batch_capacity; hipfile_err = hipFileBatchIOGetStatus(batch_handle, /*min_nr=*/1, &nr, events.data(), /*timeout=*/nullptr);
min_nris the minimum number of events to wait for.nris an in/out parameter that carries the event array capacity in and the number of events returned out. Anullptrtimeout blocks until at leastmin_nrevents are available. Passingmin_nrof1returns as soon as any request finishes, which is what lets the example refill the window promptly.Check each event and refill the window
For every returned event, the example finds the matching request by cookie, then checks both the status and the transferred byte count:
if (event.status != hipFileComplete) { /* request failed */ } if (event.ret != cookie->expected_bytes) { /* short transfer */ }
The
hipFileCompletestatus will be returned for successful requests.hipFileFailedorhipFileCanceledwill be returned for failed or canceled events.event.retis the byte count for that request, which can be short even when the status ishipFileComplete.If chunks remain, the completed slot is reconfigured for the next chunk index and resubmitted on the following loop iteration. Otherwise the slot is swapped to the end of the request vector and popped, shrinking the window. The phase ends when no request slots are left.
Finish the round trip
All reads complete before any writes are submitted, so the write phase always sees a fully populated buffer. After the write phase,
ftruncate()cuts the output file back from the 4 KiB-aligned size topayload_size, andverify_files_match()hashes the firstpayload_sizebytes of both files with FNV-1a and compares the digests. Matching hashes mean the round trip was lossless.Clean up resources
Teardown reverses setup:
hipFileBatchIODestroy(): destroy the batch context.close_file(): deregister and close each file handle.hipFileBufDeregister(): remove the GPU buffer from hipFile.hipFree(): free the device memory.
The flags
read_handle_open,write_handle_open, andbuffer_registeredtrack partial setup. If setup fails partway through, only resources that were created get torn down.
Completion ordering#
Batch requests carry no ordering guarantees:
Events can come back in any order, and one
hipFileBatchIOGetStatus()call can return anywhere frommin_nrtonrevents.Requests in a batch can overlap in time, so two requests must not modify the same file or buffer region.
Ordering between phases has to be enforced by the application. This example drains the read phase entirely before submitting any writes.
Increasing BATCH_SIZE widens the window and raises the number of concurrent
requests. Each request touches a disjoint 4 KiB region of the file and of the
buffer, so no cross-request coordination is needed.
Running the example#
./batch-roundtrip /path/to/readfile /path/to/writefile FILE_SIZE BATCH_SIZE [GPUID]
For example, a 4 MiB payload with 16 requests in flight:
./batch-roundtrip /path/to/readfile /path/to/writefile 4194304 16
On success, the program prints:
OK /path/to/readfile == /path/to/writefile (4194304 bytes, hash 0x...)