/home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-rocal/checkouts/docs-6.2.2/rocAL_pybind/amd/rocal/pipeline.py File Reference#
File containing the Pipeline class containing the pybind API functions. More...
Data Structures | |
class | rocAL_pybind.amd.rocal.pipeline.Pipeline |
Pipeline class internally calls RocalCreate which returns context which will have all the info set by the user. More... | |
Functions | |
def | rocAL_pybind.amd.rocal.pipeline.pipeline_def (fn=None, **pipeline_kwargs) |
Decorator that converts a graph definition function into a rocAL pipeline factory. More... | |
Detailed Description
File containing the Pipeline class containing the pybind API functions.
Function Documentation
◆ pipeline_def()
def rocAL_pybind.amd.rocal.pipeline.pipeline_def | ( | fn = None , |
|
** | pipeline_kwargs | ||
) |
Decorator that converts a graph definition function into a rocAL pipeline factory.
A graph definition function is a function that returns intended pipeline outputs. You can decorate this function with @pipeline_def
::
@pipeline_def def my_pipe(flip_vertical, flip_horizontal): ''' Creates a rocAL pipeline, which returns flipped and original images ''' data, _ = fn.readers.file(file_root=images_dir) img = fn.decoders.image(data, device="mixed") flipped = fn.flip(img, horizontal=flip_horizontal, vertical=flip_vertical) return flipped, img
The decorated function returns a rocAL Pipeline object::
pipe = my_pipe(True, False) # pipe.build() # the pipeline is not configured properly yet
A pipeline requires additional parameters such as batch size, number of worker threads, GPU device id and so on (see :meth:amd.rocal.Pipeline()
for a complete list of pipeline parameters). These parameters can be supplied as additional keyword arguments, passed to the decorated function::
pipe = my_pipe(True, False, batch_size=32, num_threads=1, device_id=0) pipe.build() # the pipeline is properly configured, we can build it now
The outputs from the original function became the outputs of the Pipeline::
flipped, img = pipe.run()
When some of the pipeline parameters are fixed, they can be specified by name in the decorator::
@pipeline_def(batch_size=42, num_threads=3) def my_pipe(flip_vertical, flip_horizontal): ...
Any Pipeline constructor parameter passed later when calling the decorated function will override the decorator-defined params::
@pipeline_def(batch_size=32, num_threads=3) def my_pipe(): data = fn.external_source(source=my_generator) return data pipe = my_pipe(batch_size=128) # batch_size=128 overrides batch_size=32
.. warning::
The arguments of the function being decorated can shadow pipeline constructor arguments - in which case there's no way to alter their values.
.. note::
Using ``**kwargs`` (variadic keyword arguments) in graph-defining function is not allowed. They may result in unwanted, silent hijacking of some arguments of the same name by Pipeline constructor. Code written this way would cease to work with future versions of rocAL when new parameters are added to the Pipeline constructor.
To access any pipeline arguments within the body of a @pipeline_def
function, the function :meth:amd.rocal.Pipeline.current()
can be used:: ( note: this is not supported yet)
@pipeline_def() def my_pipe(): pipe = Pipeline.current() batch_size = pipe.batch_size num_threads = pipe.num_threads ... pipe = my_pipe(batch_size=42, num_threads=3) ...