/home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-rocal/checkouts/develop/rocAL_pybind/amd/rocal/pipeline.py File Reference

• rocAL_pybind
• amd
• rocal

Data Structures | Functions

pipeline.py File Reference

File containing the Pipeline class containing the pybind API functions. More...

Data Structures
class	rocAL_pybind.amd.rocal.pipeline.Pipeline
	Pipeline class internally calls RocalCreate which returns context which will have all the info set by the user. More...

Functions
def	rocAL_pybind.amd.rocal.pipeline.pipeline_def (fn=None, **pipeline_kwargs)
	Decorator that converts a graph definition function into a rocAL pipeline factory. More...

Detailed Description

File containing the Pipeline class containing the pybind API functions.

Function Documentation

◆ pipeline_def()

def rocAL_pybind.amd.rocal.pipeline.pipeline_def	(		fn = None,
		**	pipeline_kwargs
	)

Decorator that converts a graph definition function into a rocAL pipeline factory.

A graph definition function is a function that returns intended pipeline outputs. You can decorate this function with @pipeline_def::

@pipeline_def
def my_pipe(flip_vertical, flip_horizontal):
    ''' Creates a rocAL pipeline, which returns flipped and original images '''
    data, _ = fn.readers.file(file_root=images_dir)
    img = fn.decoders.image(data, device="mixed")
    flipped = fn.flip(img, horizontal=flip_horizontal, vertical=flip_vertical)
    return flipped, img

The decorated function returns a rocAL Pipeline object::

pipe = my_pipe(True, False)
# pipe.build()  # the pipeline is not configured properly yet

A pipeline requires additional parameters such as batch size, number of worker threads, GPU device id and so on (see :meth:amd.rocal.Pipeline() for a complete list of pipeline parameters). These parameters can be supplied as additional keyword arguments, passed to the decorated function::

pipe = my_pipe(True, False, batch_size=32, num_threads=1, device_id=0)
pipe.build()  # the pipeline is properly configured, we can build it now

The outputs from the original function became the outputs of the Pipeline::

flipped, img = pipe.run()

When some of the pipeline parameters are fixed, they can be specified by name in the decorator::

@pipeline_def(batch_size=42, num_threads=3)
def my_pipe(flip_vertical, flip_horizontal):
    ...

Any Pipeline constructor parameter passed later when calling the decorated function will override the decorator-defined params::

@pipeline_def(batch_size=32, num_threads=3)
def my_pipe():
    data = fn.external_source(source=my_generator)
    return data

pipe = my_pipe(batch_size=128)  # batch_size=128 overrides batch_size=32

.. warning::

The arguments of the function being decorated can shadow pipeline constructor arguments -
in which case there's no way to alter their values.

.. note::

Using ``**kwargs`` (variadic keyword arguments) in graph-defining function is not allowed.
They may result in unwanted, silent hijacking of some arguments of the same name by
Pipeline constructor. Code written this way would cease to work with future versions of rocAL
when new parameters are added to the Pipeline constructor.

To access any pipeline arguments within the body of a @pipeline_def function, the function :meth:amd.rocal.Pipeline.current() can be used:: ( note: this is not supported yet)

@pipeline_def()
def my_pipe():
    pipe = Pipeline.current()
    batch_size = pipe.batch_size
    num_threads = pipe.num_threads
    ...

pipe = my_pipe(batch_size=42, num_threads=3)
...