AMD RunVX#
RunVX is a command-line tool to execute OpenVX graphs, with a simple, easy-to-use interface. It encapsulates most of the routine OpenVX calls, thus speeding up development and enabling rapid prototyping. As input, RunVX takes a GDF (Graph Description Format) file, a simple and intuitive syntax to describe the various data, nodes, and their dependencies. The tool has other useful features, such as, file read/write, data compares, image and keypoint data visualization, etc.
If available, this project uses OpenCV for camera capture and image display.
Prerequisites#
CPU: AMD64
GPU: AMD Radeon™ Graphics [optional]
Windows: install the latest drivers and OpenCL SDK download [optional]
Linux: install ROCm [optional]
OpenCV
3.4+[optional]Windows:SetOpenCV_DIRenvironment variable toOpenCV/buildfolder
Install
libssl-devon Linux [optional]
RunVX Usage and GDF Syntax#
runvx.exe [options] <file.gdf> [argument(s)]
runvx.exe [options] node <kernelName> [argument(s)]
runvx.exe [options] shell [argument(s)]
The argument(s) are data objects created using <data-description> syntax.
These arguments can be accessed from inside GDF as $1, $2, etc.
The available command-line options are:
-h
Show full help.
-v
Turn on verbose logs.
-root:<directory>
Replace ~ in filenames with <directory> in the command-line and
GDF file. The default value of '~' is current working directory.
-frames:[<start>:]<end>|eof|live
Run the graph/node for specified frames or until eof or just as live.
Use live to indicate that input is live until aborted by user.
-affinity:CPU|GPU[<device-index>]
Set context affinity to CPU or GPU.
-dump-profile
Print performance profiling information after graph launch.
-discard-compare-errors
Continue graph processing even if compare mismatches occur.
-disable-virtual
Replace all virtual data types in GDF with non-virtual data types.
Use of this flag (i.e. for debugging) can make a graph run slower.
-dump-data-config:<dumpFilePrefix>,<object-type>[,object-type[...]]
Automatically dump all non-virtual objects of specified object types
into files '<dumpFilePrefix>dumpdata_####_<object-type>_<object-name>.raw'
-discard-commands:<cmd>[,cmd[...]]
Discard the listed commands.
The supported list of OpenVX built-in kernel names is given below:
org.khronos.openvx.color_convert
org.khronos.openvx.channel_extract
org.khronos.openvx.channel_combine
org.khronos.openvx.sobel_3x3
org.khronos.openvx.magnitude
org.khronos.openvx.phase
org.khronos.openvx.scale_image
org.khronos.openvx.table_lookup
org.khronos.openvx.histogram
org.khronos.openvx.equalize_histogram
org.khronos.openvx.absdiff
org.khronos.openvx.mean_stddev
org.khronos.openvx.threshold
org.khronos.openvx.integral_image
org.khronos.openvx.dilate_3x3
org.khronos.openvx.erode_3x3
org.khronos.openvx.median_3x3
org.khronos.openvx.box_3x3
org.khronos.openvx.gaussian_3x3
org.khronos.openvx.custom_convolution
org.khronos.openvx.gaussian_pyramid
org.khronos.openvx.accumulate
org.khronos.openvx.accumulate_weighted
org.khronos.openvx.accumulate_square
org.khronos.openvx.minmaxloc
org.khronos.openvx.convertdepth
org.khronos.openvx.canny_edge_detector
org.khronos.openvx.and
org.khronos.openvx.or
org.khronos.openvx.xor
org.khronos.openvx.not
org.khronos.openvx.multiply
org.khronos.openvx.add
org.khronos.openvx.subtract
org.khronos.openvx.warp_affine
org.khronos.openvx.warp_perspective
org.khronos.openvx.harris_corners
org.khronos.openvx.fast_corners
org.khronos.openvx.optical_flow_pyr_lk
org.khronos.openvx.remap
org.khronos.openvx.halfscale_gaussian
The supported list of AMD's built-in kernel names is given below:
com.amd.nn_extension.argmax_layer
com.amd.nn_extension.batch_normalization_layer
com.amd.nn_extension.cast_layer
com.amd.nn_extension.concat_layer
com.amd.nn_extension.crop_layer
com.amd.nn_extension.crop_and_resize_layer
com.amd.nn_extension.detection_output
com.amd.nn_extension.permute_layer
com.amd.nn_extension.prior_box_layer
com.amd.nn_extension.scale_layer
com.amd.nn_extension.slice_layer
com.amd.nn_extension.convert_image_to_tensor
com.amd.nn_extension.convert_tensor_to_image
com.amd.nn_extension.tensor_exp
com.amd.nn_extension.tensor_log
com.amd.nn_extension.upsample_nearest_layer
com.amd.nn_extension.reshape_layer
com.amd.nn_extension.gather_layer
com.amd.nn_extension.topk_layer
com.amd.nn_extension.nms_layer
The available GDF commands are:
import <libraryName>
Import kernels in a library using vxLoadKernel API.
type <typeName> userstruct:<size-in-bytes>
Create an OpenVX user defined structure using vxRegisterUserStruct API.
The <typeName> can be used as a type in array object.
data <dataName> = <data-description>
Create an OpenVX data object in context using the below syntax for
<data-description>:
array:<data-type>,<capacity>
convolution:<columns>,<rows>
distribution:<numBins>,<offset>,<range>
delay:<exemplar>,<slots>
image:<width>,<height>,<image-format>[,<range>][,<space>]
uniform-image:<width>,<height>,<image-format>,<uniform-pixel-value>
image-from-roi:<master-image>,rect{<start-x>;<start-y>;<end-x>;<end-y>}
image-from-handle:<image-format>,{<dim-x>;<dim-y>;<stride-x>;<stride-y>}[+...],<memory-type>
image-from-channel:<master-image>,<channel>
lut:<data-type>,<count>
matrix:<data-type>,<columns>,<rows>
pyramid:<numLevels>,half|orb|<scale-factor>,<width>,<height>,<image-format>
remap:<srcWidth>,<srcHeight>,<dstWidth>,<dstHeight>
scalar:<data-type>,<value>
threshold:<thresh-type>,<input-data-type><output-data-type>
tensor:<num-of-dims>,{<dim0>,<dim1>,...},<data-type>,<fixed-point-pos>
tensor-from-roi:<master-tensor>,<num-of-dims>,{<start0>,<start1>,...},{<end0>,<end1>,...}
tensor-from-handle:<num-of-dims>,{<dim0>,<dim1>,...},<data-type>,<fixed-point-pos>,{<stride0>,<stride1>,...},<num-alloc-handles>,<memory-type>
For virtual object in default graph use the below syntax for
<data-description>:
virtual-array:<data-type>,<capacity>
virtual-image:<width>,<height>,<image-format>
virtual-pyramid:<numLevels>,half|orb|<scale-factor>,<width>,<height>,<image-format>
virtual-tensor:<num-of-dims>,{<dim0>,<dim1>,...},<data-type>,<fixed-point-pos>
where:
<master-image> can be name of a image data object (including $1, $2, ...)
<master-tensor> can be name of a tensor data object (including $1, $2, ...)
<exemplar> can be name of a data object (including $1, $2, ...)
<thresh-type> can be BINARY,RANGE
<uniform-pixel-value> can be an integer or {<byte>;<byte>;...}
<image-format> can be RGB2,RGBX,IYUV,NV12,U008,S016,U001,F032,...
<data-type> can be UINT8,INT16,INT32,UINT32,FLOAT32,ENUM,BOOL,SIZE,
KEYPOINT,COORDINATES2D,RECTANGLE,<typeName>,...
<range> can be vx_channel_range_e enums FULL or RESTRICTED
<space> can be vx_color_space_e enums BT709 or BT601_525 or BT601_625
node <kernelName> [<argument(s)>]
Create a node of specified kernel in the default graph with specified node arguments.
Node arguments have to be OpenVX data objects created earlier in GDF or data objects
specified on command-line accessible as $1, $2, etc. For scalar enumerations as node
arguments, use !<enumName> syntax (e.g., !VX_CHANNEL_Y for channel_extract node).
For optional arguments, use NULL/null to set the node argument as a null value.
(e.g. node com.amd.nn_extension.slice_layer input output starts ends NULL steps)
include <file.gdf>
Specify inclusion of another GDF file.
shell
Start a shell command session.
set <option> [<value>]
Specify or query the following global options:
set verbose [on|off]
Turn on/off verbose option.
set frames [[<start-frame>:]<end-frame>|eof|live|default]
Specify input frames to be processed. Here are some examples:
set frames 10 # process frames 0 through 9
set frames 1:10 # process frames 1 through 9
set frames eof # process all frames till end-of-file
set frames live # input is live until terminated by user
set frames default # process all frames specified on input
set dump-profile [on|off]
Turn on/off profiler output.
set wait [key|<milliseconds>]
Specify wait time between frame processing to give extra time
for viewing. Or wait for key press between frames.
set compare [on|off|discard-errors]
Turn on/off data compares or just discard data compare errors.
set use-schedule-graph [on|off]
Turn on/off use of vxScheduleGraph instead of vxProcessGraph.
set dump-data-config [<dumpFilePrefix>,<obj-type>[,<obj-type>[...]]]
Specify dump data config for portion of the graph. To disable
don't specify any config.
graph <command> [<arguments> ...]
Specify below graph specific commands:
graph auto-age [<delayName> [<delayName> ...]]
Make the default graph use vxAgeDelay API for the specified
delay objects after processing each frame.
graph affinity [CPU|GPU[<device-index>]]
Specify graph affinity to CPU or GPU.
graph save-and-reset <graphName>
Verify the default graph and save it as <graphName>. Then
create a new graph as the default graph. Note that the earlier
virtual data object won't be available after graph reset.
graph reset [<graphName(s)>]
Reset the default or specified graph(s). Note that the earlier
virtual data object won't be available after graph reset.
graph launch [<graphName(s)>]
Launch the default or specified graph(s).
graph info [<graphName(s)>]
Show graph details for debug.
rename <dataNameOld> <dataNameNew>
Rename a data object\n"
init <dataName> <initial-value>
Initialize data object with specified value.
- convolution object initial values can be:
{<value1>;<value2>;...<valueN>}
scale{<scale>}
- matrix object initial values can be:
{<value1>;<value2>;...<valueN>}
- remap object initial values can be:
dst is same as src: same
dst is 90 degree rotation of src: rotate-90
dst is 180 degree rotation of src: rotate-180
dst is 270 degree rotation of src: rotate-270
dst is horizontal flip of src: hflip
dst is vertical flip of src: vflip
- threshold object initial values can be:
For VX_THRESHOLD_TYPE_BINARY: <value>
For VX_THRESHOLD_TYPE_RANGE: {<lower>;<upper>}
- image object initial values can be:
Binary file with image data. For images created from handle,
the vxSwapHandles API will be invoked before executing the graph.
- tensor object initial values can be:
Binary file with tensor data.
To replicate a file multiple times, use @repeat~N~<fileName>.
To fill the tensor with a value, use @fill~f32~<float-value>,
@fill~i32~<int-value>, @fill~i16~<int-value>, or @fill~u8~<uint-value>.
read <dataName> <fileName> [ascii|binary] [<option(s)>]
Read frame-level data from the specified <fileName>.
- images can be read from containers (such as, .jpg, .avi, .mp4, etc.)
as well as raw binary files
- certain raw data formats support reading data for all frames from a
single file (such as, video.yuv, video.rgb, video.avi etc.)
The data objects that support this feature are image, scalar, and
threshold data objects.
- certain data formats support printf format-syntax (e.g., joy_%04d.yuv)
to read individual data from separate files. Note that scalar and
threshold data objects doesn't support this feature. Also note that
pyramid objects expect all frames of each level in separate files.
- convolution objects support the option: scale
This will read scale value as the first 32-bit integer in file(s).
write <dataName> <fileName> [ascii|binary] [<option(s)>]
Write frame-level data to the specified <fileName>.
- certain raw data formats support writing data for all frames into a
single file (such as, video.yuv, video.rgb, video.u8, etc.)
The data objects that support this feature are image, scalar, and
threshold data objects.
- certain data formats support printf format-syntax (e.g., joy_%04d.yuv)
to write individual data from separate files. Note that scalar and
threshold data objects doesn't support this feature. Also note that
pyramid objects expect all frames of each level in separate files.
- convolution objects support the option: scale
This will write scale value as the first 32-bit integer in file(s).
compare <dataName> <fileName> [ascii|binary] [<option(s)>]
Compare frame-level data from the specified <fileName>.
- certain raw data formats support comparing data for all frames from a
single file (such as, video.yuv, video.rgb, video.u8, etc.)
The data objects that support this feature are image, scalar, and
threshold data objects.
- certain data formats support printf format-syntax (e.g., joy_%04d.yuv)
to read individual data from separate files. Note that scalar and
threshold data objects doesn't support this feature.
- array objects with VX_TYPE_KEYPOINT data type support the options:
specify tolerance: err{<x>;<y>;<strength>[;<%mismatch>]}
specify compare log file: log{<fileName>}
- array objects with VX_TYPE_COORDINATES2D data type support the options:
specify tolerance: err{<x>;<y>[;<%mismatch>]}
specify compare log file: log{<fileName>}
- convolution objects support the option:
read scale value as the first 32-bit integer in file(s): scale
- image and pyramid objects support the options:
specify compare region: rect{<start-x>;<start-y>;<end-x>;<end-y>}
specify valid pixel difference: err{<min>;<max>}
specify pixel checksum to compare: checksum
specify generate checksum: checksum-save-instead-of-test
- matrix objects support the options:
specify tolerance: err{<tolerance>}
- remap objects support the options:
specify tolerance: err{<x>;<y>}
- scalar objects support the option:
specify that file specifies inclusive range of valid values: range
view <dataName> <windowName>
Display frame-level data in a window with title <windowName>. Each window
can display an image data object and optionally additional other data
objects overlaid on top of the image.
- supported data object types are: array, distribution, image, lut,
scalar, and delay.
- display of array, distribution, lut, and scalar objects are
overlaid on top of an image with the same <windowName>.
- delay object displays reference in the slot#0 of current time.
directive <dataName> <directive>
Specify a directive to data object. Only a few directives are supported:
- Use sync-cl-write directive to issue VX_DIRECTIVE_AMD_COPY_TO_OPENCL
directive whenever data object is updated using init or read commands.
Supported for array, image, lut, and remap data objects only.
- Use readonly directive to issue VX_DIRECTIVE_AMD_READ_ONLY directive
that informs the OpenVX framework that object won't be updated after
init command. Supported for convolution and matrix data objects only.
pause
Wait until a key is pressed before processing next GDF command.
help [command]
Show the GDF command help.
exit
Exit from shell or included GDF file.
quit
Abort the application.
Examples#
Here are few examples that demonstrate use of RUNVX prototyping tool.
Canny Edge Detector#
This example demonstrates building OpenVX graph for Canny edge detector. Use face1.jpg for this example.
% runvx[.exe] file canny.gdf
File canny.gdf:
# create input and output images
data input = image:480,360,RGB2
data output = image:480,360,U008
# specify input source for input image and request for displaying input and output images
read input examples/images/face1.jpg
view input inputWindow
view output edgesWindow
# compute luma image channel from input RGB image
data yuv = image-virtual:0,0,IYUV
data luma = image-virtual:0,0,U008
node org.khronos.openvx.color_convert input yuv
node org.khronos.openvx.channel_extract yuv !CHANNEL_Y luma
# compute edges in luma image using Canny edge detector
data hyst = threshold:RANGE,U008,U008:INIT,80,100
data gradient_size = scalar:INT32,3
node org.khronos.openvx.canny_edge_detector luma hyst gradient_size !NORM_L1 output
Skintone Pixel Detector#
This example demonstrates building OpenVX graph for pixel-based skin tone detector [Peer et al. 2003]. Use face1.jpg for this example.
% runvx[.exe] file skintonedetect.gdf
File skintonedetect.gdf:
# create input and output images
data input = image:480,360,RGB2
data output = image:480,360,U008
# specify input source for input image and request for displaying input and output images
read input examples/images/face1.jpg
view input inputWindow
view output skintoneWindow
# threshold objects
data thr95 = threshold:BINARY,U008,U008:INIT,95 # threshold for computing R > 95
data thr40 = threshold:BINARY,U008,U008:INIT,40 # threshold for computing G > 40
data thr20 = threshold:BINARY,U008,U008:INIT,20 # threshold for computing B > 20
data thr15 = threshold:BINARY,U008,U008:INIT,15 # threshold for computing R-G > 15
data thr0 = threshold:BINARY,U008,U008:INIT,0 # threshold for computing R-B > 0
# virtual image objects for intermediate results
data R = image-virtual:0,0,U008
data G = image-virtual:0,0,U008
data B = image-virtual:0,0,U008
data RmG = image-virtual:0,0,U008
data RmB = image-virtual:0,0,U008
data R95 = image-virtual:0,0,U008
data G40 = image-virtual:0,0,U008
data B20 = image-virtual:0,0,U008
data RmG15 = image-virtual:0,0,U008
data RmB0 = image-virtual:0,0,U008
data and1 = image-virtual:0,0,U008
data and2 = image-virtual:0,0,U008
data and3 = image-virtual:0,0,U008
# extract R,G,B channels and compute R-G and R-B
node org.khronos.openvx.channel_extract input !CHANNEL_R R # extract R channel
node org.khronos.openvx.channel_extract input !CHANNEL_G G # extract G channel
node org.khronos.openvx.channel_extract input !CHANNEL_B B # extract B channel
node org.khronos.openvx.subtract R G !SATURATE RmG # compute R-G
node org.khronos.openvx.subtract R B !SATURATE RmB # compute R-B
# compute threshold
node org.khronos.openvx.threshold R thr95 R95 # compute R > 95
node org.khronos.openvx.threshold G thr40 G40 # compute G > 40
node org.khronos.openvx.threshold B thr20 B20 # compute B > 20
node org.khronos.openvx.threshold RmG thr15 RmG15 # compute RmG > 15
node org.khronos.openvx.threshold RmB thr0 RmB0 # compute RmB > 0
# aggregate all thresholded values to produce SKIN pixels
node org.khronos.openvx.and R95 G40 and1 # compute R95 & G40
node org.khronos.openvx.and and1 B20 and2 # compute B20 & and1
node org.khronos.openvx.and RmG15 RmB0 and3 # compute RmG15 & RmB0
node org.khronos.openvx.and and2 and3 output # compute and2 & and3 as output
Feature Tracker#
The feature tracker example demonstrates building an application with two separate graphs that uses Harris Corners and Optical Flow kernels. This example requires use of delay data objects that contain multiple pyramid and array objects. Use PETS09-S1-L1-View001.avi as input video sequence.
% runvx[.exe] file feature_tracker.gdf
File feature_tracker.gdf:
# create image object for the input video sequence.
data input = image:768,576,RGB2
read input PETS09-S1-L1-View001.avi
# create output keypoint array objects inside a delay object with two slots.
# two slots are needed to keep track current keypoints from previous time.
data exemplarArr = array:KEYPOINT,10000 # max trackable keypoints are 10,000
data delayArr = delay:exemplarArr,2 # two slots inside the delay object
# request for displaying input with keypoints from delay slot[0].
view input feature_tracker
view delayArr feature_tracker
# create pyramid objects inside a delay object with two slots.
# two slots of pyramids are needed for optical flow kernel.
data exemplarPyr = pyramid:6,half,768,576,U008
data delayPyr = delay:exemplarPyr,2
# create first graph to initialize keypoints using Harris Corners and
# compute pyramid for by Optical Flow later using another graph
data iyuv = image-virtual:0,0,IYUV
data luma = image-virtual:0,0,U008
data strength_thresh = scalar:FLOAT32,0.0005
data min_distance = scalar:FLOAT32,5.0
data sensitivity = scalar:FLOAT32,0.04
data grad_size = scalar:INT32,3
data block_size = scalar:INT32,3
node org.khronos.openvx.color_convert input iyuv
node org.khronos.openvx.channel_extract iyuv !CHANNEL_Y luma
node org.khronos.openvx.harris_corners luma strength_thresh min_distance sensitivity \
grad_size block_size delayArr[0] null
node org.khronos.openvx.gaussian_pyramid luma delayPyr[0]
# request vxAgeDelay call for delay objects after each frame with
# current graph and save current graph with the name "harris"
graph auto-age delayPyr delayArr
graph save-and-reset harris
# create second graph to track keypoints using Optical Flow assuming that
# pyramid/keypoints in delay objects have been initialized with previous frame
data iyuv = image-virtual:0,0,IYUV
data luma = image-virtual:0,0,U008
data termination = scalar:ENUM,CRITERIA_BOTH
data epsilon = scalar:FLOAT32,0.01
data num_iterations = scalar:UINT32,5
data use_initial_estimate = scalar:BOOL,0
data window_dimension = scalar:SIZE,6
node org.khronos.openvx.color_convert input iyuv
node org.khronos.openvx.channel_extract iyuv !CHANNEL_Y luma
node org.khronos.openvx.gaussian_pyramid luma delayPyr[0]
node org.khronos.openvx.optical_flow_pyr_lk delayPyr[-1] delayPyr[0] \
delayArr[-1] delayArr[-1] delayArr[0] \
termination epsilon num_iterations \
use_initial_estimate window_dimension
# request vxAgeDelay call for delay objects after each frame with
# current graph and save current graph with the name "opticalflow"
graph auto-age delayPyr delayArr
graph save-and-reset opticalflow
# launch "harris" graph to process first frame in the video sequence
set frames 1
graph launch harris
# launch "opticalflow" graph to process remaining frames in the video sequence
set frames default
graph launch opticalflow