MIVisionX

MIVisionX#

MIVisionX: Main Page
MIVisionX Documentation

MIT licensed doc Build Status

MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit. AMD MIVisionX delivers highly optimized conformant open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions along with Convolution Neural Net Model Compiler & Optimizer supporting ONNX, and Khronos NNEF™ exchange formats. The toolkit allows for rapid prototyping and deployment of optimized computer vision and machine learning inference workloads on a wide range of computer hardware, including small embedded x86 CPUs, APUs, discrete GPUs, and heterogeneous servers.

Latest Release

GitHub tag (latest SemVer)

Table of Contents

Documentation

Run the steps below to build documentation locally.

cd docs
pip3 install -r .sphinx/requirements.txt
python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html

AMD OpenVX™

AMD OpenVX™ is a highly optimized conformant open source implementation of the Khronos OpenVX™ 1.3 computer vision specification. It allows for rapid prototyping as well as fast execution on a wide range of computer hardware, including small embedded x86 CPUs and large workstation discrete GPUs.

Khronos OpenVX™ 1.0.1 conformant implementation is available in MIVisionX Lite

AMD OpenVX™ Extensions

The OpenVX framework provides a mechanism to add new vision functionality to OpenVX by vendors. This project has below mentioned OpenVX modules and utilities to extend amd_openvx, which contains the AMD OpenVX™ Core Engine.

  • amd_loomsl: AMD Radeon Loom stitching library for live 360 degree video applications
  • amd_media: vx_amd_media is an OpenVX AMD media extension module for encode and decode
  • amd_migraphx: amd_migraphx extension integrates the AMD's MIGraphx into an OpenVX graph. This extension allows developers to combine the vision funcions in OpenVX with the MIGraphX and build an end-to-end application for inference.
  • amd_nn: OpenVX neural network module
  • amd_opencv: OpenVX module that implements a mechanism to access OpenCV functionality as OpenVX kernels
  • amd_rpp: OpenVX extension providing an interface to some of the RPP's (ROCm Performance Primitives) functions. This extension is used to enable rocAL to perform image augmentation.
  • amd_winml: WinML extension will allow developers to import a pre-trained ONNX model into an OpenVX graph and add hundreds of different pre & post processing vision / generic / user-defined functions, available in OpenVX and OpenCV interop, to the input and output of the neural net model. This will allow developers to build an end to end application for inference.

Applications

MIVisionX has several applications built on top of OpenVX modules, it uses AMD optimized libraries to build applications that can be used to prototype or use as a model to develop products.

  • Bubble Pop: This sample application creates bubbles and donuts to pop using OpenVX & OpenCV functionality.
  • Cloud Inference Application: This sample application does inference using a client-server system.
  • Digit Test: This sample application is used to recognize hand written digits.
  • Image Augmentation: This sample application demonstrates the basic usage of rocAL's C API to load JPEG images from the disk and modify them in different possible ways and displays the output images.
  • MIVisionX Inference Analyzer: This sample application uses pre-trained ONNX / NNEF / Caffe models to analyze and summarize images.
  • MIVisionX OpenVX Classification: This sample application shows how to run supported pre-trained caffe models with MIVisionX RunTime.
  • MIVisionX Validation Tool: This sample application uses pre-trained ONNX / NNEF / Caffe models to analyze, summarize and validate models.
  • MIVisionX WinML Classification: This sample application shows how to run supported ONNX models with MIVisionX RunTime on Windows.
  • MIVisionX WinML YoloV2: This sample application shows how to run tiny yolov2(20 classes) with MIVisionX RunTime on Windows.
  • Optical Flow: This sample application creates an OpenVX graph to run Optical Flow on a video/live.
  • External Applications

Neural Net Model Compiler & Optimizer

Neural Net Model Compiler & Optimizer converts pre-trained neural net models to MIVisionX runtime code for optimized inference.

rocAL

The ROCm Augmentation Library - rocAL is designed to efficiently decode and process images and videos from a variety of storage formats and modify them through a processing graph programmable by the user.

Toolkit

MIVisionX Toolkit, is a comprehensive set of helpful tools for neural net creation, development, training, and deployment. The Toolkit provides you with helpful tools to design, develop, quantize, prune, retrain, and infer your neural network work in any framework. The Toolkit is designed to help you deploy your work to any AMD or 3rd party hardware, from embedded to servers.

MIVisionX provides you with tools for accomplishing your tasks throughout the whole neural net life-cycle, from creating a model to deploying them for your target platforms.

Utilities

  • inference_generator: generate inference library from pre-trained CAFFE models
  • loom_shell: an interpreter to prototype 360 degree video stitching applications using a script
  • RunVX: command-line utility to execute OpenVX graph described in GDF text file
  • RunCL: command-line utility to build, execute, and debug OpenCL programs

Prerequisites

Hardware

Operating System

Windows

  • Windows 10 / 11
  • Windows SDK
  • Visual Studio 2019 or later
  • Install the latest AMD drivers
  • Install OpenCL SDK
  • Install OpenCV 4.6.0
    • Set OpenCV_DIR environment variable to OpenCV/build folder
    • Add OpenCV_DIR%\x64\vc14\bin or OpenCV_DIR%\x64\vc15\bin to your PATH

macOS

Linux

  • Linux distribution
    • Ubuntu - 20.04 / 22.04
    • CentOS - 7 / 8
    • RedHat - 8 / 9
    • SLES - 15-SP4
  • Install ROCm
  • CMake 3.0 or later
  • ROCm MIOpen for Neural Net Extensions (vx_nn)
  • Qt Creator for Cloud Inference Client
  • Protobuf for inference generator & model compiler
    • install libprotobuf-dev and protobuf-compiler needed for vx_nn
  • OpenCV 4.6.0
    • Set OpenCV_DIR environment variable to OpenCV/build folder
  • FFMPEG n4.4.2
    • FFMPEG is required for amd_media & mv_deploy modules
  • rocAL Prerequisites

Prerequisites setup script for Linux

For the convenience of the developer, we provide the setup script MIVisionX-setup.py which will install all the dependencies required by this project.

NOTE: This script only needs to be executed once.

Prerequisites for running the script
python MIVisionX-setup.py --directory [setup directory - optional (default:~/)]
--opencv [OpenCV Version - optional (default:4.6.0)]
--protobuf [ProtoBuf Version - optional (default:3.12.4)]
--rpp [RPP Version - optional (default:1.0.0)]
--pybind11 [PyBind11 Version - optional (default:v2.10.4)]
--ffmpeg [FFMPEG V4.4.2 Installation - optional (default:ON) [options:ON/OFF]]
--rocal [MIVisionX rocAL Dependency Install - optional (default:ON) [options:ON/OFF]]
--neural_net[MIVisionX Neural Net Dependency Install - optional (default:ON) [options:ON/OFF]]
--inference [MIVisionX Neural Net Inference Dependency Install - optional (default:ON) [options:ON/OFF]]
--developer [Setup Developer Options - optional (default:ON) [options:ON/OFF]]
--reinstall [Remove previous setup and reinstall (default:OFF)[options:ON/OFF]]
--backend [MIVisionX Dependency Backend - optional (default:HIP) [options:HIP/OCL/CPU]]
--rocm_path [ROCm Installation Path - optional (default:/opt/rocm) - ROCm Installation Required]

Note:

  • ROCm upgrade with sudo apt upgrade requires the setup script rerun.
  • use X Window / X11 for remote GUI app control

Build & Install MIVisionX

Building on Windows

Using <tt>Visual Studio</tt>

  • Install Windows Prerequisites

  • Use MIVisionX.sln to build for x64 platform

    NOTE: vx_nn is not supported on Windows in this release

Building on macOS

macOS build instructions

Building on Linux

Using <tt>apt-get</tt> or <tt>yum</tt>

  • ROCm supported hardware
  • Install ROCm
  • On Ubuntu
    sudo apt-get install mivisionx

  • On CentOS

    sudo yum install mivisionx

    Note:

    • vx_winml is not supported on Linux
    • source code will not available with apt-get / yum install
    • the installer will copy
      • Executables into /opt/rocm/bin
      • Libraries into /opt/rocm/lib
      • OpenVX and module header files into /opt/rocm/include/mivisionx
      • Model compiler, & toolkit folders into /opt/rocm/libexec/mivisionx
      • Apps, & samples folder into /opt/rocm/share/mivisionx
      • Docs folder into /opt/rocm/share/doc/mivisionx
    • Package (.deb & .rpm) install requires OpenCV v3+ to execute AMD OpenCV extensions

Using MIVisionX-setup.py

  • Install ROCm
  • Use the below commands to set up and build MIVisionX
git clone https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX.git
cd MIVisionX

Note: MIVisionX has support for two GPU backends: OPENCL and HIP:

  • Instructions for building MIVisionX with the HIP GPU backend (i.e., default GPU backend):
    • run the setup script to install all the dependencies required by the HIP GPU backend: ``` python MIVisionX-setup.py ```
    • run the below commands to build MIVisionX with the HIP GPU backend: ``` mkdir build-hip cd build-hip cmake ../ make -j8 sudo cmake –build . –target PyPackageInstall sudo make install ``` Note:
    • PyPackageInstall used for rocal_pybind installation
    • rocal_pybind not supported on windows.
    • sudo required for pybind installation
  • Instructions for building MIVisionX with OPENCL GPU backend:
    • run the setup script to install all the dependencies required by the OPENCL GPU backend: ``` python MIVisionX-setup.py –reinstall ON –backend OCL ```
    • run the below commands to build MIVisionX with the OPENCL GPU backend: ``` mkdir build-ocl cd build-ocl cmake -DBACKEND=OPENCL ../ make -j8 sudo make install ```

Note:

  • MIVisionX cannot be installed for both GPU backends in the same default folder (i.e., /opt/rocm/) if an app interested in installing MIVisionX with both GPU backends, then add -DCMAKE_INSTALL_PREFIX in the cmake commands to install MIVisionX with OPENCL and HIP backends into two separate custom folders.
  • vx_winml is not supported on Linux
  • PyPackageInstall used for rocal_pybind installation - supported only with HIP Backend
  • rocal_pybind not supported on windows
  • sudo required for pybind installation

Verify the Installation

Verifying on Linux / macOS

  • The installer will copy
    • Executables into /opt/rocm/bin
    • Libraries into /opt/rocm/lib
    • OpenVX and OpenVX module header files into /opt/rocm/include/mivisionx
    • Apps, & Samples folder into /opt/rocm/share/mivisionx
    • Documents folder into /opt/rocm/share/doc/mivisionx
    • Model Compiler, and Toolkit folder into /opt/rocm/libexec/mivisionx

  • Run the below sample to verify the installation

    Canny Edge Detection

export PATH=$PATH:/opt/rocm/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/lib
runvx /opt/rocm/share/mivisionx/samples/gdf/canny.gdf

Note: More samples are available here

Note: For macOS use export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/opt/rocm/lib

Verifying on Windows

  • MIVisionX.sln builds the libraries & executables in the folder MIVisionX/x64
  • Use RunVX to test the build
./runvx.exe PATH_TO/MIVisionX/samples/gdf/skintonedetect.gdf

Docker

MIVisionX provides developers with docker images for Ubuntu 20.04 and CentOS 7 / 8. Using docker images developers can quickly prototype and build applications without having to be locked into a single system setup or lose valuable time figuring out the dependencies of the underlying software.

Docker files to build MIVisionX containers are available

MIVisionX Docker

Docker Workflow Sample on Ubuntu <tt>20.04</tt>

Sample Prerequisites

Sample Workflow

  • Step 1 - Install rocm-dkms
sudo apt update -y
sudo apt dist-upgrade -y
sudo apt install libnuma-dev wget
sudo reboot
wget https://repo.radeon.com/amdgpu-install/21.50/ubuntu/focal/amdgpu-install_21.50.50000-1_all.deb
sudo apt-get install -y ./amdgpu-install_21.50.50000-1_all.deb
sudo apt-get update -y
sudo amdgpu-install -y --usecase=rocm
sudo reboot
  • Step 2 - Setup Docker
sudo apt-get install curl
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
apt-cache policy docker-ce
sudo apt-get install -y docker-ce
sudo systemctl status docker
  • Step 3 - Get Docker Image
sudo docker pull mivisionx/ubuntu-20.04
  • Step 4 - Run the docker image
sudo docker run -it --device=/dev/kfd --device=/dev/dri --cap-add=SYS_RAWIO --device=/dev/mem --group-add video --network host mivisionx/ubuntu-20.04:latest

Note:

  • Map host directory on the docker image
    • map the localhost directory to be accessed on the docker image.
    • use -v option with docker run command: -v {LOCAL_HOST_DIRECTORY_PATH}:{DOCKER_DIRECTORY_PATH}
    • usage: ``` sudo docker run -it -v /home/:/root/hostDrive/ –device=/dev/kfd –device=/dev/dri –cap-add=SYS_RAWIO –device=/dev/mem –group-add video –network host mivisionx/ubuntu-20.04:latest ```
  • Display option with docker
    • Using host display ``` xhost +local:root sudo docker run -it –device=/dev/kfd –device=/dev/dri –cap-add=SYS_RAWIO –device=/dev/mem –group-add video –network host –env DISPLAY=unix$DISPLAY –privileged –volume $XAUTH:/root/.Xauthority –volume /tmp/.X11-unix/:/tmp/.X11-unix mivisionx/ubuntu-20.04:latest ```
    • Test display with MIVisionX sample ``` export PATH=$PATH:/opt/rocm/bin export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/lib runvx /opt/rocm/share/mivisionx/samples/gdf/canny.gdf ```

Technical Support

Please email [email protected] for questions, and feedback on MIVisionX.

Please submit your feature requests, and bug reports on the GitHub issues page.

Release Notes

Latest Release Version

GitHub tag (latest SemVer)

Changelog

Review all notable changes with the latest release

Tested configurations

  • Windows 10 / 11
  • Linux distribution
    • Ubuntu - 20.04 / 22.04
    • CentOS - 7 / 8
    • RHEL - 8 / 9
    • SLES - 15-SP4
  • ROCm: rocm-core - 5.4.3.50403-121
  • miopen-hip - 2.19.0.50403-121
  • miopen-opencl - 2.18.0.50300-63
  • migraphx - 2.4.0.50403-121
  • Protobuf - V3.12.4
  • OpenCV - 4.6.0
  • RPP - 1.2.0
  • FFMPEG - n4.4.2
  • Dependencies for all the above packages
  • MIVisionX Setup Script - V2.5.4

Known issues

  • OpenCV 4.X support for some apps missing
  • MIVisionX Package install requires manual prerequisites installation: some prerequisites for MIVisionX are build from source and do not have equivalent packages in linux. The user has to manually install these prerequisites. Instructions for these is given below.
    • Boost - Version 1.72.0 ``` wget https://boostorg.jfrog.io/artifactory/main/release/1.72.0/source/boost_1_72_0.tar.bz2 tar xjvf boost_1_72_0.tar.bz2 cd ./boost_1_72_0/ ./bootstrap.sh –prefix=/usr/local –with-python=python3 ./b2 stage -j16 threading=multi link=shared cxxflags="-std=c++11" sudo ./b2 install threading=multi link=shared –with-system –with-filesystem ./b2 stage -j16 threading=multi link=static cxxflags="-std=c++11 -fpic" cflags="-fpic" sudo ./b2 install threading=multi link=static –with-system –with-filesystem ``
    • OpenCV - Version4.6.0 `` wget https://github.com/opencv/opencv/archive/4.6.0.zip unzip 4.6.0.zip mkdir -p build/OpenCV cd build/OpenCV cmake -D WITH_GTK=ON -D WITH_JPEG=ON -D BUILD_JPEG=ON -D WITH_OPENCL=OFF -D WITH_OPENCLAMDFFT=OFF -D WITH_OPENCLAMDBLAS=OFF -D WITH_VA_INTEL=OFF -D WITH_OPENCL_SVM=OFF -D CMAKE_INSTALL_PREFIX=/usr/local ../../opencv-4.6.0 make -j8 make install ldconfig ```
    • FFMPEG - Version n4.4.2
    • Turbo JPEG installation - Version 2.0.6.1 ``` git clone -b 2.0.6.1 https://github.com/rrawther/libjpeg-turbo.git cd libjpeg-turbo && mkdir build && cd build cmake -DCMAKE_INSTALL_PREFIX=/usr \ -DCMAKE_BUILD_TYPE=RELEASE \ -DENABLE_STATIC=FALSE \ -DCMAKE_INSTALL_DOCDIR=/usr/share/doc/libjpeg-turbo-2.0.3 \ -DCMAKE_INSTALL_DEFAULT_LIBDIR=lib \ .. make -j$nproc && sudo make install ```

MIVisionX Dependency Map

HIP Backend

Docker Image: sudo docker build -f docker/ubuntu20/{DOCKER_LEVEL_FILE_NAME}.dockerfile -t {mivisionx-level-NUMBER} .

  • #c5f015 new component added to the level
  • #1589F0 existing component from the previous level
Build Level MIVisionX Dependencies Modules Libraries and Executables Docker Tag
Level_1 cmake
gcc
g++ 		amd_openvx
utilities 		#c5f015 libopenvx.so - OpenVX™ Lib - CPU
#c5f015 libvxu.so - OpenVX™ immediate node Lib - CPU
#c5f015 runvx - OpenVX™ Graph Executor - CPU with Display OFF 		Docker Image Version (tag latest semver)
Level_2 ROCm HIP
+Level 1 		amd_openvx
amd_openvx_extensions
utilities 		#c5f015 libopenvx.so - OpenVX™ Lib - CPU/GPU
#c5f015 libvxu.so - OpenVX™ immediate node Lib - CPU/GPU
#c5f015 runvx - OpenVX™ Graph Executor - Display OFF 		Docker Image Version (tag latest semver)
Level_3 OpenCV
FFMPEG
+Level 2 		amd_openvx
amd_openvx_extensions
utilities 		#1589F0 libopenvx.so - OpenVX™ Lib
#1589F0 libvxu.so - OpenVX™ immediate node Lib
#c5f015 libvx_amd_media.so - OpenVX™ Media Extension
#c5f015 libvx_opencv.so - OpenVX™ OpenCV InterOp Extension
#c5f015 mv_compile - Neural Net Model Compile
#c5f015 runvx - OpenVX™ Graph Executor - Display ON 		Docker Image Version (tag latest semver)
Level_4 MIOpenGEMM
MIOpen
ProtoBuf
+Level 3 		amd_openvx
amd_openvx_extensions
apps
utilities 		#1589F0 libopenvx.so - OpenVX™ Lib
#1589F0 libvxu.so - OpenVX™ immediate node Lib
#1589F0 libvx_amd_media.so - OpenVX™ Media Extension
#1589F0 libvx_opencv.so - OpenVX™ OpenCV InterOp Extension
#1589F0 mv_compile - Neural Net Model Compile
#1589F0 runvx - OpenVX™ Graph Executor - Display ON
#c5f015 libvx_nn.so - OpenVX™ Neural Net Extension 		Docker Image Version (tag latest semver)
Level_5 AMD_RPP
rocAL deps
+Level 4 		amd_openvx
amd_openvx_extensions
apps
rocAL
utilities 		#1589F0 libopenvx.so - OpenVX™ Lib
#1589F0 libvxu.so - OpenVX™ immediate node Lib
#1589F0 libvx_amd_media.so - OpenVX™ Media Extension
#1589F0 libvx_opencv.so - OpenVX™ OpenCV InterOp Extension
#1589F0 mv_compile - Neural Net Model Compile
#1589F0 runvx - OpenVX™ Graph Executor - Display ON
#1589F0 libvx_nn.so - OpenVX™ Neural Net Extension
#c5f015 libvx_rpp.so - OpenVX™ RPP Extension
#c5f015 librocal.so - Radeon Augmentation Library
#c5f015 rocal_pybind.so - rocAL Pybind Lib 		Docker Image Version (tag latest semver)

OpenCL Backend

Docker Image: sudo docker build -f docker/ubuntu20/{DOCKER_LEVEL_FILE_NAME}.dockerfile -t {mivisionx-level-NUMBER} .

  • #c5f015 new component added to the level
  • #1589F0 existing component from the previous level
Build Level MIVisionX Dependencies Modules Libraries and Executables Docker Tag
Level_1 cmake
gcc
g++ 		amd_openvx
utilities 		#c5f015 libopenvx.so - OpenVX™ Lib - CPU
#c5f015 libvxu.so - OpenVX™ immediate node Lib - CPU
#c5f015 runvx - OpenVX™ Graph Executor - CPU with Display OFF 		Docker Image Version (tag latest semver)
Level_2 ROCm OpenCL
+Level 1 		amd_openvx
amd_openvx_extensions
utilities 		#c5f015 libopenvx.so - OpenVX™ Lib - CPU/GPU
#c5f015 libvxu.so - OpenVX™ immediate node Lib - CPU/GPU
#c5f015 libvx_loomsl.so - Loom 360 Stitch Lib
#c5f015 loom_shell - 360 Stitch App
#c5f015 runcl - OpenCL™ program debug App
#c5f015 runvx - OpenVX™ Graph Executor - Display OFF 		Docker Image Version (tag latest semver)
Level_3 OpenCV
FFMPEG
+Level 2 		amd_openvx
amd_openvx_extensions
utilities 		#1589F0 libopenvx.so - OpenVX™ Lib
#1589F0 libvxu.so - OpenVX™ immediate node Lib
#1589F0 libvx_loomsl.so - Loom 360 Stitch Lib
#1589F0 loom_shell - 360 Stitch App
#1589F0 runcl - OpenCL™ program debug App
#c5f015 libvx_amd_media.so - OpenVX™ Media Extension
#c5f015 libvx_opencv.so - OpenVX™ OpenCV InterOp Extension
#c5f015 mv_compile - Neural Net Model Compile
#c5f015 runvx - OpenVX™ Graph Executor - Display ON 		Docker Image Version (tag latest semver)
Level_4 MIOpenGEMM
MIOpen
ProtoBuf
+Level 3 		amd_openvx
amd_openvx_extensions
apps
utilities 		#1589F0 libopenvx.so - OpenVX™ Lib
#1589F0 libvxu.so - OpenVX™ immediate node Lib
#1589F0 libvx_loomsl.so - Loom 360 Stitch Lib
#1589F0 loom_shell - 360 Stitch App
#1589F0 libvx_amd_media.so - OpenVX™ Media Extension
#1589F0 libvx_opencv.so - OpenVX™ OpenCV InterOp Extension
#1589F0 mv_compile - Neural Net Model Compile
#1589F0 runcl - OpenCL™ program debug App
#1589F0 runvx - OpenVX™ Graph Executor - Display ON
#c5f015 libvx_nn.so - OpenVX™ Neural Net Extension
#c5f015 inference_server_app - Cloud Inference App 		Docker Image Version (tag latest semver)
Level_5 AMD_RPP
rocAL deps
+Level 4 		amd_openvx
amd_openvx_extensions
apps
rocAL
utilities 		#1589F0 libopenvx.so - OpenVX™ Lib
#1589F0 libvxu.so - OpenVX™ immediate node Lib
#1589F0 libvx_loomsl.so - Loom 360 Stitch Lib
#1589F0 loom_shell - 360 Stitch App
#1589F0 libvx_amd_media.so - OpenVX™ Media Extension
#1589F0 libvx_opencv.so - OpenVX™ OpenCV InterOp Extension
#1589F0 mv_compile - Neural Net Model Compile
#1589F0 runcl - OpenCL™ program debug App
#1589F0 runvx - OpenVX™ Graph Executor - Display ON
#1589F0 libvx_nn.so - OpenVX™ Neural Net Extension
#1589F0 inference_server_app - Cloud Inference App
#c5f015 libvx_rpp.so - OpenVX™ RPP Extension
#c5f015 librocal.so - Radeon Augmentation Library
#c5f015 rocal_pybind.so - rocAL Pybind Lib 		Docker Image Version (tag latest semver)

NOTE: OpenVX and the OpenVX logo are trademarks of the Khronos Group Inc.