Developer Guide

Developer Guide#

Development environment#

ROCm is a base requirement for contributing to Tensile. To begin, ensure that ROCm is supported on your platform by reviewing the installation details on the ROCm documentation site.

Note

Environment setup steps are provided for Ubuntu/Debian platforms. For other operating systems, use the appropriate package manager, or your preferred installation method.

Developing in Docker#

ROCm development images are available on Docker Hub for a variety of OS/ROCm versions. See Docker images in the ROCm ecosystem for more details.

Setting up Python dependencies#

  1. Install OS dependencies (requires elevated privileges),

    apt-get install libyaml python3-yaml \
    libomp-dev libboost-program-options-dev libboost-filesystem-dev

  2. Install one of the following, depending on your preferred Tensile data format. If both are installed, msgpack is preferred,

    apt-get install libmsgpack-dev    # If using the msgpack backend
# OR
apt-get install libtinfo-dev      # If using the YAML backend

  3. Setup a virtual environment

    python3 -m venv .venv
source .venv/bin/activate

  4. Install Python dependencies

    pip3 install -r requirements.txt

  5. Confirm your dependencies match the following listing with pip3 freeze,

You can now run Tensile’s Python applications—see Tensile/bin.

Setting up C++ dependencies#

  1. Install ROCm for your platform (Linux or Windows).

    After the installation is complete, binaries and libraries can be found at /opt/rocm. ROCm comes packaged with compilers such as amdclang++, and other useful tools including rocminfo and rocprofv2.

    Tip

    If using Bash, we recommend setting PATH=/opt/rocm/bin/:$PATH in your ~/.bashrc and refreshing your shell, e.g., source ~/.bashrc. Alternatively, export the path only for your current shell session with export PATH=/opt/rocm/bin/:$PATH.

  2. Install build tools. Additional installation methods for the latest versions for CMake can be found here.

    apt-get install build-essential cmake

  3. Verify the versions of installed tools against the following table,

    Table 3 C++ build dependencies#

    Dependency

    Version

    amdclang++

    17.0+

    Make

    4.2+

    CMake

    3.16+

You can now run Tensile’s Host library tests.

Testing#

Tensile uses pytest to manage library/kernel tests. In particular, the project makes use of pytest markers to filter which tests are run. Important markers include pre_checkin, extended, integration, and unit—refer to pytest.ini for all supported markers.

In general, a test can be run via the tox ci environment by passing the desired test marker with -m <MARKER>,

tox run -e ci -- -m {pre_checkin|extended|integration|unit}

Note that -- is used to pass options to the underlying pytest command.

Note

By default tox run -e ci will run pre-checkin tests.

Unit tests and coverage reports#

Unit tests include all tests located under Tensile/Tests/unit/. A convenience command is included that adds coverage reporting,

tox run -e unittest
# OR for 32 processes
tox run -e unittest -- -n 32

By default, coverage results will be dumped to the terminal. To generate reports in other formats (e.g. HTML) use,

tox run -e unittest -- --cov-report=html

Files and directories excluded from coverage reporting are itemized in .coveragerc.

Although it is encouraged to run unit tests through tox to support consistency, they may also be run directly with pytest for quicker feedback, for example, to debug a run a single test named test_foo, the following command may be useful

Listing 1 From Tensile/Tests/#
pytest unit/test_TensileCreateLibrary.py -k "test_foo" --capture=no -v

Host library tests#

Host library tests ensure that generated libraries remain operational when being called from client code, e.g., other libraries or applications. These tests are built on gtest; to run them you must first download the submodule. From Tensile’s project root run,

git submodule update --init

Next, you can configure and build the host library tests through tox,

tox run -e hostlibtest

Note

Note that this tox command wraps invoke, a tool to manage CLI-invokable tasks. Since tox is, fundamentally, a Python environment manager and test runner, any reusable shell commands that fall outside its purview are managed by invoke (which are then sometimes encapsulated by tox). See tasks.py for more details.

You also can configure, build, and run host library tests directly with invoke,

invoke hostlibtest --configure --build --run

An executable TensileTests will be generate upon build, which can be used to run the tests.

If you wish to build and run the tests manually, checkout the commands in tasks.py. For advanced usage, like filtering or repeating test cases, see the gtest documentation.

Static analysis#

Python#

Use the top-level tox label static to run all static analysis, this may reformat your code, so be sure to commit your changes after running the command,

tox run -m static

Linting is evaluated with flake8, and formatting is conducted with black and isort. To run a check in isolation refer to tox.ini, or use one the following commands,

tox run -e lint
tox run -e format     # add `-- --check` to check formatting without applying changes
tox run -e isort      # add `-- --check` to check imports without applying changes

Tip

To ensure consistent formatting, we recommend setting up your editor to format on save using the same formatter settings as in tox.ini. Either way, ensuring you commit changes after running static analysis will reduce wait-times caused by simple CI failures.

C++#

Formatting is conducted with clang-format. For example, the following command will format all provided files, however, we recommend that you setup your editor to format on save.

clang-format -i style=file <files>

Styling rules are configured in .clang-format.

Profiling#

Python#

Profiling is enabled through the @profile decorator, and can be imported from the Tensile.Utilities.Profile module. Under the hood, the decorator wraps the function in a cProfile context, and generates a .prof file inside the profiling-results-<date> directory.

Note

Due to a current limitation with the profiling decorator, nested profiling is not supported, that is, if func1 calls func2 in a loop, and both are marked for profiling, the resulting .prof file for func1 will display incorrect results.

Documentation#

Tensile uses ROCm/rocm-docs-core as the documentation engine, which itself wraps Read the Docs and Sphinx.

You can build the documentation locally with,

tox run -e docs

After the documentation is built, the generated HTML files can be found at docs/_build/html.

Versioning#

Tensile follows semantic versioning practices, e.g., major.minor.patch. See server.org for more details.