Use Spack

Applies to Linux

2024-09-26

14 min read time

Introduction to Spack

Spack is a package management tool designed to support multiple software versions and configurations on a wide variety of platforms and environments. It was designed for large supercomputing centers, where many users share common software installations on clusters with exotic architectures using libraries that do not have a standard ABI. Spack is non-destructive: installing a new version does not break existing installations, so many configurations can coexist on the same system.

Most importantly, Spack is simple. It offers a simple spec syntax, so users can concisely specify versions and configuration options. Spack is also simple for package authors: package files are written in pure Python, and specs allow package authors to maintain a single file for many different builds of the same package.

Installing prerequisites for Spack

Note

You must install all prerequisites before installing Spack.

Ubuntu

# Install some essential utilities:
apt-get update
apt-get install make patch bash tar gzip unzip bzip2 file gnupg2 git gawk
apt-get update -y
apt-get install -y xz-utils
apt-get install build-essential
apt-get install vim
# Install Python:
apt-get install python3
apt-get upgrade python3-pip
# Install Compilers:
apt-get install gcc
apt-get install gfortran

SLES

# Install some essential utilities:
zypper update
zypper install make patch bash tar gzip unzip bzip xz file gnupg2 git awk
zypper in -t pattern
zypper install vim
# Install Python:
zypper install python3
zypper install python3-pip
# Install Compilers:
zypper install gcc
zypper install gcc-fortran
zypper install gcc-c++

Building ROCm components using Spack

1. To use the Spack package manager, clone the Spack project from spack/spack.

   git clone https://github.com/spack/spack.git
   

2. Initialize Spack.

   The setup-env.sh script initializes the Spack environment.

   cd spack
   . share/spack/setup-env.sh
   

   Spack commands are available once the above steps are completed. To list the available commands, use help.

   root@computername:~$ spack help
   

ROCm packages in Spack

Component	Package name
rocm-cmake	rocm-cmake
thunk	hsakmt-roct
rocm-smi-lib	rocm-smi-lib
hsa	hsa-rocr-dev
lightning	llvm-amdgpu
devicelibs	rocm-device-libs
comgr	comgr
rocclr (vdi)	hip-rocclr
hipify_clang	hipify-clang
hip (hip_in_vdi)	hip
ocl (opencl_on_vdi )	rocm-opencl
rocminfo	rocminfo
clang-ocl	rocm-clang-ocl
rccl	rccl
rocm_debug_agent	rocm-debug-agent
rocm_bandwidth_test	rocm-bandwidth-test
rocprofiler	rocprofiler-dev
roctracer-dev-api	roctracer-dev-api
roctracer	roctracer-dev
dbgapi	rocm-dbgapi
rocm-gdb	rocm-gdb
openmp-extras	rocm-openmp-extras
rocBLAS	rocblas
hipBLAS	hipblas
rocFFT	rocfft
rocRAND	rocrand
rocSPARSE	rocsparse
hipSPARSE	hipsparse
rocALUTION	rocalution
rocSOLVER	rocsolver
rocPRIM	rocprim
rocThrust	rocthrust
hipCUB	hipcub
hipfort	hipfort
ROCmValidationSuite	rocm-validation-suite
MIOpen(Hip variant)	miopen-hip
MIOpen(opencl)	miopen-opencl
MIVisionX	mivisionx
AMDMIGraphX	migraphx
rocm-tensile	rocm-tensile
hipfft	hipfft
RDC	rdc
hipsolver	hipsolver
mlirmiopen	mlirmiopen

Installing ROCm components using Spack

1. rocm-cmake

   Install the default variants and the latest version of rocm-cmake.

   spack install rocm-cmake
   

   To install a specific version of rocm-cmake, use:

   spack install rocm-cmake@<version number>
   

   For example, spack install [email protected]

2. info

   The info command displays basic package information. It shows the preferred, safe, and deprecated versions, in addition to the available variants. It also shows the dependencies with other packages.

   spack info mivisionx
   

   For example:

   root@computername:~$ spack info mivisionx
   CMakePackage: mivisionx
   
   Description:
   MIVisionX toolkit is a set of comprehensive computer vision and machine
   intelligence libraries, utilities, and applications bundled into a
   single toolkit.
   
   Homepage: <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX>
   
   Preferred version:
   5.3.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-5.3.0.tar.gz>
   
   Safe versions:
   5.3.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-5.3.0.tar.gz>
   5.2.3 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-5.2.3.tar.gz>
   5.2.1 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-5.2.1.tar.gz>
   5.2.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-5.2.0.tar.gz>
   5.1.3 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-5.1.3.tar.gz>
   5.1.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-5.1.0.tar.gz>
   5.0.2 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-5.0.2.tar.gz>
   5.0.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-5.0.0.tar.gz>
   4.5.2 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-4.5.2.tar.gz>
   4.5.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-4.5.0.tar.gz>
   
   Deprecated versions:
   4.3.1 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-4.3.1.tar.gz>
   4.3.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-4.3.0.tar.gz>
   4.2.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-4.2.0.tar.gz>
   4.1.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-4.1.0.tar.gz>
   4.0.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-4.0.0.tar.gz>
   3.10.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-3.10.0.tar.gz>
   3.9.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-3.9.0.tar.gz>
   3.8.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-3.8.0.tar.gz>
   3.7.0 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/rocm-3.7.0.tar.gz>
   1.7 <https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/archive/1.7.tar.gz>
   
   Variants:
   Name [Default] When Allowed values Description
   ==================== ==== ==================== ==================
   
   build_type [Release] -- Release, Debug, CMake build type
   RelWithDebInfo
   hip [on] -- on, off Use HIP as backend
   ipo [off] -- on, off CMake interprocedural optimization
   opencl [off] -- on, off Use OPENCL as the backend
   
   Build Dependencies:
   cmake ffmpeg libjpeg-turbo miopen-hip miopen-opencl miopengemm opencv openssl protobuf rocm-cmake rocm-opencl
   
   Link Dependencies:
   miopen-hip miopen-opencl miopengemm openssl rocm-opencl
   
   Run Dependencies:
   None
   

Installing variants for ROCm components

The variants listed above indicate that the mivisionx package is built by default with build_type=Release and the hip backend, and without the opencl backend. build_type=Debug and RelWithDebInfo, with opencl and without hip, are also supported.

For example:

spack install mivisionx build_type=Debug (Backend will be hip since it is the default one)
spack install mivisionx+opencl build_type=Debug (Backend will be opencl and hip will be disabled as per the conflict defined in recipe)

• spack spec command

  To display the dependency tree, the spack spec command can be used with the same format.

  For example:

  root@computername:~$ spack spec mivisionx
  Input spec
  --------------------------------
  mivisionx
  
  Concretized
  --------------------------------
  [email protected]%[email protected]+hip\~ipo\~opencl build_type=Release arch=linux-ubuntu20.04-skylake_avx512
  

Creating an environment

You can create an environment with all the required components of your version.

1. In the root folder, create a new folder when you can create a .yaml file. This file is used to create an environment.

   mkdir /localscratch
   cd /localscratch
   vi sample.yaml
   

2. Add all the required components in the sample.yaml file:

   spack:
   concretization: separately
   packages:
   all:
   compiler: [[email protected]]
   specs:
   - matrix:
   - ['%[email protected]\^[email protected]']
   - [[email protected], [email protected], [email protected], [email protected],
   [email protected], [email protected], [email protected], [email protected], [email protected],
   [email protected], [email protected], [email protected], [email protected], [email protected],
   [email protected], [email protected]]
   view: true
   

3. Once you’ve created the .yaml file, you can use it to create an environment.

   spack env create -d /localscratch/MyEnvironment /localscratch/sample.yaml
   

4. Activate the environment.

   spack env activate /localscratch/MyEnvironment
   

5. Verify that you want all the component versions.

   spack find # this command will list out all components been in the environment (and 0 installed )
   

6. Install all the components in the .yaml file.

   cd /localscratch/MyEnvironment
   spack install -j 50
   

7. Check that all components are successfully installed.

   spack find
   

8. If any modification is made to the .yaml file, you must deactivate the existing environment and create a new one in order for the modifications to be reflected.

   To deactivate, use:

   spack env deactivate
   

Creating and applying a patch before installation

Spack installs ROCm packages after pulling the source code from GitHub and building it locally. In order to build a component with any modification to the source code, you must generate a patch and apply it before the build phase.

To generate a patch and build with the changes:

1. Stage the source code.

   spack stage [email protected] # (This will pull the 5.2.0 release version source code of hip and display the path to spack-src directory where entire source code is available)
   
   root@computername:~/spack$ spack stage [email protected]
   ==> Fetching <https://github.com/ROCm-Developer-Tools/HIP/archive/rocm-5.2.0.tar.gz>
   ==> Fetching <https://github.com/ROCm-Developer-Tools/hipamd/archive/rocm-5.2.0.tar.gz>
   ==> Fetching <https://github.com/ROCm-Developer-Tools/ROCclr/archive/rocm-5.2.0.tar.gz>
   ==> Moving resource stage
   source: /tmp/root/spack-stage/resource-hipamd-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src/
   destination: /tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src/hipamd
   ==> Moving resource stage
   source: /tmp/root/spack-stage/resource-opencl-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src/
   destination: /tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src/opencl
   ==> Moving resource stage
   source: /tmp/root/spack-stage/resource-rocclr-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src/
   destination: /tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src/rocclr
   ==> Staged hip in /tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7
   

2. Change directory to spack-src inside the staged directory.

   root@computername:~/spack$ cd /tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7
   root@computername:/tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7$ cd spack-src/
   

3. Create a new Git repository.

   root@computername:/tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src$ git init
   

4. Add the entire directory to the repository.

   root@computername:/tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src$ git add .
   

5. Make the required changes to the source code.

   root@computername:/tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src$ vi hipamd/CMakeLists.txt
   (Make required changes in the source code)
   

6. Generate the patch using the git diff command.

   diff > /spack/var/spack/repos/builtin/packages/hip/0001-modifications.patch
   

7. Update the recipe with the patch file name and any conditions you want to apply.

   root@computername:/tmp/root/spack-stage/spack-stage-hip-5.2.0-wzo5y6ysvmadyb5mvffr35galb6vjxb7/spack-src$ spack edit hip
   

8. Provide the patch file name and the conditions for the patch.

   patch("0001-modifications.patch", when="@5.2.0")
   

   Spack applies 0001-modifications.patch on the 5.2.0 release code before starting the hip build.

9. After each modification, you must update the recipe. If there is no change to the recipe, run

   touch /spack/var/spack/repos/builtin/packages/hip/package.py