Magma Installation for ROCm#

Applies to Linux


6 min read time


Matrix Algebra on GPU and Multi-core Architectures, abbreviated as MAGMA, is a collection of next-generation dense linear algebra libraries that is designed for heterogeneous architectures, such as multiple GPUs and multi- or many-core CPUs.

MAGMA provides implementations for CUDA, HIP, Intel Xeon Phi, and OpenCL™. For more information, refer to

Using MAGMA for PyTorch#

Tensor is fundamental to Deep Learning techniques because it provides extensive representational functionalities and math operations. This data structure is represented as a multidimensional matrix. MAGMA accelerates tensor operations with a variety of solutions including driver routines, computational routines, BLAS routines, auxiliary routines, and utility routines.

Build MAGMA from Source#

To build MAGMA from the source, follow these steps:

  1. In the event you want to compile only for your uarch, use:

    export PYTORCH_ROCM_ARCH=<uarch>

    <uarch> is the architecture reported by the rocminfo command.

  2. Use the following:

    export PYTORCH_ROCM_ARCH=<uarch>
    # "install" hipMAGMA into /opt/rocm/magma by copying after build
    git clone
    pushd magma
    # Fixes memory leaks of magma found while executing linalg UTs
    git checkout 5959b8783e45f1809812ed96ae762f38ee701972
    echo 'LIBDIR += -L$(MKLROOT)/lib' >>
    echo 'LIB += -Wl,--enable-new-dtags -Wl,--rpath,/opt/rocm/lib -Wl,--rpath,$(MKLROOT)/lib -Wl,--rpath,/opt/rocm/magma/lib' >>
    echo 'DEVCCFLAGS += --gpu-max-threads-per-block=256' >>
    export PATH="${PATH}:/opt/rocm/bin"
    if [[ -n "$PYTORCH_ROCM_ARCH" ]]; then
      amdgpu_targets=`echo $PYTORCH_ROCM_ARCH | sed 's/;/ /g'`
      amdgpu_targets=`rocm_agent_enumerator | grep -v gfx000 | sort -u | xargs`
    for arch in $amdgpu_targets; do
      echo "DEVCCFLAGS += --amdgpu-target=$arch" >>
    # hipcc with openmp flag may cause isnan() on __device__ not to be found; depending on context, compiler may attempt to match with host definition
    sed -i 's/^FOPENMP/#FOPENMP/g'
    make -f make.gen.hipMAGMA -j $(nproc)
    LANG=C.UTF-8 make lib/ -j $(nproc) MKLROOT=/opt/conda
    make testing/testing_dgemm -j $(nproc) MKLROOT=/opt/conda
    mv magma /opt/rocm


C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” CoRR, p. abs/1512.00567, 2015

PyTorch, [Online]. Available:

PyTorch, [Online]. Available:

Stanford, [Online]. Available:

Wikipedia, [Online]. Available:

AMD, “ROCm issues,” [Online]. Available: RadeonOpenCompute/ROCm#issues

PyTorch, [Online image].

TensorFlow, [Online image].

MAGMA, [Online image].

Advanced Micro Devices, Inc., [Online]. Available:

Advanced Micro Devices, Inc., [Online]. Available: ROCmSoftwarePlatform/AMDMIGraphX

Docker, [Online].

Torchvision, [Online]. Available