Composable Kernel User Guide

Introduction

This document contains instructions for installing, using, and contributing to Composable Kernel (CK).

Methodology

Composable Kernel (CK) library aims to provide a programming model for writing performance critical kernels for machine learning workloads across multiple architectures including GPUs, CPUs, etc, through general purpose kernel languages, like HIP C++.

CK utilizes two concepts to achieve performance portability and code maintainability:

• A tile-based programming model

• Algorithm complexity reduction for complex ML operators, using innovative technique we call “Tensor Coordinate Transformation”.

Code Structure

Current CK library are structured into 4 layers:

• “Templated Tile Operators” layer

• “Templated Kernel and Invoker” layer

• “Instantiated Kernel and Invoker” layer

• “Client API” layer

Documentation Roadmap

The following is a list of CK documents in the suggested reading order:

Contents:

• 1. CK Hello world
  • 1.1. Motivation
  • 1.2. Description
  • 1.3. Hardware targets
  • 1.4. Build the library
  • 1.5. Run examples and tests
  • 1.6. Summary
• 2. CK docker hub
  • 2.1. Why do I need this?
  • 2.2. So what is Composable Kernel?
  • 2.3. And what is inside?
  • 2.4. Which image is right for me?
  • 2.5. DIY starts here
  • 2.6. License
• 3. Supported Primitives Guide
  • 3.1. Softmax
• 4. API Reference Guide
  • 4.1. Introduction
  • 4.2. Using CK API
  • 4.3. CK Datatypes
    • 4.3.1. DeviceMem
      • DeviceMem
    • 4.3.2. Kernels For Flashattention
      • GridwiseBatchedGemmSoftmaxGemm_Xdl_CShuffle
      • ThreadGroupTensorSliceTransfer_v4r1
      • BlockwiseGemmXdlops_v2
      • BlockwiseSoftmax
      • ThreadwiseTensorSliceTransfer_StaticToStatic
• 5. Contributor’s Guide
  • 5.1. Pull-request guidelines