Supported Primitives Guide#
This document contains details of supported primitives in Composable Kernel (CK). In contrast to the API Reference Guide, the Supported Primitives Guide is an introduction to the math which underpins the algorithms implemented in CK.
Softmax#
For vectors
where
For a matrix
For
If
Else if
Update running max, running sum and column blocks
to
Initialize column block
of output and reset running max and running sum variables: