Usecases#

Refer to the applicable guides to optimize specific usecase performance.

Usecases

• vLLM
  • vLLM Docker image for Llama2 and Llama3
    • Prerequisites
    • Download and install Docker image
  • GEMM tuning for model inferencing with vLLM
    • Collect GEMM shape details
    • Conduct GEMM tuning
    • Run vLLM inference with tuned GEMM