high-performance-inference

Features and purpose

High-performance LLM inference with vLLM, quantization (AWQ, GPTQ, FP8), speculative decoding, and edge deployment. Use when optimizing inference latency, throughput, or memory.

Quick start

Follow the repository instructions for this skill and your agent. Review the files and dependencies before adding it to your workflow.