1 package âĸ â 77,587 total stars
A high-throughput and memory-efficient inference and serving engine for LLMs
