Kan Zhu

BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching

Yilong Zhao, Shuo Yang, Kan Zhu, Lianmin Zheng, Baris Kasikci, Yang Zhou, Jiarong Xing, Ion Stoica (2024). BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching. , DOI: https://doi.org/10.48550/arxiv.2411.16102.