arXiv · cs.AI · Anshu Raina, Yao Fu, Zhang Jiejing, Peyman Razaghi, Shiksha Patel, Ravi Gupta, Deval Shah, Adeem Jassani, Subhajit Dutta Chowdhury, Lorri Rao, Kailash Gogineni, Zhenyu Gu, Onil Gunawardana, Emad Barsoum · 5 小时前模型评分44
ROCm InferaSim：用少量测量预测 大语言模型（LLM，Large Language Model） 服务性能，降低配置试错成本
ROCm InferaSim: Cut LLM Serving Trial Costs and Deploy the Best Configuration
导读
ROCm InferaSim 通过少量“锚点”测量预测大语言模型（LLM，Large Language Model）在不同部署配置下的服务性能，避免逐项实测。
分类置信度
中 · 待人工复核
- 只有单一媒体来源，需要人工复核。
来源级别：媒体与个人
事件类型：研究发布
证据摘录
- “ROCm InferaSim predicts serving performance from a few short measurements.”
- “ROCm InferaSim benchmarks each distinct set of kernels once, on at most one node.”
- “On AMD Instinct MI355X GPUs with vLLM, SGLang, and ATOM, 11 anchors took 9.1 GPU-hours, against 204 GPU-hours to measure the 226 fixed-length configurations they cover.”
- “On 172 configurations, time per output token has 8.1% median error.”
- “For 159 disaggregated deployments, none used as an anchor, throughput is within 5.5%.”
来源：arXiv · cs.AI · arxiv.org