arXiv · cs.AI · Zhichen Zeng, Xichong Zhang, Junpan Wu, Yifei Zuo, Chi-Chih Chang, Jiayi Wang, Maohua Nie, Ji Liu, Ang Li, Banghua Zhu · 7 小时前模型评分45
Step-dLLM：面向扩散大语言模型高效推理的自适应步感知稀疏注意力
Step-dLLM: Adaptive Step-aware Sparse Attention for Efficient Diffusion LLM Inference
导读
Step-dLLM 通过按去噪轨迹在锚点步定期刷新块稀疏掩码、并在局部一致区间内复用，同时按注意力集中程度为各注意力头分配自适应预算，实现扩散大语言模型（dLLM，Diffusion Large Language Model）的高效推理。
分类置信度
中 · 待人工复核
- 存在未解决的相邻类别：ai-tooling，需要人工复核。
- 只有单一媒体来源，需要人工复核。
来源级别：媒体与个人
事件类型：研究发布
证据摘录
- “To this end, we design Step-dLLM, which periodically refreshes a block-sparse mask at anchor steps along the denoising trajectory and reuses it within each locally consistent interval, while employing a head-adaptive global budget that more aggressively compresses heads with concentrated attention.”
- “Step-dLLM applies to pre-trained LLaDA-1.5 and Dream-7B-Instruct without retraining”
- “achieving a 2.78x end-to-end speedup over dense FlashAttention on LLaDA-1.5 at 8K context with 256 denoising steps and 90% sparsity”
来源：arXiv · cs.AI · arxiv.org