arXiv · cs.AI · Tian Wang, Yongding Tao, Yihong Dong, Xiaoming Wu, Ying Guo, Ge Li · 11 小时前模型评分32
专家存在结构化冗余：面向混合专家大模型的保结构专家剪枝方法
Experts Exhibit Structured Redundancy: Structure-Preserving Expert Pruning for Mixture-of-Experts Large Language Models
导读
研究提出保结构专家剪枝框架，包含投影能量专家剪枝（PEEP，Projection Energy Expert Pruning）和分布覆盖专家剪枝（DCEP，Distributional Coverage Expert Pruning）两种方法，利用路由器权重作为先验，在保留专家间全局结构关系的同时兼顾单个专家重要性。
分类置信度
中 · 待人工复核
- 只有单一媒体来源，需要人工复核。
来源级别：媒体与个人
事件类型：研究发布
证据摘录
- “We propose two expert pruning methods under the unified structure-preserving framework: Projection Energy Expert Pruning (PEEP), which preserves the functional subspace by maximizing weighted projection energy, and Distributional Coverage Expert Pruning (DCEP), which preserves the distributional geometry by minimizing weighted coverage error.”
- “Experiments on DeepSeek-V2-Lite and Qwen3-30B-A3B demonstrate that, at the expert retention ratios of 75% and 50%, our methods outperform the state-of-the-art baselines on the majority of benchmarks and consistently achieve the best average performance across both general tasks and mathematical reasoning tasks.”
来源：arXiv · cs.AI · arxiv.org