arXiv · cs.AI · Annan Zhang, Shengjie Song, Wenshuai Yao, Lihao Zhang, Tianmiao · 8 小时前模型评分39
用共享梯度边界守护 Adam 优化器状态压缩
Guarded Compression of Adam Optimizer State with Shared-Gradient Bounds
导读
研究提出“守护式”Adam 优化器状态压缩：只有当解码后的动量能在一个梯度区间内复现 FP32 原生更新、且误差在学习率单位下不超过容差 τ 时，才接受该坐标的压缩值，否则原样存储。
分类置信度
中 · 待人工复核
- 只有单一媒体来源，需要人工复核。
来源级别：媒体与个人
事件类型：研究发布
证据摘录
- “We study guarded moment compression, in which a coordinate's decoded moments are accepted only if they reproduce the native FP32 Adam update to within a tolerance $\tau$, in units of the learning rate, for every gradient in a clipped domain; all other coordinates are stored exactly.”
- “The guarantee covers the first restored update under a pinned Torch operation model; we measure subsequent training empirically.”
来源：arXiv · cs.AI · arxiv.org