arXiv · cs.AI · Sangdo Han (CEO Business School), Moonsoo Kim (CEO Business School) · 3 小时前模型评分56
预注册研究：更换模型裁判会改变系统排名
Swapping the Judge Changes the Ranking: A Preregistered Study of Cross-Provider Replication in LLM-Judged System Comparisons
导读
一项预注册研究让来自三家厂商的五个裁判模型重新评判 4,000 场 Chatbot Arena 对战（九个系统、33 个系统对），每个对战都按两种回答顺序各评一次。
分类置信度
中 · 待人工复核
- 只有单一媒体来源，需要人工复核。
来源级别：媒体与个人
事件类型：研究发布
证据摘录
- “In a preregistered study (with a disclosed post-registration reduction of the judge set), five judge models from three providers re-judged 4,000 Chatbot Arena battles (nine systems, 33 system pairs) that carry human votes, each in both response orders.”
- “Code and judge verdicts are provided as ancillary files”
来源：arXiv · cs.AI · arxiv.org