arXiv · cs.AI · Zihao Zheng (Washington University in St. Louis), Junyi Yao (Washington University in St. Louis), Baichuan Li (Southern Methodist University), Jiayu Long (Washington University in St. Louis) · 10 小时前模型评分44
医疗 大语言模型（LLM，Large Language Model） 智能体升级可靠性审计：工作流状态失效的扰动控制基准
Auditing Escalation Reliability in Healthcare LLM Agents: A Perturbation-Controlled Benchmark of Workflow-State Failures
导读
一项针对医疗工作流 大语言模型（LLM，Large Language Model） 智能体的基准测试用 400 个合成医疗工作流片段，在 5 个 1.7B–9.7B 开源模型配置上评估执行、询问澄清、弃答与升级四类动作选择。
分类置信度
中 · 待人工复核
- 只有单一媒体来源，需要人工复核。
来源级别：媒体与个人
应用领域：医疗与生命科学
事件类型：研究发布
证据摘录
- “We instantiate an episode-level evaluation protocol as 400 synthetic healthcare workflow episodes: 40 hand-authored semantic seeds across four task classes, two surface parameterizations, and five controlled variants, scored on ANSWER/ASK CLARIFICATION/ABSTAIN/ESCALATE under a pre-declared 5:1 missed-versus-unnecessary escalation cost.”
- “Reference actions are single-author study-policy labels, not clinician-adjudicated ground truth.”
- “However, a 30-concept lexical stress test reduces each rule detector's recall from 1.00 on trigger phrases to 0.00 on unseen paraphrases, limiting the controller claim to this benchmark.”
来源：arXiv · cs.AI · arxiv.org