Artificial Analysis推文
MLCR-AA：医疗长上下文推理基准测试排行榜发布
Artificial Analysis 发布了针对 Wisedocs 的 MLCR（医疗长上下文推理）基准的排行榜 MLCR-AA，测试模型在长医疗案例文件上的推理能力。排行榜使用最难的保留问题集，Claude Fable 5 以 64.4% 的得分领先，Kimi K3 是领先的开源模型。结果显示，当前模型在医疗记录审查中部分可行，但成本高且完整性不足。
译文
Announcing MLCR-AA, our leaderboard for Wisedocs' MLCR (Medical Long Context Reasoning) benchmark for reasoning over long medical case files. We run the hardest, held-out question tiers, with Claude Fable 5 achieving the top score of 64.4% MLCR-AA tests models on realistic synthetic medical and insurance case files built by the team at @wisedocsai, based on the record review work their platform specializes in. Our leaderboard runs the private hold-out set of 60 questions from the two hardest categories: Expert, which requires specialist medical reasoning across a full case file, and Compound, which packs several independent questions into a single query. Each question is answered against a complete case file of ~70-150 pages, and graded by a three-model judge panel on completeness and accuracy alongside a concision test limiting verbose responses compared to expert answers. Accuracy verifies that the response is well-grounded in the source documents and case context while completeness assesses whether the model produced the same essential details that were included in expert-annotated responses. The concision test verifies that models are not producing excessively verbose content (>5x the length of expert responses). Recent Claude releases from @AnthropicAI lead MLCR-AA, with Claude Fable 5 at 64.4%, and Claude Opus 5 (scoring 53.9% to 59.4% across efforts). and Kimi K3 (max) from @Kimi_Moonshot is the leading open weights model at 38.3%. Key results from MLCR-AA: ➤ Medical record review is partly achievable with AI today, but at high cost and with room to improve: the leading model (Claude Fable 5) achieves a score of 64.4% at a cost of $1 per task, the median model scores <15% ➤ Models stay faithful to source documents but miss key details required for a complete response: nearly 40% of models tested score above 80% for accuracy, the vast majority score below 50% for completeness. Models are largely right about what they do report, and omit a lot of information. GPT-5.6 Terra (max) records the highest accuracy at 93.7% and still places 10th, held back by completeness ➤ Anthropic models lead due to completeness rather than accuracy: the highest-scoring models are from Anthropic, but their accuracy is comparable to or below the strongest OpenAI models. The separation comes from covering the full scope of the expert reference answer We would like to thank Wisedocs for building MLCR and for their collaboration in bringing it to Artificial Analysis!
Announcing MLCR-AA, our leaderboard for Wisedocs' MLCR (Medical Long Context Reasoning) benchmark for reasoning over long medical case files. We run the hardest, held-out question tiers, with Claude Fable 5 achieving the top score of 64.4% MLCR-AA tests models on realistic synthetic medical and insurance case files built by the team at @wisedocsai, based on the record review work their platform specializes in. Our leaderboard runs the private hold-out set of 60 questions from the two hardest categories: Expert, which requires specialist medical reasoning across a full case file, and Compound, which packs several independent questions into a single query. Each question is answered against a complete case file of ~70-150 pages, and graded by a three-model judge panel on completeness and accuracy alongside a concision test limiting verbose responses compared to expert answers. Accuracy verifies that the response is well-grounded in the source documents and case context while completeness assesses whether the model produced the same essential details that were included in expert-annotated responses. The concision test verifies that models are not producing excessively verbose content (>5x the length of expert responses). Recent Claude releases from @AnthropicAI lead MLCR-AA, with Claude Fable 5 at 64.4%, and Claude Opus 5 (scoring 53.9% to 59.4% across efforts). and Kimi K3 (max) from @Kimi_Moonshot is the leading open weights model at 38.3%. Key results from MLCR-AA: ➤ Medical record review is partly achievable with AI today, but at high cost and with room to improve: the leading model (Claude Fable 5) achieves a score of 64.4% at a cost of $1 per task, the median model scores <15% ➤ Models stay faithful to source documents but miss key details required for a complete response: nearly 40% of models tested score above 80% for accuracy, the vast majority score below 50% for completeness. Models are largely right about what they do report, and omit a lot of information. GPT-5.6 Terra (max) records the highest accuracy at 93.7% and still places 10th, held back by completeness ➤ Anthropic models lead due to completeness rather than accuracy: the highest-scoring models are from Anthropic, but their accuracy is comparable to or below the strongest OpenAI models. The separation comes from covering the full scope of the expert reference answer We would like to thank Wisedocs for building MLCR and for their collaboration in bringing it to Artificial Analysis!
