German audit finds AI judges miss 10.2% of clinical errors in medical documents

An audit published in npj Digital Medicine on 4 November 2026 found that large language models used to grade clinical documents are not a safe replacement for expert reviewers, at least for German-language texts¹. The practice under test, known as LLM-as-a-judge, uses one model to score the output of another, and it is increasingly used to evaluate clinical writing¹.

What the audit found

The researchers compared six open-weight models with expert judges on four German clinical datasets¹. The tasks covered summarising reports and nursing documentation, generating discharge summaries and writing patient letters¹. Each model was run under three evaluation approaches and four prompt configurations¹.

How closely the models agreed with the experts depended on the criterion and the dataset, and the authors found no configuration that held up across all three approaches¹. Every model showed position bias and identity bias¹.

When the model judges stood in for the expert panel, the results moved materially¹:

• 57.2% of pointwise categories shifted;
• 41.7% of pairwise winners swapped;
• 10.2% of clinical errors went undetected.

The models also turned clinical priorities upside down, giving too much weight to whether a text was succinct and accurate¹. The authors conclude that thresholds for deploying LLM-as-a-judge in clinical settings should rest on an analysis of how its scores would change real decisions¹.

What it means for the market

The finding goes to the evidence behind generative AI documentation tools of the kind European health systems are buying, from discharge summaries to letters for patients. Automated grading by another model is a cheap way for developers to compare versions of a product or show it performs well. This audit suggests that, in German at least, such scores could rank the wrong product first in four out of ten head-to-head comparisons and let one clinical error in ten pass unnoticed¹.

For hospital buyers and digital teams, the practical point is that a supplier's quality claims may rest on automated evaluation rather than review by clinicians, and the two can diverge¹. For developers, the authors' call for decision-impact analysis implies more expert review in validation work, which adds cost and time¹. The paper also notes that most earlier evidence on model judges used English data and pooled agreement scores that can hide failures that matter clinically¹. That leaves open how far reassuring English-language results carry over to other European languages.

The abstract does not say how many documents or expert reviewers were involved, which hospitals supplied the data, or which models were tested beyond their being open-weight¹. It also does not report any comparison with commercial closed models.

Caveats

The study covers German clinical texts and open-weight models only, and its conclusion is framed for German clinical text evaluation¹. The results describe how well models judge documents, not how safe the documents themselves are. The authors recommend that any clinical use of model judges be set against its effect on decisions before deployment¹.