Skip to content

AI Compass / News

AI as a document checker: run a counter-test and a repeat test before relying on it

General RA & medtech

Two preprints, not yet peer reviewed: in tests, AI checkers often gave the same verdict with or without the rule, and differing verdicts for the same request.

Source: Sadhu et al.: Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems (arXiv),

Note: I provide engineering services in regulatory affairs for medical devices.

Technical drawing: three check runs, each made of a rule sheet with a section sign, a document, an LLM and a verdict. In the second run the rule is struck out and the tick stays the same (=); in the third run with the same request it turns into a cross (≠).

What happened: Two arXiv preprints, not yet peer reviewed, test large language models (LLMs) as checkers. For the first (8 October 2026), a team from Lexsi Labs gave five open models cases from 20 regulatory and policy domains and deleted, swapped or negated the governing rule. According to the authors, the verdict changed in about 7% of cases on average, and in about 5% when the rule was deleted. For the second (15 October 2026), a single author repeatedly sent 14 commercial model deployments the same request for a preliminary assessment of public FDA documents on blood glucose meters. According to the author, every deployment returned differing assessments at least once, and in twelve the verdict changed. Conflict of interest: His employer, GFLOPS, funded the study and sells its own pipeline, which comes out as reproducible in the paper; the author is the inventor on a provisional patent application covering it. The announced data were not available on 27 October 2026.

What this means: A high hit rate does not prove that an AI checker applies your requirement. For audit and change control, “temperature 0”, the setting meant to keep answers as consistent as possible, is not enough either: according to the author, most deployments rejected or ignored it, and the only model that ran with it changed its verdict on 0 or 4 of 70 prompts, depending on the access route.

What you can do:

  1. Counter-test: Have your cases checked additionally without the rule and with the rule negated. If the verdict barely changes, there is no evidence that it rests on your requirement.
  2. Repeat test: Send the same request several times, on different days too, and compare verdict, reasoning and cited evidence.
  3. Log the setup: Record model, version, access route and settings in the validation file.
  4. People decide: The AI provides a preliminary assessment; a qualified person signs off.

Context: The first paper did not test medical devices and cut the rule in the prompt to 300 characters. According to the authors, the test cases mostly gave the answer away; where the rule decided the outcome, the models followed it closely. On two other test sets, the largest model’s verdict changed three to four times as often. The second covers only two days per deployment, and its pipeline has not yet been reproduced outside the company.

What does this mean for your team? Let’s talk it through. Request an intro call