Today for AI
HOT RADAR
llmHEAT 0.0°

AI CURATED HOT STORY · 10/8/2026

Harvey Releases LAB-AA v1.1 Benchmark: Introducing Hallucination-Gated All-Pass Rate for Legal Agents

2 reports archived0 independent sourcesupdated 10/9/2026, 02:33:38
Synthesis & Latest Updates

Harvey and Artificial Analysis have released Harvey LAB-AA v1.1, updating the scoring methodology for AI agents performing real-world legal work. The new version introduces the 'Hallucination-Gated All-Pass Rate' as a headline metric, requiring deliverables to satisfy all rubric criteria while containing no material hallucinations. Based on 120 private legal tasks covering areas like M&A and litigation, the benchmark uses a three-judge panel to grade each criterion, aiming to raise reliability standards for agentic legal applications.

LATEST/Harvey and Artificial Analysis have released Harvey LAB-AA v1.1, updating the scoring methodology for AI agents performing real-world legal work. The new version introduces the 'Hallucination-Gated All-Pass Rate' as a headline metric, requiring deliverables to satisfy all rubric criteria while containing no material hallucinations. Based on 120 private legal tasks covering areas like M&A and litigation, the benchmark uses a three-judge panel to grade each criterion, aiming to raise reliability standards for agentic legal applications.

HEAT TRENDHourly heat curve

157 fully observed hours
Now
0.0
Peak
41.604:00
24h change
–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

TIMELINECoverage timeline

Total 2 reports · Latest first
  1. Artificial Analysis (@ArtificialAnlys)T1·68 pts
    • LAB-AA v1.1 introduces the 'Hallucination-Gated All-Pass Rate,' crediting only tasks that meet all criteria without material errors.
    • Data shows over 60% of otherwise passing model outputs contain misleading hallucinations, such as incorrect contractual dates.
    • Grok 4.7 (xhigh) leads with a 9.4% score, while Muse Spark 1.3 (max) offers strong performance at ~$4.20 per task.
  2. Artificial AnalysisT1·68 pts
    • Harvey LAB-AA v1.1 introduces the 'Hallucination-Gated All-Pass Rate', crediting tasks only if they are hallucination-free and meet all criteria.
    • The benchmark comprises 120 private legal tasks spanning specialized areas such as M&A, tax, and litigation.
    • A three-judge panel mechanism is employed to independently grade each rubric criterion, enhancing evaluation consistency and accuracy.