NuclearBench Atomic Canyon

NuclearBench

A public benchmark for how well AI answers questions from real nuclear documents.

See the Results Read the Card
Built by
Motivation

What NuclearBench measures that general benchmarks miss

General AI benchmarks don't tell you whether a model can be trusted with dense, technical nuclear documentation. NuclearBench is built to.

Real nuclear documents combine OCR artifacts, buried cross-references, dense vocabulary, and revision-sensitive limits. NuclearBench tests whether models can connect those scattered facts while staying grounded, then scores substance and faithfulness separately.

  1. 1

    OCR noise

    Scanned pages split words and swap characters: cladding becomes c1adding. A model has to read the meaning through the damage.

  2. 2

    Cross-references

    The fact often isn't on the page you're reading. It's behind a pointer to another section, table, or appendix.

  3. 3

    Dense vocabulary

    Domain terms and acronyms carry exact meaning. A confident paraphrase can quietly change what was said.

  4. 4

    Revision sensitivity

    A limit that changed between revisions makes the superseded value confidently and dangerously wrong.

Results

Model Performance

NuclearBench model results. Sortable by model, completeness score, and faithfulness score; switchable between the Core and Adversarial sets.
Model Completeness Score Faithfulness Score
Claude Fable 5.1 95.1% 91.3%
Claude Opus 5.5 95.0% 91.5%
Gemini 4 Argon 93.6% 89.3%
Claude Sonnet 5.5 92.8% 88.3%
GPT-6 Astra 90.6% 92.7%
GPT-6.1 Sol 90.1% 93.7%
Claude Haiku 5.5 87.2% 88.4%
GPT-6.1 Terra 86.8% 94.9%
Gemini 3.8 Flash 84.8% 87.3%
Mistral Large 4 84.1% 87.1%
GPT-6 Luna 81.3% 93.4%
DeepSeek V4 Pro 80.3% 93.3%
Nemotron 3.5 Super VL 78.6% 90.5%
Nemotron 3.5 Lightning 74.6% 77.5%
Gemma 4 31B 72.4% 90.0%
Gemini 3.5 Flash-Lite 70.9% 90.4%

Completeness by document length (core set)

One row per model, with a point for its score on short, medium, and long source documents.

Dataset

What the benchmark covers

280 questions built on 247 real nuclear documents: long, messy, and all publicly available, so any result is reproducible with no export-control barriers.

NuclearBench · dataset manifest Apache-2.0
280
Questions 219 core + 61 adversarial · plus 125 holdout to detect contamination
247
Publicly available source documents LERs · NUREGs · Reg Guides · Inspection Reports
1,013
Scoring criteria INL-authored, per-question rubric
Evaluation

Scoring

Every answer is judged on two axes, completeness and faithfulness, reported separately, never averaged, because they capture different failure modes.

01 Completeness

Did the answer cover what the question required?

Every question ships with an explicit rubric: 1,013 criteria in all, the specific facts, conditions, and limits a correct answer must state, written and validated by Idaho National Laboratory experts. A high score means nothing required was left out.

Evaluated byCriteriaJudge
02 Faithfulness

Did the answer stay true to the source?

Each claim is checked against the document for invented details, wrong specifics, or confident drift. A high score means the answer added nothing the source doesn't back up.

Evaluated byFineSurE
Faithful but thin true, but incomplete Fails both incomplete + unsupported Complete but drifting adds unsupported claims Complete & grounded strong on both scores Completeness → Faithfulness →
Methodology & Resources

Where NuclearBench fits

Nuclear AI runs on retrieval: models can't rely on memorized, export-controlled, revision-specific documents. Retrieval and answer generation fail in different ways, so each gets its own benchmark.

Find the evidence

FermiBench

Visit FermiBench ↗
  1. 01
    Retrieve

    Find the governing documents.

  2. 02
    Validate

    Confirm the source and revision.

Use the evidence

NuclearBench

  1. 03
    Generate

    Answer only from the validated source.

  2. 04
    Evaluate

    Score completeness and faithfulness.

Open resources

Read the full card, run it yourself

The technical card documents the full methodology, judges, and confidence-interval approach. The dataset and code are openly licensed for independent evaluation.

Technical card & dataset License: Apache 2.0
View on Hugging Face
Credits

A collaboration

NuclearBench was built together by Idaho National Laboratory and Atomic Canyon.

Question authoring, source-document pairing, criteria validation, and review.

Visit inl.gov ↗

Benchmark engineering, synthetic generation, evaluation pipeline, and release.

Visit atomic-canyon.com ↗