Why do two runs of the same prompt differ?

Sampling with temperature above zero, provider-side model updates and non-deterministic batching all introduce variation between calls.

What does JSON mode do?

It extracts the first JSON object from each reply, even from inside a code fence, and pretty-prints both with identical formatting so only real value changes show up.

What is word overlap?

The Jaccard index over unique words — a quick order-independent measure of how much meaning the two answers share.

Can I use it for regression testing?

Yes — keep a golden output, paste the new run and watch the similarity score before shipping a prompt change.

Is my data uploaded to a server?

No. This tool runs entirely in your browser using standard web APIs, so your input never leaves your device and nothing is logged or stored on our servers.

LLM Output Diff

Compare two model responses to measure drift between runs or models.

Support

Works Offline

Privacy First

No Login

No API

Mode

Line diff

Ignore case

Character similarity

53.9%

Word overlap (Jaccard)

56.3%

Changed blocks

Output A (run 1)

Output B (run 2)

Inline diff

the capital of france is paris.paris, ithome hasto aboutroughly 2.12.2 million residents.

About the LLM Output Diff

Paste the answers from two runs, two temperatures or two models and get a highlighted diff, character similarity, word overlap and a changed-block count. JSON mode extracts and pretty-prints structured replies first so formatting noise does not pollute the diff.

Examples

Two runs

2.1 million residents vs 2.2 million residents

Output

94% similar, 2 changed blocks

Keyboard shortcuts

Copy the main outputCtrl / ⌘ + Shift + C
Download the resultCtrl / ⌘ + S
Share this toolCtrl / ⌘ + Shift + S
Reset the inputsAlt + R
Open the tool search paletteCtrl / ⌘ + K

Related tools

Context Window Packer

AI & LLM

Fit as many RAG chunks as possible into a model context budget.

Prompt Diff & Compare

AI & LLM

Compare two prompt versions word by word with a token delta.

AI Dataset Deduplicator

AI & LLM

Remove exact and near-duplicate rows from fine-tuning datasets.

AI Dataset Statistics Analyzer

AI & LLM

Profile a JSONL training set: token stats, roles, duplicates and cost.

AI PII & Secret Redactor

AI & LLM

Strip emails, keys, tokens and personal data before prompting an LLM.

AI Token Counter

AI & LLM

Count exact LLM tokens for GPT, Claude and Gemini prompts.

Frequently asked questions

Version 1.0.0 · Updated 2026-08-15 · Runs entirely in your browser