Two numbers explain most of a detector score. Neither is about authorship.

8 min read · Published October 2026 · Method and data at the bottom. This is the follow-up to our control test, where four of six human-written passages were flagged as AI.

The question people actually ask

"Which detector should I trust?" is the wrong question. Two detectors reading the same paragraph routinely disagree, and the same detector changes its mind after a model update. That looks like a bug until you understand what the number is made of.

So we stopped arguing about vendor claims and measured it. Every text we had a ZeroGPT score for — nineteen of them, human and machine — got the same treatment: sentence-length statistics, stock-phrase counts, vocabulary variety, length. Then we correlated each feature with the score.

What detectors say they measure

• Burstiness. Variation in sentence length. Language models drift toward a comfortable middle; people don't. High variation is supposed to read human.
• Perplexity. How predictable each word is given the previous one. Fluent, obvious word choices lower it.
• Stock phrasing. "Delve", "furthermore", "in today's fast-paced world". A real but shallow signal.

Two of those three are computable on any text without an API key. That is what makes this test possible.

The correlations

n = 19, one detector, English, Pearson. Small sample — treat these as directional, not definitive.

Two things this settles

Burstiness is real. The detector is not a black box making things up. It responds to sentence-length variation the way it says it does, and the effect is large.

Vocabulary advice is useless. A generation of writing guides says "vary your word choice to sound human". In our data, vocabulary variety correlated at 0.01 — statistically indistinguishable from noise. Swapping synonyms does nothing, which matches what we found when we asked a model to "polish the prose": the score did not move at all.

And one thing it breaks

If burstiness were the whole story, texts with similar variation would get similar scores. They don't. Here are the five rewrites our own tool produced:

Look at the middle pair. Burstiness 0.58 both times. One scored 32.5, the other 96.6. Whatever separated them, it is not sentence-length variation — and we could not find it in any feature we measured.

The original texts behave the same way. The two highest-variation samples in the whole set were both written by a person: a work email at 0.72 scored 0%, a 341-word memoir at 0.62 scored 69.8%. Same feature, opposite verdicts.

So why do two detectors disagree?

Three reasons, in order of how much they matter:

1. They weight features differently. One leans on burstiness, another on perplexity, a third on n-gram overlap with known model output. Same text, different arithmetic.
2. They set thresholds differently. A raw score of 0.6 can be "likely AI" on one site and "mixed" on another. The threshold is a product decision, not a measurement.
3. They were trained on different data. A model trained before your favourite writing assistant existed will misjudge its output in predictable ways.

None of those three is a disagreement about who wrote the text. They are disagreements about how to score a style, which is why neither answer can settle the question.

What actually moves a score

Based on our data, in order of effect size:

1. Write short sentences. The share of sentences at eight words or fewer was the strongest predictor we found. Three, four in a row is fine — people do that.
2. Break the rhythm. Long, short, long, very short. Not medium, medium, medium.
3. Cut the stock phrases. Real effect (+0.55), smaller than most people assume.
4. Do not bother swapping synonyms. Effect: none measurable.
5. Do not pad the word count. Effect: none measurable.

What this does not tell you

• Nothing about authorship. A style score cannot establish who typed the words. Our control test remains the clearest demonstration: four of six human-written passages came back at 50% or above.
• Nothing about other detectors. We tested one. GPTZero's API needs a paid key and Turnitin needs an institutional account, so we have no numbers for them and will not pretend otherwise.
• Very little about long text. Our longest sample is 341 words. Beyond a few thousand words, section-level scores start to diverge and the aggregate means less than it appears to.

Frequently asked

Data: metrics-vs-score-20261019.json — 19 texts, every feature, every score, plus the script that produces it. Corrections to [email protected].