It Was Never the Speaker
By Noam Shemla · Data & Method
TL;DR We put 20 recordings by five speakers (Churchill, King, Jobs, Sinek and Obama, 1940 to 2018) through the same analysis our members’ practice recordings go through, and compared them. A speaker varies from themselves far more than speakers vary from each other. On every one of the five axes, the variance attributable to speaker identity comes back at or below zero.
Every previous article in this series measured a single recording, and every one of them carries the same unstated weakness. When we published that Churchill's delivery scored 45 on Tonality, the fair reading of that sentence was "one recording of Churchill, made in a studio in 1949, scored 45," and nothing as broad as "Churchill was monotone." Whether that number says anything about the man is a question a sample of one cannot answer.
So we went and got more recordings. Three or four per speaker, spanning a career: Churchill from 1940 to 1949, King from 1963 to 1968, Jobs from 1997 to 2007, Sinek from 2009 to 2018, Obama from 2004 to 2018. Twenty in total, four per speaker, about nine hours of audio, every one of them run through the identical call a member's own Practice Studio recording makes, with the coaching track and the prior-session memory withheld, so nothing about who was speaking could tilt the result.
The whole corpus, at once
Read across a row rather than down a column. If a Speaker Signature were a property of the speaker, each row would be four copies of one shape with minor wobble. Look at King's row.
Five speakers, one spread
This is the entire study in one chart. For each axis we computed two things: how far apart the five speakers' own averages sit, and how far a single speaker travels across their own recordings. The second number is between two and five times the first, on every axis without exception.
The formal version is an intraclass correlation: the share of total variance attributable to speaker identity. It comes back at or below zero on all five axes. A negative ICC does not mean the calculation broke. The estimator returns one when the spread inside a group exceeds the spread between groups. Read plainly: knowing which of these five men is speaking tells you almost nothing about what the instrument will say.
The clearest single case
King's four recordings are all the same kind of occasion: a live address to an audience. Same man, same format, five years apart at the outside. His Tonality scores are 80, 93, 96 and 99.
The low mark is Beyond Vietnam, delivered at Riverside Church in April 1967: a long, dense, read policy address, and the instrument labelled its delivery flat. The high one is I've Been to the Mountaintop, Memphis, April 1968: the last speech of his life, largely improvised, and the highest score anywhere in this corpus. Twelve months apart.
What that does not mean King did not get better in a year, and Beyond Vietnam is no lesser speech; it is widely held to be among his most important. The instrument is responding to something that changed between the two occasions, and whatever that was, the speaker stayed the same.
Check it yourself
The axis selector is there because a summary statistic asks for trust and a scatter does not. Switch between axes and the pattern holds: the points belonging to one speaker are spread across the whole range, and the speakers overlap almost completely.
Switch Against to length and something else appears: the instrument's own bias. Score falls as recordings get longer, most steeply on Intellect, where the correlation with duration is −0.60. This tool was built to score a practice clip of up to three minutes. Every recording in this corpus is longer than that; five of them run past forty minutes.
That raised an obvious possibility: perhaps the speakers are consistent and we were simply watching our own length bias. So we regressed each axis on log duration and recomputed the correlation on the residuals. Identity did not reappear. The ICC stayed at or below zero on every axis. The length effect is real and it is our bug, but it does not explain the result.
Does the occasion explain it?
Partly, and less cleanly than we expected. Several occasion types here rest on a single recording, so they are shown but carry no weight on their own. Among the categories with more than one, nothing separates decisively, and the strongest counter-example is King, whose four recordings are all the same occasion type and who nonetheless produces the widest spread of anyone in the corpus.
The most defensible statement is the negative one: it is not the speaker. What it is instead (the room, the register, the day, the length, some combination) this corpus can narrow but not settle.
Every recording
What we got wrong before this
An earlier version of the Signature cards on these articles did not come from this pipeline at all. It came from four proxy formulas we built by hand, and every one of them measured something other than its label. Energy was computed from the recording's mean volume, so what it really tracked was audio mastering: King's 1963 open-air tape came out lowest in the entire set, below every keynote and every studio recording. Running the real analysis on that same recording puts it at the very top, with the highest reading on any axis anywhere in this corpus.
Structure was computed from a longest-to-mean sentence ratio, which meant it tracked caption punctuation: automatic transcription chopped Jobs into nine-word fragments, so a single long sentence handed him a perfect score while the Gettysburg Address landed near the bottom. And five of the six Structure values we shipped did not match even that formula. They had been typed by hand.
We are describing this at length because the fix mattered less than the failure mode. Every one of those numbers looked plausible on the page. The only reason they were caught is that somebody looked at the chart and said the scores felt wrong.
The same audio analysis our members’ practice recordings go through, called directly, with coaching track and prior-session memory withheld so nothing about the speaker biases the score.
Measured
- Five axis scores plus an overall, per recording, exactly as returned
- The tone classification the analysis assigns (dynamic, steady or flat)
- Variance decomposition across speakers: ICC(1), between-speaker SD, within-speaker SD
- Correlation of each axis with recording duration
- ICC recomputed on duration-residualised scores
Inferred, not measured
- Nothing. Every number in this article is read from the stored analysis output
Three limits worth naming Decade is perfectly confounded with speaker in the 1940s and 1960s (every 1940s recording is Churchill, every 1960s one is King), so no era effect can be separated from a speaker effect here. Recording quality varies enormously across seventy-six years. And the analysis is being run well outside its designed range on every single recording.
Reading your own Signature after this
If your own Signature moves between recordings, you have not necessarily got better or worse. It is the most common behaviour in this corpus, produced by five of the most practised speakers of the last century. A single recording is a reading of one occasion.
So comparing your score with someone else's tells you little. The speakers here differ from each other by about ten points on average, which is less than most of them differ from themselves. The useful comparison is your own trend across many recordings, which is exactly why a member's Signature is built from every session rather than the last one.
And the finding cuts at our own framing. We call it a Speaker Signature. On this evidence, for recordings of this kind, it is closer to an occasion signature, and we would rather publish that than keep the nicer name unexamined.
Sources
Every recording was obtained from its publicly available published version and analysed locally; none is hosted or reproduced by us, and no transcript is republished here. The speeches are individually cited in the six articles of this series.