Attention Is All You Have

"Attention Is All You Need" is a Beatles joke. In June 2017, eight researchers at Google, listed in random order, posted a paper under that title, a nod to "All You Need Is Love," and the joke carried a claim: that a single ingredient, attention, was enough to build a new kind of language software from. The design was called the Transformer. It sits under ChatGPT, Gemini, and most of the chatbots you have used, and the T in GPT stands for it.

Earlier systems read like a person at a keyhole, one word at a time, each step waiting on the last. Attention let every word look at every other word at once, so the work could be split across as many chips as you could pay for.

What attention is

The word was not new. In 1890, the psychologist William James wrote that attention "implies withdrawal from some things in order to deal effectively with others." Attending, on his account, is mostly a matter of ignoring.

You do it all day without noticing. At a loud party you follow one voice and let the rest blur into noise. A mind with nothing to ignore could not follow a conversation.

The machine does the same thing with arithmetic. Given the sentence "The trophy didn't fit in the suitcase because it was too big," a model settles what "it" points to by weighing every other word against it, leaning hard on "trophy" and giving the rest a sliver each. The weights are shares of a fixed total, so more for one word means less for the others.

One floppy disk against two million

A machine's attention is a line item. Training is the stage in which a model reads an enormous pile of text, and one lab, DeepSeek, published the numbers for a 2024 run: a cluster of 2,048 chips worked through a trillion tokens, the word-sized pieces these systems read, every 3.7 days. Sixteen hours of that is about 180 billion tokens. At the usual rule of thumb of four characters a token, that is roughly 720 gigabytes of text, most of the terabyte drive in a decent laptop.

Your side of the ledger is shorter. A couple of weeks ago I wrote about the fifty bits a second that reach conscious awareness, a 1989 estimate. A 2024 paper from Caltech's Jieyu Zheng and Markus Meister, reviewing decades of measurements of human performance, put behavior lower still, near ten bits a second, against roughly a billion bits a second arriving at the senses.

Run the arithmetic at fifty. Sixteen waking hours is 57,600 seconds, and 57,600 times fifty is 2.9 million bits, which is 360 kilobytes. That is the capacity of the 360K floppy disk of the 1980s, with a few kilobytes to spare. On the Caltech figure, a fifth of the disk would do.

Same sixteen hours. Your day fits on one floppy disk. The cluster's would fill two million of them.

The oldest warning

None of this is a new worry. In 1971, the economist Herbert Simon pointed out that "a wealth of information creates a poverty of attention," because information consumes the attention of whoever receives it.

The phrase "attention economy" came a generation later. Michael Goldhaber published an essay called "The Attention Economy and the Net" in 1997, and Thomas Davenport and John Beck put the phrase on a business-book cover in 2001. For most of its life the term described the web and then social feeds, an arrangement in which your attention was the inventory.

The flood

In the last piece in this series I called tokens the new electrons: a commodity metered by the unit and getting cheaper. A competent paragraph, summary, report, or block of code now costs almost nothing to produce, and each token a model writes for a human reader is a claim on that reader's attention. The cost of making the claim is falling toward nothing. The budget on the receiving end is the one in your skull, and no vendor ships a bigger one.

What that does to the value of whatever else is made of words is the subject of the next piece.

Who gets paid

What stays scarce gets sold. Converting attention into money is the business of the feed, and Cory Doctorow named what that business does to the platforms running it. He called it enshittification: first a service is good to its users, then it squeezes them to favor its business customers, then it squeezes those customers to keep the value for itself. The American Dialect Society made it the word of 2023, and his book on the subject followed in 2025.

The people on the receiving end have been keeping score. The researcher Gloria Mark, who has tracked how people use screens for decades, found that the average time on one screen before switching fell from about two and a half minutes in 2004 to about forty-seven seconds in recent years. A 2025 analysis in Psychological Bulletin, pooling 71 studies and nearly 100,000 people, linked heavy use of short-form video to worse attention and weaker impulse control, though most of those studies were snapshots that cannot show cause. Causation is unproven, but the target is plain.

The trade started in social feeds and has spread to nearly every medium that carries words. Whatever can be made cheaply is being aimed at the same fixed supply.

Fifteen hundred hours

The feed harvests seconds, but seconds are the raw material of skill, and language shows it most clearly. A few other species manage crude versions of it, and only humans build one that can say anything at all, which makes it a fair candidate for the trait that defines us. It is also a skill with no shortcut.

I work on LinguaMama, so the figure I carry around is about fifteen hundred hours to get fluent in a language. The Foreign Service Institute's numbers for English speakers bracket it: roughly 600 class hours for Spanish, 2,200 for Japanese. Gifted polyglots may well have better circuitry, but they run it through the same channel as the rest of us, and the channel is narrow. A 2019 study of seventeen languages, from Basque to Vietnamese, found that fast languages pack less into each syllable and slow ones pack more, and that all of them end up near thirty-nine bits a second. It is as if the channel set the pace.

A model skips the hours and pays in electricity. Meta's 2022 translation system covers 200 languages in a single model, and the chatbots you use can work in dozens. The 2024 training run I described took under two months and used 2.8 million GPU-hours, which at the chips' rated 700 watts comes to about two gigawatt-hours before cooling. A brain at twenty watts spends thirty kilowatt-hours over fifteen hundred hours. That is roughly 65,000 times the electricity, for a model that learned far more than one language, but the shape of the trade is plain. The machine buys fluency with power. You pay with attention, and there is nowhere to buy more.

The same arithmetic runs through every skill. In 1993, K. Anders Ericsson found that the best violin students at a Berlin conservatory had averaged about ten thousand hours of practice by age twenty. Malcolm Gladwell turned that average into a rule, and Ericsson objected that the kind of practice counted, not the count: deliberate practice, work at the edge of what you cannot yet do. It runs on attention, spent on purpose, for years. A chatbot can do the task for you, which is the subject of an earlier piece in this series, "There Is No Kung Fu Download," but it cannot put in the hours for you.

Skill is what attention buys that lasts.

Attention was all the machine needed. It is all you have.

The Transformer got better by learning where to put its weight, and then it was given a data center. You get the first half of that recipe.