This essay is part II of a trilogy of related concepts in languages, information theory, geometry, metaphysics, relativity, and the information economy.
The trilogy began with Part I — Cavemen Created the First Large Language Model — on Sunday 9 August, and will conclude with Part III — Attention Is the Last Paywall — on Sunday 16 August.
I — The grammar of possible things
A protein is a sentence
In 2001 two chemists in Massachusetts built a library containing six trillion random protein sequences. Each contained a stretch of eighty amino acids generated without ancestry, selection or design: a physical sample from the vastly larger number of proteins that could in principle be written. They then searched the library for any capable of one modest trick — binding ATP, the molecule cells use to transfer energy.
Four came back. Out of six thousand billion.
A protein is a string in a twenty-letter alphabet, made from amino acids rather than letters, and almost every possible string lies outside the narrow manifold occupied by natural proteins. Some fold. Some bind. Most arrive fitted to no biological job at all.
Not a slightly inferior solution. Biologically useless junk. Something separates the vanishing few that work from the immense remainder that do not. Nobody designed it, but cells have been reading it fluently in one way or another for four billion years.
The pharmaceutical industry earns a living inside this problem. In 2020 a team screened more than a billion purchasable molecules against one protein involved in the cellular response to oxidative stress. They knew the shape of the target, the site they wanted to occupy, and enough chemistry to reject almost everything cheaply. What they did not know was how to identify the winning molecule.
The best few million were examined more closely; 590 reached the laboratory; dozens bound. The grammar was known well enough to screen a billion sentences, but not well enough to generate valid answers at the outset. In the end, it was brute force what won it.
Both searches were up against the same uncodified rule: the one that let four sequences out of six trillion through, and stopped the rest at the door.
Biologists call it a grammar. They are not being whimsical. The same architecture that encodes patterns in English exists in amino acid chains too, one masked residue at a time, and absorbs enough of the rule to help predict how a sequence will fold.
Spoken word might have been the original large language model. But biology appears to have been running a comparable alpha model for æons.
A wedding seating plan is not a language
A wedding seating plan has structure. Twelve tables, eighty names, and the two branches of the family who have not spoken since 2011 placed at opposite ends of the room. Every element is where it is for a reason. But nothing in the plan tells you what the eighty-first name could be, because there is no eighty-first name. The plan is finished, and being finished is the whole of what it is.
That is structure. But a grammar can be more than just a few fixed relations – it can be generative.
Noam Chomsky pointed out in 1957 that no list of approved sentences could ever add up to a language. Speakers continually produce and understand sentences nobody has produced before — including this one, and the one before it, and the one you are about to read and probably forget.
A language is not a catalogue or acceptrable sentences. It is a machine for making valid prose, and the machine is far smaller than its vast potential outputs.
Here are five sentences. Each is grammatically sound.
The committee removed everything useful.
The opening was narrower than advertised.
By Thursday, the courgette had acquired a name.
The landlord blamed natural moisture.
Daniel had arrived wearing mostly confidence.
Not one sentence is broken. Together they are gibberish. We can infer a committee, an opening, a landlord and Daniel relying heavily upon confidence for dignity. What we cannot infer is what any of them are doing next to each other.
Well-formedness does not inherit upward. You can be coherent at one level and comprehensively ruined at a higher level of abstraction, which is the experience of hearing Dame(!) Andrea Jenkyns speaking in mostly complete sentences.
Each floor’s well-formed output becomes the next floor’s alphabet. Letters make words. Words make sentences. Sentences make paragraphs, which must develop rather than piling up without supplying additional meaning.
Paragraphs make a piece of writing that goes somewhere — and if this paragraph stops going somewhere, you will notice the failure on that floor, not before. This relationship continues upwards: paragraphs to chapters, chapters to books, books to their sequels and source material, and so on.
Proteins exhibit a comparable hierarchy: amino acids form motifs, motifs contribute to folded domains, and domains combine into structures capable of doing a job. At every level, a valid component can still be assembled into a useless whole.
A generative grammar is not mere order. It is a codification of structured constraints that makes order carry permutations of meaning.
The village excluded the young. The young excluded the village.
The same five words; opposite directions of consequence. The village blocks new homes; the young leave; the school, pub and bus routes eventually follow. The villagers win the planning meeting and lose the village.
English word order can carry who did what to whom only because most arrangements are forbidden, or change the meaning. If every ordering were equally valid, the arrangement would tell you nothing.
Therefore, meaning lives mostly in the exclusion of the infinite combinations of what is not valid for a given communicative intent.
The two ways to say nothing
It is tempting to conclude that more constraint must therefore produce more meaning. Tighten the rules, narrow the possibilities, increase the information carried by whatever remains. But take that to its limit, and what do you get? A diamond.
A crystal is the most ordered object in everyday experience: one pattern repeated in every direction. Part of its grammar is its space group, the set of translations, rotations, reflections and combined operations that constrains how an asymmetric unit is repeated through the crystal. There are exactly 230 space-group types in three dimensions, which is a surprisingly small number of ways for matter to be tidy.
But once the rule is known, almost nothing remains to be learned. See one unit cell and you can predict the next. Perfectly ordered; after the opening sentence, nothing new to add.
Now run the other way. Perfect noise permits every arrangement equally. Each new element is unpredictable, but the surprises accumulate into nothing. There is no stable structure against which one variation can matter more than another: no syntax, no deviation, no development.
Crystals have structure without novelty. Noise has novelty without structure. Neither can sustain a message.
Schrödinger reached the same requirement in 1944, before anybody knew what a gene was made of. Hereditary information needed something ordered enough to survive and be copied, but irregular enough to carry a message: an aperiodic crystal. Nine years later the shape of one turned up in Cambridge. It was deoxyribonucleic acid: DNA.
But constraint alone cannot describe a generative grammar. Entropy tells you how much surprise remains in the outputs; the grammar is the rule system producing and excluding them. Eight inches is something you can measure about a person without it being the person. Entropy is something you can measure about a grammar without being the grammar.
The distinction matters because small rulebooks can produce radically different worlds. Conway’s Game of Life runs on a few rules about which cells live and die, yet generates gliders, oscillators and — given sufficient patience — a universal computer. A crystal also proceeds from a compact rulebook. Its library contains one sentence, repeated until the edge of the crystal, or perhaps an inclusion.
Two rulebooks of similar brevity. Wildly different room in the output. A structure is fixed; but a grammar can be a generative machine — one that accepts valid combinations and excludes an infinity of others.
Meaning has a shape
The word grammar has now been applied to amino acids, English sentences, paragraphs and diamonds without noticeably changing its meaning. This is the tell.
Strip away the symbols and something survives: a concept which governs which arrangements are valid, which are near-misses, which are catastrophes, which components can be exchanged without anybody noticing, and which cannot be exchanged at all.
The temptation is to say those relations simply are a geometry. But closeness alone isn’t enough to define the relations. Human similarity judgements are routinely asymmetric: a poodle resembles a dog more readily than a dog resembles a poodle. Tversky demonstrated the problem in 1977.
Geometry does not arrive free within a metaphor. But you can try to fit one and see whether it holds. This is what embeddings do. A word, sentence, image or protein becomes an embedding linked by vectors: a position in a multidimensional space arranged so that useful relations become navigable.
This is how a large language model works. In classic word embeddings, the route from man to woman resembles the route from king to queen:
Word2Vec made this structure famous, although not every relation reduces to one clean vector. Modern transformers make the map contextual: bank beside river does not end in the same place as bank beside mortgage. The coordinate is not stored once. It is rebuilt from the company the word keeps.
The representation is not the concept itself. It is a map of the relations the model found useful to describe the concept space.
A caricature exaggerates whatever makes a face distinctive and throws away the rest, and for a familiar face it can still be identified as readily as an accurate drawing. Discarding information is not a failure when the distinctions dropped are the ones the task never used. An embedding discards them freely while keeping others with extraordinary precision.
The useful data also occupy far less space than all conceivable data. Real sentences, proteins and images tend to sit on restricted surfaces inside the enormous spaces capable of containing them — the broad intuition behind the manifold hypothesis.
Almost every direction away from a valid sentence leads immediately out of the language. The model compresses partly by declining to spend dimensions on things the grammar had already forbidden.
Different relations may require different shapes. Analogies often behave usefully in flat space. Hierarchies fit negative curvature, where there is exponentially more room as a tree branches outward.
Periodic structure coheres neatly towawrds a torus — which is also the shape of the map the brain probably uses to keep track of position in a room.
Each of these shapes is the cheapest geometry yet discovered for embedding its particular flavour of relations. None of them was chosen for how it looks in a scientific paper: they were discovered in nature by evolution.
One further prediction follows. If some structures belong to the world rather than the symbols used to describe it, systems trained on different substrates should sometimes recover comparable arrangements.
The Platonic Representation Hypothesis, proposed with a certain amount of nerve, argues that better neural networks trained on different data and modalities increasingly converge on a shared statistical model of reality. A model raised on photographs and one raised on text begin from opposite ends yet sometimes organise what they learn in comparable ways.
This remains a hypothesis. Similar objectives can produce similar solutions without requiring Plato to have been right about anything. But photographs and sentences do not share an alphabet. If both repeatedly recover some of the same relations, the economical explanation is that those relations belong at least partly to what they represent.
Two cartographers can make similar mistakes. It becomes less likely when one has never seen a coastline and the other has never seen a map. The symbols vary. But the relations recur.
II — Is the map the machinery itself?
The map and the machinery
Everything so far has been geometry inside a representation: a shape built to preserve the relations somebody found useful. Embeddings, manifolds, hyperbolic trees, the toroidal grid. All of them are maps. The next step is more conceptual – that mathematics can relate to a system in two very different ways.
Sometimes it is only a map. A Tube map preserves the relationships needed to travel around London while distorting almost everything else. An embedding does something similar: it preserves the relationships useful to a model’s task without claiming to contain the whole meaning of the things represented.
In other cases, the mathematics does more than describe the system. Conway’s Game of Life is not a grid with a separate mechanism hidden underneath it. Its rules are the mechanism: apply them to one state and they produce the next.
A crystal’s symmetry group likewise does not merely provide a convenient summary of the lattice. It specifies the transformations from which the repeating structure is generated. The difference is between mathematics as a description of the machinery and mathematics as the machinery itself.
That distinction matters because we are approaching a larger question. When mathematics describes the structure of language, a protein or the universe, are we looking at a useful map of something deeper? Or have we reached the level at which the map and the thing mapped are the same thing?
Large language models provide a useful intermediate case. Their behaviour emerges from a structure distributed across billions of numerical parameters and activations. A model can generate a fluent explanation of why it gave a particular answer, but that explanation is not a transcript of the probabilistic computation that produced it. Asking a person why they recognised a face does not produce the activity of their visual cortex either.
Interpretability research tries to translate these distributed processes into forms human beings can inspect. Under the superposition hypothesis, networks can represent many overlapping features in the same dimensions rather than storing one neatly labelled concept in each neuron. Sparse autoencoders have begun to recover millions of more interpretable directions, which can be labelled, stimulated and linked to changes in behaviour.
That is genuine explanation, but it remains a translation. Any account shorter than the system must either leave out some of its internal relationships or replace them with a rule compact enough to regenerate them.
Sometimes such a rule exists: that is what the Game of Life has. Where none does, a complete account has nowhere to economise, and approaches the size of the thing it describes. At that point, the explanation would no longer be a summary. It would be another copy.
The most faithful representation of a sufficiently complicated structure may therefore be the structure itself.
The useful disappearance of of
Physics provides the cleanest example of what happens when description and constitution collapse. In Newtonian mechanics, gravity is a force acting between bodies. Geometry describes where the bodies are and the paths they follow under that force. Cause and geometry occupy different columns in the account.
General relativity removes the extra column. Matter and energy shape spacetime; objects follow its geometry. There is no gravitational hand pulling the apple downward while curvature records the incident. The apple falls because its most probable path leads through curved spacetime.
Gravity is not merely described by geometry. The moving geometry of spacetime is the mechanism. The word of disappears: not the geometry of the gravitational field, but the geometry that constitutes it.
Philosophy has a test for exactly this. Quine asked what a theory must quantify over to come out true — what has to exist for its statements to be about anything at all. General relativity quantifies over the geometry. It does not quantify over a force.
This does not mean that every successful mathematical model should be promoted immediately into reality. The result king − man + woman ≈ queen is telling us something about English, not about the universe. It means the promotion is conceptually legitimate.
Harder cases exist: the Earth–Moon centre of mass is a mathematical point about 4,670 kilometres from Earth’s centre, inside the planet, around which Earth and Moon both move. It is not a separate material object, but it identifies a real relation their motion obeys. Mathematics sometimes maps the machinery, and sometimes identifies it. The question is: can we tell which is which?
Predictive success alone cannot settle it. Two accounts may make identical predictions while disagreeing about what their mathematics means. Nor does elegance guarantee existence, though it might confer it.
But if every observable consequence is fixed by relations, transformations and invariants, an additional substance hidden behind them begins to look unemployed. That thought has a name.
Structure without scaffolding
Scientific realism says successful theories work because they capture something real. Hilary Putnam gave the argument its usual form: realism is the only account that does not make the success of science a miracle. If atoms are a convenient fiction, the fact that chemistry works needs some other explanation, and nobody has a good one.
Its historical problem is that successful theories have repeatedly been abandoned.
Ether, caloric and phlogiston were not fringe hobby theories dreamt up by bunsen burner crack addicts. They originated in serious thinking, and were at least partially accepted by the scientific profression before accumulating evidence moved the academic world forward.
John Worrall’s structural realism asks what survives a revolution. The ontology — what the old theory said was there — may not. But mathematical relations sometimes do. The continuity he identifies is not merely observational: it lies in theoretical structure carried from one account into its successor.
Structure seems to be what science can know reliably. Ontic structural realists go further: perhaps structure is not merely what survives our descriptions. Perhaps it is all there is. Objects do not first exist and then enter into relations; their identity is defined already by the relational role they inhabit within a system.
Relations all the way down sounds like a missing noun. Relations between what? The structuralist answer is that the demand for a noun may be a grammatical habit. English prefers a subject before a verb. But the universe is under no obligation to provide one.
Modern physics makes that suggestion less strange than it sounds. Swap two electrons and nothing changes — not that no experiment could tell them apart, but that there is no difference for an experiment to find. Being this electron rather than that one is not among the properties an electron has. What it has is a role.
Max Tegmark’s Mathematical Universe Hypothesis takes the strongest step: the universe does not instantiate a mathematical structure. It is one. In its strongest form, every consistent mathematical structure exists in the same broad sense. Nobody could accuse Tegmark of conceptual timidity.
But such audacity risks explaining why our universe is mathematical by declaring every possible mathematics real. A theory that accommodates everything can become difficult to inconvenience with evidence, which is traditionally considered a disadvantage outside theology.
Tegmark’s hypothesis clarifies the question behind what Wigner called the unreasonable effectiveness of mathematics. Abstract structures developed in one context repeatedly describe phenomena in another, sometimes predicting effects not used to construct them.
Richard Hamming’s reply was to deny part of the puzzle. We notice the mathematics that fits, we build the mathematics we need, and we quietly forget the enormous quantity that describes nothing. Selection accounts for some of what the miracle was invoked to explain.
Structuralism disagrees. Mathematics works so well because mathematics studies structure, and structure is what reality is made from. If that is true, its effectiveness is not unreasonable. It is barely avoidable. An attractive answer.
But history advises against becoming too pleased with it.
The basement keeps moving
Worrall’s case begins with a genuine pattern. Fresnel believed light travelled as waves through an ether. The ether later disappeared from physics, but important mathematical relationships emergent in Fresnel’s theory survived into Maxwell’s account of electromagnetism.
That suggests science may be progressively recovering the structure of reality even while repeatedly misidentifying what carries it. The difficulty is that the relations change as well.
Newtonian gravity survives within general relativity, but only as an approximation: accurate when gravity is weak, speeds are low and nobody has done anything unnecessarily close to a black hole. Space and time, previously the fixed stage on which physics occurred, became part of the action.
Kuhn’s account of these episodes denied that the old theory survives inside the new one as a tidy special case. Mass in Newton and mass in relativity are not the same quantity measured more carefully. The word is shared; the concept is not. If he is right, whatever continuity science has must be found in the mathematics, because it will not be found in the vocabulary.
General relativity is itself expected to be incomplete because it does not fit cleanly with quantum mechanics. The theory that replaced Newton is now waiting, with some dignity, to be superceded in turn.
The history of physics is therefore not a succession of identical mathematical structures with the nouns crossed out. It is a succession of remarkably successful models whose limits become visible only when somebody produces a better one.
Larry Laudan made the wider objection in 1981. The history of science contains too many theories that predicted successfully while being seriously wrong about what existed. Success tells us that a model has caught something structurally real. However, it seems rarely, if ever to provide an itemised receipt for the ontology.
Kyle Stanford sharpened the objection. The recurring failure is not that earlier scientists were entirely wrong; it is that they could not conceive of an alternative theory that could beat the current champion, and nothing marks us out as the first generation to have imagined all the options.
Structural realism handles this better than ordinary realism because it expects the supposed objects to change. But it risks becoming suspiciously good at winning after the result is known.
Whatever survives is declared the structure. Whatever disappears is dismissed as scaffolding. That is the retrospective danger. Stathis Psillos pressed a related objection against Worrall: the line between structure and nature is not clean enough to bear the weight structural realism places on it.
Scientific theories repeatedly announce the ground floor has been found, only to subquently discover a basement.
Perhaps mathematics is steadily converging on the final structure of reality. Perhaps each successful geometry is merely the best available map of something deeper. Nothing in the history guarantees that the digging stops.
One theory, two universes
There is a second problem that no better experiment obviously resolves. Imagine that physics eventually produces a final theory: one mathematical structure capable of predicting every observable event, from the orbit of a galaxy to the decay of a particle.
Two accounts would still fit. The first says that the mathematical structure is the universe. The second says that it perfectly describes a universe whose underlying nature remains something else. A perfect model, just one that could be compressed into a description requiring fewer bits: the real real reality.
No experiment could choose between them. Any difference capable of being measured would already appear in the theory. The disagreement would not be waiting for a larger collider, a cleaner telescope or a grant application with more ambitious nouns.
It would have run out of observable consequences. James Ladyman calls this metaphysical underdetermination: the same physics supporting incompatible accounts of what exists, with nothing in the physics to separate them.
Occam’s razor favours the first account. If the additional substance changes nothing, predicts nothing and can never be detected, science has no reason to include it.
But Occam’s razor is a rule for constructing theories. It does not patrol the universe confiscating surplus ontology. Something can be explanatorily unemployed without being impossible. Elliott Sober separates razors of silence from razors of denial: declining to assert the extra substance is cheap, and denying it is a much stronger move requiring much stronger grounds.
The obvious objection is to point at E = mc² and call the argument closed. Nothing is more compact, nothing has been more thoroughly vindicated, and Albert Einstein derived it rather than fitting it to anything.
The usual reply is that the compactness is a choice of units. It is. But the choice is available only because distance and duration are the same kind of interval, exchangeable at one rate everywhere and for everyone. Newton’s universe has no such rate.
That rate is not bookkeeping; it is the finding. Take the units away and an equals sign is left standing. Mass and energy are not two quantities that track each other closely. They are one quantity written twice — for a body at rest, which is all the famous form covers. That is the stronger reading, earned in a single case.
So the honest conclusion is narrower. Mathematics plainly captures real structure. Mathematical relations connect phenomena first encountered separately, survive tests they were not designed merely to flatter and sometimes outlive the physical stories originally attached to them. Wigner’s puzzle is real.
What we cannot establish is whether structure is the whole of reality. Geometry sometimes describes the machinery, often in few bits with incredible fidelity. In general relativity, it appears to be the machinery. That does not prove that every successful geometry deserves the same promotion.
So, a proposal: keep the of, but hold it loosely.
The geometry of language.
The geometry of a protein.
The geometry of spacetime.
Perhaps, eventually, the geometry of reality.
The word may turn out to be unnecessary. We do not yet know enough to delete it. And even a perfect description of the universe would not arrive inside a human mind whole. It would have to be compressed, transmitted, read, checked and fitted into whatever the recipient already knew.
Whatever reality is made from, our access to it still arrives as information. Somebody has to unpack it.
Part III — Attention Is the Last Paywall — follows on Sunday 16 August. It begins where this essay ends: information must still be compressed, transmitted, read, checked and fitted into whatever the recipient already knows.
It then asks what becomes scarce when anything can be written instantly. The answer is not information, but the finite attention of the person receiving it.
Never mind Putnam and Quine, how about Kant, Carnap or Claude?