Philip Ball worked for over twenty years as an editor for Nature, writes regularly in the scientific and popular media, and has authored many books on the interactions of the sciences, the arts, and the wider culture. His most recent books include Patterns in Nature, Invisible, and Serving the Reich. He lives in London.

Beyond Weird

Why Everything You Thought You Knew about Quantum Physics is Different

The University of Chicago Press

No one can say what

quantum mechanics means (and this is a book about it)

Richard Feynman said that in 1965. In the same year he was awarded the Nobel Prize in Physics, for his work on quantum mechanics.

In case we didn't get the point, Feynman drove it home in his artful Everyman style. 'I was born not understanding quantum mechanics,' he exclaimed merrily, '[and] I still don't understand quantum mechanics!' Here was the man who had just been anointed one of the foremost experts on the topic, declaring his ignorance of it.

What hope was there, then, for the rest of us?

Feynman's much-quoted words help to seal the reputation of quantum mechanics as one of the most obscure and difficult subjects in all of science. Quantum mechanics has become symbolic of 'impenetrable science', in the same way that the name of Albert Einstein (who played a key role in its inception) acts as shorthand for scientific genius.

Feynman clearly didn't mean that he couldn't do quantum theory. He meant that this was all he could do. He could work through the math just fine – he invented some of it, after all. That wasn't the problem. Sure, there's no point in pretending that the math is easy, and if you never got on with numbers then a career in quantum mechanics isn't for you. But neither, in that case, would be a career in fluid mechanics, population dynamics, or economics, which are equally inscrutable to the numerically challenged.

No, the equations aren't why quantum mechanics is perceived to be so hard. It's the ideas. We just can't get our heads around them. Neither could Richard Feynman.

His failure, Feynman admitted, was to understand what the math was saying. It provided numbers: predictions of quantities that could be tested against experiments, and which invariably survived those tests. But Feynman couldn't figure out what these numbers and equations were really about: what they said about the 'real world'.

One view is that they don't say anything about the 'real world'. They're just fantastically useful machinery, a kind of black box that we can use, very reliably, to do science and engineering. Another view is that the notion of a 'real world' beyond the math is meaningless, and we shouldn't waste our time thinking about it. Or perhaps we haven't yet found the right math to answer questions about the world it purports to describe. Or maybe, it's sometimes said, the math tells us that 'everything that can happen does happen' – whatever that means.

This is a book about what quantum math really means. Happily, we can explore that question without having to look very deeply into the math itself. Even what little I've included here can, if you prefer, be gingerly set aside.

I am not saying that this book is going to give you the answer. We don't have an answer. (Some people do have an answer, but only in the sense that some people have the Bible: their truth rests on faith, not proof.) We do, however, now have better questions than we did when Feynman admitted his ignorance, and that counts for a lot.

What we can say is that the narrative of quantum mechanics – at least among those who think most deeply about its meaning – has changed in remarkable ways since the end of the twentieth century. Quantum theory has revolutionized our concept of atoms, molecules, light and their interactions, but that transformation didn't happen abruptly and in some ways it is still happening now. It began in the early 1900s and it had a workable set of equations and ideas by the late 1920s. Only since the 1960s, however, have we begun to glimpse what is most fundamental and important about the theory, and some of the crucial experiments have been feasible only from the 1980s. Several of them have been performed in the twenty-first century. Even today we are still trying to get to grips with the central ideas, and are still testing their limits. If what we truly want is a theory that is well understood rather than simply one that does a good job at calculating numbers, then we still don't really have a quantum theory.

This book aims to give a sense of the current best guesses about what that real quantum theory might look like, if it existed. It rather seems as though such a theory would unsettle most if not all we take for granted about the deep fabric of the world, which appears to be a far stranger and more challenging place than we had previously envisaged. It is not a place where different physical rules apply, so much as a place where we are forced to rethink our ideas about what we mean by a physical world and what we think we are doing when we attempt to find out about it.

In surveying these new perspectives, I want to insist on two things that have emerged from the modern renaissance – the word is fully warranted – in investigations of the foundations of quantum mechanics.

First, what is all too frequently described as the weirdness of quantum physics is not a true oddity of the quantum world but comes from our (understandably) contorted attempts to find pictures for visualizing it or stories to tell about it. Quantum physics defies intuition, but we do it an injustice by calling that circumstance 'weird'.

Second – and worse – this 'weirdness' trope, so nonchalantly paraded in popular and even technical accounts of quantum theory, actively obscures rather than expresses what is truly revolutionary about it.

Quantum mechanics is in a certain sense not hard at all. It is baffling and surprising, and right now you could say that it remains cognitively impenetrable. But that doesn't mean it is hard in the way that car maintenance or learning Chinese is hard (I speak with bitter experience of both). Plenty of scientists find the theory easy enough to accept and master and use.

Rather than insisting on its difficulty, we might better regard it as a beguiling, maddening, even amusing gauntlet thrown down to challenge the imagination.

For that is indeed what is challenged. I suspect we are, in the wider cultural context, finally beginning to appreciate this. Artists, writers, poets and playwrights have started to imbibe and deploy ideas from quantum physics: see, for instance, plays such as Tom Stoppard's Hapgood and Michael Frayn's Copenhagen, and novels such as Jeanette Winterson's Gut Symmetries and Audrey Niffenegger's The Time Traveler's Wife. We can argue about how accurately or aptly these writers appropriate the scientific ideas, but it is right that there should be imaginative responses to quantum mechanics, because it is quite possible that only an imagination sufficiently broad and liberated will come close to articulating what it is about.

There's no doubt that the world described by quantum mechanics defies our intuitions. But 'weird' is not a particularly useful way to talk about it, since that world is also our world. We now have a fairly good, albeit still incomplete, account of how the world familiar to us, with objects having well-defined properties and positions that don't depend on how we choose to measure them, emerges from the quantum world. This 'classical' world is, in other words, a special case of quantum theory, not something distinct from it. If anything deserves to be called weird, it is us.

* * *

Here are the most common reasons for calling quantum mechanics weird. We're told it says that:

• Quantum objects can be both waves and particles. This is wave-particle duality.

• Quantum objects can be in more than one state at once: they can be both here and there, say. This is called superposition.

• You can't simultaneously know exactly two properties of a quantum object. This is Heisenberg's uncertainty principle.

• Quantum objects can affect one another instantly over huge distances: so-called 'spooky action at a distance'. This arises from the phenomenon called entanglement.

• You can't measure anything without disturbing it, so the human observer can't be excluded from the theory: it becomes unavoidably subjective.

• Everything that can possibly happen does happen. There are two separate reasons for this claim. One is rooted in the (uncontroversial) theory called quantum electrodynamics that Feynman and others formulated. The other comes from the (extremely controversial) 'Many Worlds Interpretation' of quantum mechanics.

Yet quantum mechanics says none of these things. In fact, quantum mechanics doesn't say anything about 'how things are'. It tells us what to expect when we conduct particular experiments. All of the claims above are nothing but interpretations laid on top of the theory. I will ask to what extent they are good interpretations (and try to give at least a flavour of what 'interpretation' might mean) – but I will say right now that none of them is a very good interpretation and some are highly misleading.

The question is whether we can do any better. Regardless of the answer, we are surely being fed too narrow and too stale a diet. The conventional catalogue of images, metaphors and 'explanations' is not only clichéd but risks masking how profoundly quantum mechanics confounds our expectations.

It's understandable that this is so. We can hardly talk about quantum theory at all unless we find stories to tell about it: metaphors that offer the mind purchase on such slippery ground. But too often these stories and metaphors are then mistaken for the way things are. The reason we can express them at all is that they are couched in terms of the quotidian: the quantum rules are shoehorned into the familiar concepts of our everyday world. But that is precisely where they no longer seem to fit.

* * *

It's very peculiar that a scientific theory should demand interpretation at all. Usually in science, theory and interpretation go together in a relatively transparent way. Certainly a theory might have implications that are not obvious and need spelling out, but the basic meaning is apparent at once.

Take Charles Darwin's theory of evolution by natural selection. The objects to which it refers – organisms and species – are relatively unambiguous (if actually a little challenging to make precise), and it's clear what the theory says about how they evolve. This evolution depends on two ingredients: random, inheritable mutations in traits; and competition for limited resources that gives a reproductive advantage to individuals with certain variants of a trait. How this idea plays out in practice – how it translates to the genetic level, how it is affected by different population sizes or different mutation rates, and so on – is really rather complex, and even now not all of it is fully worked out. But we don't struggle to understand what the theory means. We can write down the ingredients and implications of the theory in everyday words, and there is nothing more that needs to be said.

Feynman seemed to feel that it was impossible and even pointless to attempt anything comparable for quantum mechanics:

We can't pretend to understand it since it affronts all our commonsense notions. The best we can do is to describe what happens in mathematics, in equations, and that's very difficult. What is even harder is trying to decide what the equations mean. That's the hardest thing of all.

Most users don't worry too much about these puzzles. In the words of the physicist David Mermin of Cornell University, they 'shut up and calculate'. For many decades quantum theory was regarded primarily as a mathematical description of phenomenal accuracy and reliability, capable of explaining the shapes and behaviours of molecules, the workings of electronic transistors, the colours of nature and the laws of optics, and a whole lot else. It would be routinely described as 'the theory of the atomic world': an account of what the world is like at the tiniest scales we can access with microscopes.

Talking about the interpretation of quantum mechanics was, on the other hand, a parlour game suitable only for grandees in the twilight of their career, or idle discussion over a beer. Or worse: only a few decades ago, professing a serious interest in the topic could be tantamount to career suicide for a young physicist. Only a handful of scientists and philosophers, idiosyncratically if not plain crankily, insisted on caring about the answer. Many researchers would shrug or roll their eyes when the 'meaning' of quantum mechanics came up; some still do. 'Ah, nobody understands it anyway!'

How different this is from the attitude of Albert Einstein, Niels Bohr and their contemporaries, for whom grappling with the apparent oddness of the theory became almost an obsession. For them, the meaning mattered intensely. In 1998 the American physicist John Wheeler, a pioneer of modern quantum theory, lamented the loss of the 'desperate puzzlement' that was in the air in the 1930s. 'I want to recapture that feeling for all, even if it is my last act on Earth', Wheeler said.

Wheeler may indeed have had some considerable influence in making this deviant tendency become permissible again, even fashionable. The discussion of options and interpretations and meanings may no longer have to remain a matter of personal preference or abstract philosophizing, and if we can't say what quantum mechanics means, we can now at least say more clearly and precisely what it does not mean.

This re-engagement with 'quantum meaning' comes partly because we can now do experiments to probe foundational issues that were previously expressed as mere thought experiments and considered to be on the border of metaphysics: a mode of thinking that, for better or worse, many scientists disdain. We can now put quantum paradoxes and puzzles to the test – including the most famous of them all, Schrödinger's cat.

These experiments are among the most ingenious ever devised. Often they can be done on a benchtop with relatively inexpensive equipment – lasers, lenses, mirrors – yet they are extraordinary feats to equal anything in the realm of Big Science. They involve capturing and manipulating atoms, electrons or packets of light, perhaps one at a time, and subjecting them to the most precise examination. Some experiments are done in outer space to avoid the complications introduced by gravity. Some are done at temperatures colder than the void between the stars. They might create completely new states of matter. They enable a kind of 'teleportation'; they challenge Werner Heisenberg's view of uncertainty; they suggest that causation can flow both forwards and backwards in time or be scrambled entirely. They are beginning to peel back the veil and show us what, if anything, lies beneath the blandly reassuring yet mercurial equations of quantum mechanics.

Such work is already winning Nobel Prizes, and will win more. What it tells us above all else is very clear: the apparent oddness, the paradoxes and puzzles of quantum mechanics, are real. We cannot hope to understand how the world is made up unless we grapple with them.

Perhaps most excitingly of all, because we can now do experiments that exploit quantum effects to make possible what sounds as though it should be impossible, we can put those tricks to work. We are inventing quantum technologies that can manipulate information in unprecedented ways, transmit secure information that cannot be read surreptitiously by eavesdroppers, or perform calculations that are far beyond the reach of ordinary computers. In this way more than any other, we will all soon have to confront the fact that quantum mechanics is not some weirdness buried in remote, invisible aspects of the world, but is our current best shot at uncovering the laws of nature, with consequences that happen right in front of us.

What has emerged most strongly from this work on the fundamental aspects of quantum theory over the past decade or two is that it is not a theory about particles and waves, discreteness or uncertainty or fuzziness. It is a theory about information. This new perspective gives the theory a far more profound prospect than do pictures of 'things behaving weirdly'. Quantum mechanics seems to be about what we can reasonably call a view of reality. More even than a question of 'what can and can't be known', it asks what a theory of knowability can look like.