Most people, when they sit down to write a post explaining something, do so with the intention that their readers will reach the end of the article feeling less confused about the subject than when they started. Today, I am hoping to do the opposite. Today I am discussing quantum physics.

You see, there is a commonly held maxim, I think universally accepted, that if you think that you understand quantum physics then, in reality, you don't. The obvious corollary is that if you think you don't understand it, then there is a chance that in reality you do. So my hope is to leave you completely bewildered and befuddled. To run around the room saying ``No, that can't be! That doesn't make any sense!'' If that's you at the end of reading this, then congratulations! You are well on the way to becoming a quantum physicist.

Of course, there are different ways in which you can be confused. Some of them are good, and others bad. So I am not going to prevent your confusion, but, I hope, at the end of this, you will at least be confused about the right things.

In the last post I began a discussion about one reason why quantum physics isn't intuitive. I am focussing on how uncertainty is treated in quantum physics. The intuitive way of treating uncertainty numerically is to use probability. The reason for this is that a probability distribution is directly analogous to a frequency distribution, and probability can be used to predict measurements of frequencies. Frequencies are derived from counting objects, something which we are very familiar with. So while probabilities and frequencies are different (probabilities being things we compute from premises, frequencies being things we measure), the correspondence means that probabilities nonetheless are intuitive to us. That implies that any other way to classify uncertainty is unintuitive.

Like all mathematical systems, probability theory is based on a number of axioms and definitions. These axioms are not provable within the system, but once assumed everything else can be derived from them. So you put in a small number of axioms, and get out a vast array of useful results. There are different, equivalent, sets of axioms one could use. In one way of looking at it, you take A as an axiom, and from it deduce B. The other way would take B as an axiom, and from it deduce A. But either way, you have some statements as axioms and others as results derived from those axioms.

To use probability theory in a physical situation, you have to show that aspects of the physical system can be mapped onto a numerical representation where the same axioms behind probability theory apply. To be useful, this mapping between reality and representation must be reversible (i.e. it is possible to go back from the representation to reality), though it need not be, and I would argue cannot be, complete (i.e. not everything in reality can be represented numerically; and not everything in the representation is a genuine feature of reality).

In the previous post, I listed one set of these axioms which could be used to define probability theory. The terminology is defined in that post.

1. The possible outcomes can be expressed as a set of states, which I will call the basis. For example, in classical physics, we would express a particle at one location and momentum as one state; the same particle with a different location and momentum as a different state; and so on for every possible location and momentum.
2. The basis is irreducible.
3. The basis is complete.
4. These states in the basis are unique.
5. These states in the basis are orthogonal.
6. A being (for example a particle or system of particles) can only actually be in one of these states at a time; at that moment of time, the other states are potential. The being could in principle be in those states (there is nothing in the nature of the particle preventing it), but in practice happens not to be.
7. A probability is a number assigned to each of these states which represents our uncertainty that the particle is actually in that state.
8. The probability is a real number greater than or equal to zero.
9. A probability of zero implies that we are certain that the state isn’t actual.
10. A probability of one implies that we are certain that the state is actual.
11. The sum over all states of the probabilities is one (we are certain that the particle is in one state or another).
12. For two basis states A and B (so the being can’t be in both A and B simultaneously), A∪B represents a compound state where the particle is either in state A or state B. If P(A) represents the probability that the being is in state A (and so on), then P(A ∪ B) = P(A) + P(B).

Strictly speaking, of course, because probability is a type of deduction, every probability depends on the premises we put into that deduction. Thus every probability should be written as conditional upon those premises. For example, we might want the probability that the particle is in state A at time t₁ given that it was in state C at time t₀ assuming that it evolves under set of physical laws denoted by L. The notation for this is P(A(t₁)|C(t₀),L). Conditional probabilities of this sort are all we are interested in in physical studies. Everything in physics (of this type of problem, i.e. which state is the system is in at a given moment of time) depends on initial conditions and the laws which describe the system. Formally the conditional probability is defined by

where A∩C indicates that the particle was both in state C at time t₀ and state A at time t₁.

We also saw last time that probability theory doesn’t directly apply in quantum systems. This means that the representation of reality needed in quantum physics violates at least one of the axioms listed above. Instead of probabilities, we need to represent our uncertainty in a quantum system in some other way. I call this way a likelihood, and denote _Q as the likelihood that the being is in state A at time t₁ given that it was in state C at time t₀ and some laws of physics denoted by Q. The basis itself is denoted by the set {|A⟩,|B⟩,…}, where |A⟩ and |B⟩ and so on represent individual states. The statement that the likelihood that state |A⟩ is actual is c_A and the likelihood that state |B⟩ is actual is c_B is denoted using the expression

|ψ(t)⟩, which I will call a knowledge state, represents our knowledge of the state of the being at time t.

The question we want to ask is What is the likelihood that state A will be actual at time t₁ given our knowledge of the particle’s state |ψ(t₀)⟩ at time t₀ and applicable physical law Q? For this we need the comparison operator ⟨A(t₁)|. This is applied to a knowledge state |ψ⟩ to give a likelihood ⟨A(t₁)|ψ(t)⟩. The rule for applying the comparison to a state vector is ⟨A(t₁)|A(t₁)⟩ = 1 while ⟨A(t₁)|B(t₁)⟩ = 0 (as long as the state denoted by B is not the same as the state denoted by A). We can only compare states at the same time. If we want to compare a comparison state defined at one time t₁ with a knowledge state, |ψ(t₀)⟩ at a different time t₀, we first of all have to use our laws of physics Q to figure out what the corresponding knowledge state at time t₁,|ψ(t₁)⟩_Q,ψ(t0) would be.

So far I haven’t done anything except assume that a numerical representation of our uncertainty is possible, called that number the likelihood, and write down a notation to represent it. I have not proved that this notation is useful, that is, that it is applicable to reality. That requires experimental testing. In fact, I haven’t even defined what the likelihood means; right now it is just a number that in some way represents our uncertainty. But since this is a theoretical discussion, I’ll skip ahead and say that this notation is useful; it can be used to represent reality. It need not necessarily be the only way we could create an abstract representation of physical reality, but it is something that works, and all we need to proceed is one representation that works.

So now I need to list the axioms listed above, and state which of them are valid in quantum theory, and which are invalid, and if they are invalid what should replace them. I’ll discuss each axiom, but not in the same order. Technically, what I describe below is electrodynamics, the theory describing the electric and magnetic forces. We need to extend this framework to apply it to the other types of physical interaction, but the fundamental ideas are very similar, and the step from quantum electrodynamics to the full physical theory is much smaller than the step from classical to quantum physics. If you understand electrodynamics (which, of course, you won’t), then you are well on your way to understanding more complex physical theories.

1. There is a set of basis states. Yes. This is a firmer premise in quantum physics than classical physics. In classical physics, although one can express physical systems in terms of states, one doesn’t have to. In quantum systems one has no choice. We have to think of reality in terms of actual and potential states (albeit with complications I will discuss below). Mathematically, these states are described by the eigenstates of various operators, most importantly the time evolution operator, called the Hamiltonian. An operator is an intellectual process (recall this whole discussion focusses on an abstract representation of reality rather than reality itself) which takes in one state as input and spews out a superposition of different states multiplied by various numbers as outputs. For example, a matrix is one type of operator. It takes in one vector as input, and gives us another vector as output.

   The time evolution operator Ĥ describes how the system evolves over an infinitesimal time period δt. So

   (The is just a convention needed when we get to the finer details). Some systems stay in the same state over time, in which case

   E_* is known as the eigenvalue of the operator Ĥ and |ψ_*⟩ the eigenstate. Each operator will have numerous different eigenstates, each with their own distinct eigenvalues. In the case of the time evolution operator, the eigenvalue has a special name. It is called the energy.

   These energy eigenstates provide a natural way of defining a set of basis states. The one thing everyone agrees about beings is that they endure over time. The only states in this description which do endure over time are these energy eigenstates, so we use them to represent the potential states that beings could be in. At any given moment of time, one (baring a caveat that I will discuss below) of these eigenstates will be actual and the remaining states potential.

   Thus the language of actual and potential states fits in very nicely with quantum physics.

2. The basis is irreducible. Yes; while one can construct different representations from the basic system to describe compound objects (as one can in standard probability theory), the eigenstates of the Hamiltonian describing fundamental physics is irreducible.
3. The basis is complete. Yes. This system can describe any possible state of the system.
4. A likelihood is a number assigned to each of these states which represents our uncertainty that the particle is actually in that state. Still holds in quantum physics.
5. A likelihood of zero implies that we are certain that the state isn’t actual. Yes, in the conventional way of expressing things (in terms of complex numbers). I’m a little unconventional in by description below, but there is still a likelihood that implies a certainty of non-existence.

6. The likelihood is a real number greater than or equal to zero. No. This is where we start diverging from probability theory.

   The problem is that in classical physics, there are only two choices. An electron would either exist or it doesn’t. Therefore it is quite reasonable to represent our uncertainty of its existence by a line segment between those two states. In quantum physics, however, there are numerous different states in which the electron could exist, and these states are all distinct from each other. We have to distinguish between them. This means that we need to express the likelihood in terms of two numbers, one distinguishing between these different states, and one expressing our certainty of the electron existing. The parameter distinguishing the states needs to be periodic (the reason for this is buried in the details of the construction which I haven’t discussed, and is confirmed by experiment), so if you keep adding to it, eventually you get back to where you started. Thus it is best to represent the uncertainty by a circle of radius one. The distance from the centre of the circle represents whether or not the particle exists, with the centre meaning that we are certain that it doesn’t exist and the circumference meaning that we are certain that it does, with the angle used to distinguish between the different states.

   Thus the likelihood can be represented by two numbers, one a distance from the centre of a circle and the other a direction.

   We add together two likelihoods as we add together any two geometrical lines. We represent each point as an arrows going from the centre of the circle to the point in question. When we add the two likelihoods together, we simply move the tail of one arrow to the head of the other. This will provide us with a new point within a circle (this time a circle of radius two).

   Thus it is possible to add together two likelihoods on the circumfrence of the circle, representing certainty of existence, and get a point at the circle’s centre, representing a certainty of a lack of existence. This is generally accepted to be weird. But it is what we see, for example in the diffraction experiment described at the end of the last post.

   Of course, the classical treatment of uncertainty is just a subset of the quantum theory of uncertainty. In classical probability, we just restrict ourselves to an angle to zero. Thus quantum theory of uncertainty can do everything that the classical theory can do, but it can also do more. And that makes it better.

7. A likelihood of one implies that we are certain that the state is actual. Not quite. Any point on the radius of the circle indicates that we are certain that the state is actual; the centre of the circle indicates that the we are certain that the state is not actual. The degree of confidence we have in the state being actual is the distance from the centre of the circle. If v is the vector describing the location of the point, then this distance is denoted by |v|. |v| = 1 indicates certainty.
8. The sum over all states of the likelihoods is one. No. because likelihoods have directions as well as magnitudes, there is no garantee that when you add all of them up, you will get one. In practice, not even the sum over all states of |v| is one.

9. These states in the basis are unique. Not necessarily. There is not a unique basis. There are numerous different ways in which we can parametrise the likelihood. The likelihood can be expressed as a distance plus an angle, but an angle lies between two lines. One of these lines is the vector to the point that represents our confidence; what is the other? We have to arbitrarily choose one direction as the basis of our measurements. We are free to pick any direction as our line which marks zero degrees. This freedom is known by physicists as gauge symmetry. Because this choice is arbitrary, no physical result can depend on it, which means that physics can only depend on relative rather than absolute angles. The angle between two different likelihoods is independent of the gauge and thus could be physical; the absolute angle is not and thus can’t be. Thus nothing we calculate which we want to compare against measurement can depend on a choice of gauge. Each gauge represents a different, but ultimately equivalent, way of splitting reality into states.

   We can think of this choice as analogous to the choice about where we draw up our coordinate axes in Cartesian geometry. We usually draw one axis as horizontal and the other as vertical. But we don’t have to. We can pick any two non-parallel directions. The mathematics becomes harder if these directions are not at right-angles to each other, but it can still be done. Each point on the paper is still given a unique identifier by marking off perpendiculars to the axes. Each point is still described by a unique pair of numbers in any coordinate system; and in that sense they are all equally good. But the numbers will be different in different coordinate systems. The numbers alone are not good enough to identify where the point is. When we identify the point, we need to both give the numbers, and specify which coordinate system we use. Once we have done this, anyone can take our measurement, convert it into their own preferred coordinate system, and will be able to precisely point where the point on the graph is.

   This is more pronounced when we consider another physical quantity, known rather unfortunately as the spin of particles. The spin for an electron can take two states (the operator describing spin has two eigenvalues), known as spin up and spin down. But it also has a direction. Each direction represents a new way we can represent the basis. These bases are all equivalent – each of them can describe all the information required to describe the electron, and we can convert from one basis to another – but we have to pick one, and once again this choice is arbitrary. We usually unconsciously make it when we set up our experiment.

   So when we specify the state of an electron, we need to give both whether it is spin up or spin down, and, because we have a choice of basis, we must also state which basis we are using to describe it. This is entirely new. It is not something we see in classical physics, which assumes that there is only one possible basis of states one can use to describe the system. One again, it is impossible for us to imagine what it means in practice. The situation is just wholly alien to our experiences; our imagination is just too limited to cope with it. But thanks to mathematics, we can understand it intellectually.

10. These states in the basis are orthogonal. This is a little more complicated. The eigenstates within a given basis are orthogonal. So once we fix the basis, we cannot be in both eigenstates at the same time. So, for example, once we pick one direction for the spin, the electron must be either spin up in that direction or spin down in that direction but it can’t be both at the same time. This is all very intuitive.

    The problem comes because of our freedom to choose a different basis. Each basis can fully describe the system. Each basis allows two possible states, spin up and spin down. But the states in one basis are not orthogonal to the states in another basis.

    So let us pick to different directions, D₁ and D₂; it doesn’t matter which directions we choose as long as they are not parallel to each other. We construct a basis B₁ which measures the spin along direction D₁, and a basis B₂ which measures the spin along direction D₂. Either one of these allows us to describe any state in the system; we construct a knowledge state which is a superposition of the spin up and spin down directions.

    The Hamiltonian doesn’t help us distinguish between these bases, because all their states have the same energy, at least until we perform a measurement. We can only acknowledge that there isn’t a singe unique way to describe nature, and include a description of the basis alongside specifying the eigenstate. We need both to fully describe the particle.

    Now the spin up state in basis B₁, which I will denote as |U₁⟩, is not orthogonal to the spin up state in basis B₂, so it can be spin up in B₁ and spin up in B₂ (|U₂⟩) at the same time. The spin up state in basis B₁ is also not orthogonal to the spin down state in basis B₂ (|D₂⟩), so can also be spin up in B₁ and spin down in B₂. Thus if the electron is spin up in basis B₁, it could be either spin up or spin down in basis B₂; but (and this is where it gets weird) it must be one or the other. But we don’t and can’t know which one it is. Saying that the electron is in state |U₁⟩ is enough to completely determine the state of the electron. But whether it is also in state |U₂⟩ or |D₂⟩ is completely undetermined. We can compute the likelihood for each occurrence easily enough, using ⟨D₂|U₁⟩ and so on, but that is the best we can do.

    When we perform a measurement, we adjust the Hamiltonian, so that the energy levels in one direction are distinguished. Thus the particle will naturally fall into a state in that basis. When we take the measurement, we will therefore observe it to be in one or the other of those states.

    Once again, our imagination fails us (and if you think that you can imagine what I just described, then you didn’t understand what I was trying to say). We can understand intellectually using mathematics, but it is impossible to imagine and therefore impossible to describe in words or construct an analogy to explain it. All we can do is study and understand the mathematics, and use that as the basis for our philosophy.

    Note, however, that these states refer mostly to our knowledge of the situation. When we take a measurement, our knowledge changes. We know after the measurement that the particle is in one particular state in a particular basis.

11. Only one state can be actual at a time. Because states in the same basis are orthogonal, in one basis this is true. The particle can either be spin up or spin down but not both. However, when we talk about the spin along a different direction neither the spin up or spin down states are actual. Aristotle’s philosophy is well placed to deal with this. If the particle is actually spin up in direction D₁, then it is both potentially spin up along D₂ and potentially spin down along D₂.

    This statement, however, seems to be complicated by the existence of different bases and in particular the superposition principle. If the electron is spin up along D₁, then it is neither in one state or another along D₂. Some people describe this as it being sort of in both states at the same time. The reason people make this mistake is that they thing that things can be in either actual existence or non-existence. They forget about Aristotle’s middle ground, potential existence. Once we put this in place, I believe that only one state can be actual at a time, but the state is determined both by the eigenvalue and the underlying basis.

    The |U₂⟩ and |D₂⟩ both exist with the |U₁⟩ state, but exist potentially rather than actually. We cannot say which of these potentials will be actualised when we take the measurement (the system is indeterminate, that is it is not possible to predict the outcome), but we can calculate the likelihoods for each result.

    Thus in quantum physics, we cannot be satisfied with the dichotomy between existence and non-existence. Instead, we have to add a third option. Now Aristotle knew about as much quantum physics as a below average whelk. But when it came to general philosophical principles, he knew considerably more than the whelks, and was even further ahead of the Germans. He was aware that the third option was required to account for the reality of change. Now quantum physics is all about transitions from one state to another. So even though Aristotle didn’t understand quantum physics, because he started by focussing on the right question, he nonetheless developed the groundwork of the philosophical basis required, not to understand it (Feynman forbid that anyone could dare to understand!), but at least to be confused about the right things. So Aristotle introduced an intermediate state between actual existence and non-existence, namely potential existence. And he built his entire philosophy around this concept. The primary reason why post-Cartesian philosophy is a complete failure and gets almost everything wrong is that each branch of it starts out by denying the possibility of potential existence. But to do so just leads to absurdity upon absurdity.

    This potential existence is precisely what we observe in the case of spin, and other cases where there is not a unique basis. The spin up and spin down spins in other directions neither exist not don’t exist; so we stick them into a third state of potential existence. We are justified in using Aristotle’s term because what Aristotle meant by potential existence for is exceptionally close to what a physicist means in this case. (Note, however, that I am using potentia in two senses; both to describe the non-actual eigenstates in the same basis, and to describe the eigenstates of a non-orthogonal basis to the actual state).

    However, the quantum physicist has done better than Aristotle. All Aristotle was able to do was list the potentia. The physicist can not only list the potentia, but calculate the likelihood of each of them occurring in a given circumstance. This means that the physicist can make far more detailed predictions and be tested far more rigorously than Aristotle and his disciples could ever dream of. I do not believe that Aristotle is the end of philosophy (he made mistakes, most importantly in his dismissal of geometrical physics); but he is the beginning, and we would do well to go back to the road he started down.

    In quantum physics, we often write states in terms of superpositions. For example, one can write

    What this means is that if state |U₁⟩ exists actually, then states |U₂⟩ and |D₂⟩ exist potentially, with the magnitude of the likelihood in each case. Likelihoods are specified by both magnitude and direction. The magnitude is listed by that . The direction is specified by our choice of the B₂ basis. If we choose a different basis, we can again express |U₁⟩ as a superposition of the spin up and spin down states; both of these would again exist potentially, but with different likelihoods.

12. The corresponding expression for quantum likelihoods is

    So when considering two alternatives, we add together states rather than the likelihoods. This golds together even when the states are in a different basis to each other.

    Now, we can also construct a comparison state from this equation,

    Combining the comparison state with the knowledge state gives, if |A⟩ and |B⟩ are orthogonal (i.e. from the same basis),

    
    So if we write

    and so on, we see that this P satisfies the defining equation for probability. If we use a single basis for the whole the calculation, we can also show (it involves getting a bit deeper into the mathematics than I am willing to do here) that the other axioms of probability theory are satisfied for this quantity. Thus

    is a probability. And because it is a probability, we can use it to make predictions for frequencies, and thus link it to experimental measurements.

Expressing uncertainty using likelihoods and quantum states is different than using probabilities. It is a more general approach. The most important difference is that it allows us to consider systems where the basis is not unique: there are different ways in which we can express the information contained in the physical reality. Probability theory, and classical physics, assumes that the basis is unique. It assumes that all irreducible ways of describing the possible outcomes are ultimately identical to each other. But when we map reality to a numerical representation in practice, we find that this axiom of probability theory is not satisfied. We therefore need something which allows us to express uncertainty without this axiom. That led to the development (inspired by the already developed theory of optics) of probability amplitudes (what I have been calling likelihoods). This is a more general system, and needs fewer axioms. One can impose the remaining axioms of probability theory, and then the theory of likelihoods becomes functionally equivalent to probability theory. But one need not so so, and if one doesn’t, then one will get what seem like weird results.

We can’t imagine the rotation of the basis used to describes physical states. We see a black cow in a field. Another day, we don’t see the cow. From this we construct two possible states, where the cow is either present or absent. Effectively, we have constructed a basis for the possible states of a being (the cow) which contains two possible states, present in the field or absent. And we can’t imagine any other way of dividing the system up. Because physically, the cows are either there or they aren’t.

But quantum physics gives us more options. If |cow₁⟩ denotes the state where the cow is present and |cow₀⟩ denotes the state where the cow is absent, then the natural basis to use to describe this system is the set {|cow₁⟩,|cow₀⟩}. The natural way of describing this system is to restrict the system to these two states: either the cow is actually present, or it isn’t. But, in practice, there is another option, namely that the cow exists potentially. The natural way of thinking can’t cope with this, but quantum physics can, by writing a state as a superposition of the two states

for any angle θ.

When it comes to cows, this superposition of states to indicate potential existence isn’t obviously useful. But when it comes to quantum particles, it is necessary that we include states such as this.

But if we say that the cow is in this state, doesn’t that mean that there is only potentiality and no actuality? No. This is where the non-uniqueness of the basis saves us. We naturally think in terms of the basis of two states, the cow existing or it not existing. But we don’t have to. Once we allow superpositions, then there are other ways of describing the system. We can construct a basis where the cow’s existence or non-existence is described by the states

And in this basis the cow exists actually.

To specify actuality, we need to describe the state that is actual (as Aristotle said, after a bit of translation) and a basis which has that state as a member (as Aristotle didn’t say; well nobody’s perfect).

The quantum way of thinking can describe all the same circumstances as the more intuitive way, but because the quantum theory of uncertainty is built on less restrictive axioms than the classical (or intuitive) theory of uncertainty, it is applicable in other circumstances as well. And we need those other circumstances to describe the physical world.

Quantum physics is weird, in part, because we are forced to use these rotated bases if we are to describe reality. We are forced to consider cases where the cow could potentially exist or could potentially not exist. And thus we need to expand our notions of uncertainty to allow us to consider this.

So I hope that you are suitably confused by now. If so, you might have understood what I was trying to say. Quantum physics is weird. But it works.