I cannot provide you with a complete answer as to how to deal in details with Penrose's argument. I have not even read his latest book as yet, "Fashion, Faith, and Fantasy in the New Physics of the Universe," so everything that I know about his arguments derives from my reading of his contribution to the commemorative issue "The Future of Theoretical Physics and Cosmology" in the occasion of Hawking anniversary, called "On the Instability of Extra Space Dimensions". I have seen that the issues discussed in that paper are repeated, almost verbatim, in Penrose's "The Road to Reality" (chapter 31, if I recall well).
My belief is that the basic issue concerning that argument made by Penrose, on the matter of the excess of functional degrees of freedom on higher-dimensional theories, lies on the fact Penrose ignored the implications of the holographic principle for the counting of degrees of freedom and how it is related to dimensional reduction in quantum gravity. For example, we have learned from black hole thermodynamics that the entropy of a black hole is related to the former horizon area and not its volume, as one would naively expect if one is to ignore holography and its inherent dimensional reduction.
To understand it, we should here recall that the entropy of a system is related to the availability of its micro-states, and the latter is determined by the functional degrees of freedom of the system. This being the case, the fact that the entropy of a black hole is proportional to the horizon area implies that the degrees of freedom involved here (the available micro-states) are to be associated to the horizon, and not to the full volume of the black hole. This is an instance of how the holographic principle induces dimensional reduction.
I propose therefore that the excess of functional degrees of freedom pointed out by Penrose in the paper cited above (and possibly the book quoted by this question's author) can be eliminated if one carefully considers the holographic principle. In the opening pages of Dirac's monograph on QM, Dirac declares the death of classical mechanics by the inability of its basics premises to account for the elimination of the excess degrees of freedom which makes classical theory incapable of accounting for the specific heat of ordinary matter and also responsible for the ultraviolet catastrophe. Quantization reduces the system's available degrees of freedom, which is necessary for the correct application of statistical mechanics for deriving completely the thermodynamics of ordinary matter. Similarly, I believe that the holographic principle is fully necessary for calculating the degrees of freedom for a theory of quantum gravity, which is the case of ST (Penrose's main target), something that is ignored from Penrose's arguments.
If this is indeed the case, it would be interesting to note further that all these arguments made by Penrose against higher dimensional theories seems to be by now more than a decade old, and he continuously says that string theory community ignores his criticism. If my arguments are correct, then it is not true that the ST community ignored this issues, is just that is has been already solved by holography decades earlier. Consider, from example, this paper, "Dimensional Reduction in Quantum Gravity," by 't Hooft. It has been available on the arXiv since 1993, while the paper "On the Instability of Extra Space Dimensions" which I quoted above and where the criticism by Penrose is first addressed in print, dates from 2003, a decade later!
Note. The speculative remarks written below are actually wrong, as has been pointed out by Greg Graviton in the comment section. The interested reader is advised to read Greg's comment immediately, otherwise the remainder of this answer should be ignored. This author is sorry for any possible inconvenience.
Technical remark. Now, what I have to say below has to do with the reliability of the method employed by Penrose in order to count the degrees of functional freedom for a field theory. I think that his method is doubtful even if holography is ignored. I do not think it is any surprise that a higher dimensional theory should posses more degrees of freedom than a lower dimensional one. (For example, in ST we have a lot of very important additional fields in the string spectrum, like the dilaton or Ramond fields, each of which have been identified with an important role in the dynamics of the theory.) What Penrose seems to be arguing is that the number of degrees of freedom that exists in higher-dimensional theories is far greater than what is expected. I think that Penrose do not provide any rigorous mathematical estimation of what is an acceptable excess of functional degrees of freedom, or any way of how one should estimate them.
Instead, he based his arguments on a very informal superposition of Wheeler notation for calculating degrees of freedom of a theory and the Cartan characters that appears in the theory of exterior differential systems. The idea is (very primitively speaking) that, given a classical field theory, represented as a set of partial differential equations, one can represent this system in a coordinate-free manner in terms of what is called an exterior differential system (an ideal of the exterior algebra in an appropriate configuration space), so that a solution to this system is then represented as an integral manifold of the exterior forms that constitute that ideal. Some invariant quantities, say \((\sigma_1,...,\sigma_N)\), associated with these integral manifolds, and which are a characteristic of that system of PDEs, forms what one calls the Cartan characters. These characters are related to the coefficients of the polynomial \(P_{(\sigma_1,...,\sigma_N)}\) that appears in the notation used by Wheeler and Penrose as \(\infty^{P_{(\sigma_1,...,\sigma_N)}(\infty)}\) denoting the functional degrees of freedom for that system.
Remark. To see how one can provide an invariant, namely, geometric and coordinate-free, meaning to these Cartan characters \((\sigma_1,...,\sigma_N)\), I suggest you to consult Cartan's classic monograph "Les systèmes différentiells extérieurs et leurs application en géométrie," in particular sections 68 and 69. A modern description can be found in Bryant et al textbook on "Exterior Differential Systems," particularly the first three sections of chapter 3. In these texts you can even find that argument that Penrose cites in his paper, the 1-d. heat equation. The notation \(\infty^{F\times \infty^D}\) for the functional freedom of a theory for a field of \(F\) components in \(D\)--spatial dimensions, was used all the time by Wheeler in his discussions of theories of Superspace (the space of all space geometries), see for example "Superspace and the Nature of Quantum Geometrodynamics", in DeWitt, C. M., Wheeler, J. A., (Eds.) Battelle Rencontres.
What I am going to argue is that this procedure for counting degrees of freedom for a field theory actually suffers from an overcounting of the possible set of initial configurations, principally when one of the dimensions is compact, say, with periodic boundary conditions. The geometric analysis of Cartan, using exterior systems, can be seen as a generalization of the Cauchy–Kowalevski theorem, which applies for arbitrary analytic solutions. Physically, however, we are interested in solutions which are (in quantum theory) normalizable, have a nice asymptotic behavior and, if periodic conditions are imposed, have discrete spectrum. This is going to severely restrict the counting of initial states.
For definiteness, let us deal with a Schrodinger equation \(i\partial_t\phi= \mathcal{H} \phi\) for a field \(\phi:\mathbb{R}\times\mathbb{E}^D\longrightarrow\mathbb{F}\), where we may have a D-Euclidean space \(\mathbb{E}^D=\mathbb{R}^D\), a D-torus \(\mathbb{E}^D=S^D\), or even a product of these, say, \(\mathbb{E}^D=\mathbb{R}^{D_1}\times S^{D_2}\), and \(\mathbb{F}\) may be the complex field \(\mathbb{F}=\mathbb{C}\) for a spinless particle, or any Clifford algebra (for example, generated by the Dirac matrices for spin one-half particles) of F dimensions, so that \(\phi\) is actually a F-components field. Instead of applying a discretization of the Cauchy data for our system of PDEs in order to derive the degrees of functional freedom, lets consider its Fourier decomposition.
1. Euclidean Case: \(\mathbb{E}^D=\mathbb{R}^D\). The momentum states \(|\vec{P}\rangle\) in \(\mathbb{E}^D\) are the plane-waves \(\langle \vec{R}|\vec{P}\rangle=\Phi_{\vec{P}}(\vec{R})=e^{i\vec{P} \cdot \vec{R}/(2\pi)^{D/2}}\) and a general solution to the Cauchy problem above can be written as \(\phi(t,\vec{R})=\langle\vec{R}|e^{i\mathcal{H}t}|\phi\rangle\) where the initial configuration is given by the Fourier decomposition \(|\phi\rangle=\int_{\mathbb{R}^D}a(\vec{P})|\vec{P}\rangle d^D\vec{P}\), or in the position representation,
\[\phi_0(\vec{R})=\langle \vec{R}| \phi \rangle=\int_{\mathbb{R}^D}a(\vec{P}) e^{i\vec{R}\cdot \vec{P}} \frac{d^D\vec{P}}{(2\pi)^{D/2}}.\]
This means that the space of all available initial configurations, which is nothing but the Cauchy data for our problem, is
\[\mathcal{M}=\{ a(P^1,...,P^D)\in\mathbb{F};\vec{P}=(P^1,...,P^D)\in \mathbb{R}^D \}=\mathbb{F}^{\mathbb{{R}^D}},\]
where we have used set theory notation \(A^B = \{f:B \longrightarrow A\}\). Since the algebra \(\mathbb{F}\) is supposed to be F-dimensional, so that our field \(\phi \in \mathbb{F}\) have F components, the cardinality of the "configuration" space \(\mathcal{M}\) is
\[o(\mathcal{M})=\mathfrak{c}^{F\times \mathfrak{c}^D},\]
where \(\mathfrak{c}=o(\mathbb{R})\) is the continuum, in the sense of Cantor. From cardinal arithmetic, \[\mathfrak{c}^{F\times \mathfrak{c}^D} = (2^{\aleph_{0}})^{\mathfrak{c}}=2^{\aleph_0\times\mathfrak{c}}=2^\mathfrak{c}=\beth_2,\]
(where \(\aleph_0 = o(\mathbb{N})\) is the cardinality of the integers) and we get that the degrees of freedom for that field theory are \(o(\mathcal{M})=\beth_2\).
2. D-Torus Case: \(\mathbb{E}^D=S^D\). The momentum states \(|\vec{N}\rangle\) in the compact space \(S^D\) are labelled by integers \(\vec{N}=(n_1,...,n_D)\in \mathbb{N}^D\) and, when expressed in position representation, are just the Fourier modes \(\langle\vec{\theta}|\vec{N}\rangle=e^{i2\pi \vec{\theta}\cdot\vec{N}/L}/L^{D/2}\). The general solution for the compact space \(S^D\) Cauchy problem can be expressed again as \(\phi(t,\vec{R})=\langle\vec{R}|e^{i\mathcal{H}t}|\phi\rangle\), but now the initial state is given by the Fourier sum \(|\phi\rangle=\Sigma_{\vec{N}\in \mathbb{N}^D} a(\vec{P})|\vec{N}\rangle \), which in position representation yields
\(\phi_{0}(\vec{\theta})=\langle\vec{\theta}|\phi\rangle=\frac{1}{L^{D/2}}\sum_{\vec{N}\in\mathbb{N}^D}a(n_1,...,n_D)e^{i2\pi\vec{N}\cdot \vec{\theta}/L}.\)
It follows that the space of all initial states is
\[\mathcal{N}=\{a(n_1,...,n_D)\in \mathbb{F}; \vec{N}=(n_1,...,n_D)\in \mathbb{N} \} = \mathbb{F}^{\mathbb{N}^D}.\]
Its cardinality is determined by
\[o(\mathcal{N})=\mathfrak{c}^{F\times \aleph_{0}^D},\]
Using cardinal arithmetic again,
\[\mathfrak{c}^{F\times \aleph_{0}^D}=(2^{\aleph_0})^{\aleph_0}=2^{\aleph_0\times \aleph_0}=2^{\aleph_0}=\mathfrak{c}.\]
Follows than that the functional degrees of freedom in compact space is just the continuum, \(o(\mathcal{N})=\mathfrak{c}\).
Comparing the above examples, we see that the the Cauchy problem for the F-component Schrodinger equation, when formulated in a space of large dimensions, particularly Euclidean space, is such as to have its functional degrees of freedom equal to \(o(\mathcal{M})=\beth_2\) (where \(\mathcal{M}\) is the space of all available initial states), while in the compact case, particularly the D-torus, the functional degrees of freedom are \(o(\mathcal{N})=\mathfrak{c}\) (with \(\mathcal{N}\) as the initial configuration space). Follows from cardinal theory that \(o(\mathcal{M})=\beth_2>\mathfrak{c}=o(\mathcal{N})\), which means that the case of large spatial dimensions posses much more functional freedom than the case of compact ones, far greater than one have real numbers exceeding the number of integers. This is at the root of the fact that quantization eliminates the excess of degrees of freedom that exists in classical field theory, which makes it impossible for a classical theory to explain the thermodynamic properties of matter from statistical considerations. I think it should be clear that this have an equally important (if not even more) relevance in quantum field theories.
The considerations elucidated above are all lost in the application made by Penrose of the notation introduced by Wheeler because, when Penrose consider the discretization of the Cauchy data and then takes the limit \(N \longrightarrow \infty\), he obtains that the degrees of freedom are of the form \(\infty^{P_{(\sigma_1,...,\sigma_N)}(\infty)}\) for both the large dimensions as to the compact dimensions Cauchy problems. Observe that by the symbolic replacement \(\aleph_0 \mapsto \infty\) and \(\mathfrak{c} \mapsto \infty\) in the above formulae for \(o(\mathcal{M})\) and \(o(\mathcal{N})\), we recover the answer \(o(\mathcal{M})=o(\mathcal{N})=\infty^{F\times \infty^D}\) which one would expect from Penrose analysis, but we lose any sign from the differences arising from large dimensions versus compact ones.
Q&A (4870)
