An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model

Introduction

The TAP equations for the Sherrington-Kirkpatrick model describe the quenched expectations of the spin variables in a large system.

We write $\left\langle\cdot\right\rangle_{N,\beta,h,\omega}$ for the expectation under this measure. We will often drop the indices $N,\beta,h,\omega$ if there is no danger of confusion. We set

which have to be understood in a limiting sense, as $N\rightarrow\infty.$ $q=q\left(\beta,h\right)$ is the solution of the equation

where $\phi\left(dz\right)$ is the standard normal distribution. It is known that this equation has a unique solution $q>0$ for $h>0$ (see Proposition 1.3.8). If $h=0,$ then $q=0$ is the unique solution if $\beta\leq 1,$ and there are two other (symmetric) solutions when $\beta>1,$ which are supposed to be the relevant ones. Mathematically, the validity of the TAP equations has only been proved in the high temperature case, i.e. when $\beta$ is small, although in the physics literature, it is claimed that they are valid also at low temperature, but there they have many solutions, and the Gibbs expectation has to be taken inside “pure states”. For the best mathematical results, see Chap. 1.7.

The appearance of the so-called Onsager term $\beta^{2}\left(1-q\right)m_{i}$ is easy to understand. From standard mean-field theory, one would expect an equation

but one has to take into account the stochastic dependence between the random variables $m_{j}$ and $g_{ij}.$ In fact, it turns out that the above equation should be correct when one replaces $m_{j}$ by $m_{j}^{\left(i\right)}$ where the latter is computed under a Gibbs average dropping the interactions with the spin $i.$ Therefore $m_{j}^{\left(i\right)}$ is independent of the $g_{ik},\ 1\leq k\leq N,$ and one would get

The Onsager term is an Itô-type correction expanding the dependency of $m_{j}$ on $g_{ji}=g_{ij},$ and replacing $m_{j}^{\left(i\right)}$ on the right hand side by $m_{j}.$ The correction term is non-vanishing because of

i.e. exactly for the same reason as in the Itô-correction in stochastic calculus. We omit the details which are explained in .

In the present paper, there are no results about SK itself. We introduce an iterative approximation scheme for solutions of the TAP equations which is shown to converge below and at the de Almayda-Thouless line, i.e. under condition (2.1) below (see ). This line is supposed to separate the high-temperature region from the low-temperature one, but although the full Parisi formula for the free energy of the SK-model has been proved by Talagrand , there is no proof yet that the AT line is the correct phase separation line.

The iterative scheme we propose reveals, we believe, an interesting structure of the dependence of the $m_{i}$ on the family $\left\{g_{ij}\right\},$ even below the AT line. The main technical result, Proposition 2.5 is proved at all temperatures, but beyond the AT-line, it does not give much information.

We finish the section by introducing some notations.

As mentioned above, we suppress $N$ in notations as far as possible, but this parameter is present everywhere.

$\mathbf{g}=\left(g_{ij}\right)$ is a Gaussian $N\times N$-matrix where the $g_{ij}$ for $i<j$ are independent centered Gaussians with variance $1/N$, and where $g_{ij}=g_{ji},\ g_{ii}=0.$ We will exclusively reserve the notation $\mathbf{g}$ for such a Gaussian matrix.

We will use $Z,Z^{\prime},Z_{1},Z_{2},\ldots$ as generic standard Gaussians. Whenever several of them appear in the same formula, they are assumed to be independent, without special mentioning. We then write $E$ when taking expectations with respect to them. (This notation is simply an outflow of the abhorrence probabilists have of using integral signs, as John Westwater once put it).

provided there exists a constant $C>0$ such that

Clearly, if $X_{N}\simeq Y_{N},$ and $X_{N}^{\prime}\simeq Y_{N}^{\prime}$, then $X_{N}+X_{N}^{\prime}\simeq Y_{N}+Y_{N}^{\prime}.$

We will use $C>0$ as a generic positive constant, not necessarily the same at different occurrences. It may depend on $\beta,h,$ and on the level $k$ of the approximation scheme appearing in the next section, but on nothing else, unless stated otherwise.

In order to avoid endless repetitions of the parameters $h$ and $\beta,$ we use the abbreviation

We always assume $h\neq 0,$ and as there is a symmetry between the signs, we assume $h>0.$ $q=q\left(\beta,h\right)$ will exclusively be used for the unique solution of (1.2). In the case $h=0,\ \beta>1,$ there is a unique solution of (1.2) which is positive. Proposition 2.5 is valid in this case, too, but this does not lead to a useful result. So, we stick to the $h>0$ case.

Gaussian random variables are always assumed to be centered.

The recursive scheme for the solutions of the TAP equations

$\mathbf{1}$ here the vector with coordinates all $1,$ and $q=q\left(\beta,h\right)$ is the unique solution of (1.2). We define

$k$ will exclusively been used to number this level of the iteration. Our main result is

Assume $h>0.$ If $\beta>0$ is below the AT-line, i.e. if

If there is strict inequality in (2.1), then there exist $0<\lambda\left(\beta,h\right)<1,$ and $C>0,$ such that for all $k$

The theorem is a straightforward consequence of a computation of the inner products $\left\langle\mathbf{m}^{\left(i\right)},\mathbf{m}^{\left(j\right)}\right\rangle.$ We explain that first. The actual computation of these inner products will be quite involved and will depend on clarifying the structural dependence of $\mathbf{m}^{\left(k\right)}$ on $\mathbf{g.}$

where $Z,Z^{\prime},Z^{\prime\prime},$ as usual, are independent standard Gaussians. Remember that $\operatorname*{Th}\left(x\right)=\tanh\left(h+\beta x\right).$

$\psi$ satisfies $0<\psi\left(0\right)=\alpha^{2}<\psi\left(q\right)=q,$ and is strictly increasing and convex on $\left[0,q\right].$

$\psi\left(0\right)=\alpha^{2},$ and $\psi\left(q\right)=q$ are evident by the definition of $\alpha,q.$ We compute the first two derivatives of $\psi:$

the second equality by Gaussian partial integration.

In both expressions, we can first integrate out $Z^{\prime},Z^{\prime\prime},$ getting

and the similar expression for $\psi^{\prime\prime}$ with $\operatorname*{Th}\nolimits^{\prime}$ replaced by $\operatorname*{Th}\nolimits^{\prime\prime}.$ So, we see that $\psi$ is increasing and convex. Furthermore, as

If (2.1) is satisfied, then $q$ is the only fixed point of $\psi$ in the interval $\left[0,q\right].$ If (2.1) is not satisfied then there is a unique fixed point of $\psi\left(t\right)=t$ inside the interval $\left(0,q\right).$

If there is strict inequality in (2.1) , then $\Gamma_{k}^{2}$ and $\rho_{k}$ converge to $q$ exponentially fast.

We prove by induction on $k$ that $\rho_{k}>\Gamma_{k-1}^{2}.$ For $k=1,$ as $\rho_{1}=\gamma_{1}\sqrt{q},$ the statement follows.

i.e. $\rho_{k}>\Gamma_{k}^{2}.$ As $\rho_{k+1}>\rho_{k},$ the statement follows.

c) Linearization of $\psi$ around $q$ easily shows that the convergence is exponentially fast if $\psi^{\prime}\left(q\right)<1.$ ∎

Remark that by a) of the above lemma, one has $\gamma_{k}>0$ for all $k.$

As the variables $\mathbf{m}^{\left(k\right)}$ are bounded, (2.5) implies

Taking the $N\rightarrow\infty$ limit, using Proposition 2.5, this converges to $2q-2\rho_{k-1}.$ From Lemma 2.4, the claim follows. ∎

Proposition 2.5 is true for all temperatures. However, beyond the AT-line, it does not give much information on the behavior of the $\mathbf{m}^{\left(k\right)}$ for large $k.$ It would be very interesting to know if these iterates satisfy some structural properties beyond the AT-line.

The main task is to prove the Proposition 2.5. It follows by an involved induction argument. We first remark that (2.7) is a consequence of (2.5) and (2.6).

$\operatorname{COND}\left(J\right)$ implies that for all $k\leq J,$ we have with

Evidently, all variables $\mathbf{\phi}^{\left(k\right)}$ are bounded by a constant on $A_{J},$ if $k\leq J.$ The constant may depend on $J,$ of course. The $\mathbf{m}^{\left(k\right)}$ are bounded by $1$ everywhere.

Iterative modifications of the interaction variables

Let $\mathcal{G}$ be a sub-$\sigma$-field of $\mathcal{F},$ and $\mathbf{y}=\left(y_{ij}\right)_{1\leq i,j\leq N}$ be a random matrix. We are only interested in the case where $\mathbf{y}$ is symmetric and on the diagonal, but this is not important for the moment. We assume that $\mathbf{y}$ is jointly Gaussian, conditioned on $\mathcal{G},$ i.e. there is a positive semidefinite $N^{2}\times N^{2}$- $\mathcal{G}$-m.b. matrix $\Gamma$ such that

(We do not assume that $\mathbf{y}$ is Gaussian, unconditionally). Consider a $\mathcal{G}$-measurable random vector $\mathbf{x}$, and the linear space of random variables

We consider the linear projection $\pi_{\mathcal{L}}\left(\mathbf{y}\right)$ of $\mathbf{y}$ onto $\mathcal{L},$ which is defined to be the unique matrix with components $\pi_{\mathcal{L}}\left(y_{ij}\right)$ in $\mathcal{L}$ which satisfy

As $\mathbf{y}$ is assumed to be conditionally Gaussian, given $\mathcal{G},$ it follows that $\mathbf{y}-\pi_{\mathcal{L}}\left(\mathbf{y}\right)$ is conditionally independent of the variables in $\mathcal{L},$ given $\mathcal{G}.$

If $\mathbf{y}$ is symmetric, then clearly $\pi_{\mathcal{L}}\left(\mathbf{y}\right)$ is symmetric, too.

If $X$ is a $\mathcal{G}$-measurable random variable then $\mathbf{y}X$ is conditionally Gaussian as well and

$\mathbf{g}^{\left(k\right)}$ is conditionally Gaussian, given $\mathcal{F}_{k-1}.$

$\mathbf{m}^{\left(k\right)},\ \mathbf{M}^{\left(k\right)},$ and $\mathbf{\phi}^{\left(k\right)}$ are $\mathcal{F}_{k-1}$-measurable

i.e. we perform the above construction with $\mathcal{G}=\mathcal{F}_{k-1}$ and $\mathbf{x}=\mathbf{M}^{\left(k\right)}$.

In order that the construction is well defined, we have to inductively prove the properties (C1) and (C2). We actually prove a condition which is stronger than (C1):

Conditionally on $\mathcal{F}_{k-2},$ $\mathbf{g}^{\left(k\right)}$ is Gaussian, and conditionally independent of $\mathcal{F}_{k-1}.$

(C1’) implies that $\mathbf{g}^{\left(k\right)}$ is conditionally Gaussian, given $\mathcal{F}_{k-1},$ and the conditional law, given $\mathcal{F}_{k-1},$ is the same as given $\mathcal{F}_{k-2}.$

The case $k=1$ is trivial. We first prove (C2) for $k\geq 2,$ using (C1’), (C2) up to $k-1.$ We claim that

where $\mathbf{R}^{\left(k-2\right)}$ stands for a generic $\mathcal{F}_{k-2}$-measurable random variable, not necessarily the same at different occurrences.

As $\mathbf{g}^{\left(k-1\right)}\mathbf{M}^{\left(k-1\right)}=\left\|\mathbf{M}^{\left(k-1\right)}\right\|\mathbf{\xi}^{\left(k-1\right)},$ and $\mathbf{M}^{\left(k-1\right)}$ is $\mathcal{F}_{k-2}$-measurable, by the induction hypothesis, it follows from (3.2) that $\mathbf{m}^{\left(k\right)}$ is $\mathcal{F}_{k-1}$-measurable The statements for $\mathbf{M}^{\left(k\right)},\mathbf{\phi}^{\left(k\right)}$ are then trivial consequences.

We therefore have to prove (3.2). We prove by induction on $j$ that

The case $j=1$ follows from the definition of $\mathbf{m}^{\left(k\right)},$ and the case $j=k-1$ is (3.2).

Assume that (3.3) is true for $j<k-1.$ We replace $\mathbf{g}^{\left(j\right)}$ by $\mathbf{g}^{\left(j+1\right)}$ through the recursive definition

as $\pi_{\mathcal{L}_{j}}\left(\mathbf{g}^{\left(j\right)}\right)$ is $\mathcal{F}_{j}$-measurable and therefore $\pi_{\mathcal{L}_{j}}\left(\mathbf{g}^{\left(j\right)}\right)\mathbf{M}^{\left(k-1,j-1\right)}$ is $\mathcal{F}_{k-2}$-measurable

Using (3.1), one gets $\mathbf{g}^{\left(j+1\right)}\mathbf{M}^{\left(j\right)}=0,$ and therefore

This proves (3.2), and therefore (C2) for $k.$

We condition on $\mathcal{F}_{k-2}.$ By (C2), $\mathbf{M}^{\left(k-1\right)}$ is $\mathcal{F}_{k-2}$-measurable As $\mathbf{g}^{\left(k-1\right)},$ conditioned on $\mathcal{F}_{k-3}$, is Gaussian, and independent of $\mathcal{F}_{k-2},$ it has the same distribution also conditioned on $\mathcal{F}_{k-2}.$ By the construction of $\mathbf{g}^{\left(k\right)},$ this variable is, conditioned on $\mathcal{F}_{k-2},$ independent of $\mathcal{F}_{k-1},$ and conditionally Gaussian. ∎

The proof is by induction on $k.$ For $k=1,$ there is nothing to prove.

Assume that the statement is proved up to $k.$ We want to prove $\mathbf{g}^{\left(k+1\right)}\mathbf{\phi}^{\left(m\right)}=0$ for $m\leq k.$ The case $m=k$ is covered by (3.1). For $m<k,$ it follows by Remark 3.1, as $\mathbf{\phi}^{\left(m\right)}$ is $\mathcal{F}_{k-1}$-measurable, that

as $\mathbf{g}^{\left(k\right)}\mathbf{\phi}^{\left(m\right)}=0$ by the symmetry of $\mathbf{g}^{\left(k\right)}$ and the induction hypothesis. ∎

We write $O_{k}\left(N^{-r}\right)$ for a generic $\mathcal{F}_{k}$-measurable random variable $X$ which satisfies

for some $K>0.$ The constants $C,K>0$ here may depend on $h,\beta,$ and the level $k,$ and on the formula where they appear, but on nothing else, in particular not on $N,$ and any further indices. For instance, if we write

we mean that there exists $C\left(\beta,h,k\right),\ K\left(\beta,h,k\right)>0$ with

Furthermore, in such a case, it is tacitly assumed that $X_{ij}-Y_{ij}$ are $\mathcal{F}_{k}$-measurable

Evidently, if $X,Y$ are $O_{k}\left(N^{-r}\right),$ then $X+Y$ is $O_{k}\left(N^{-r}\right),$ and if $X$ is $O_{k}\left(N^{-r}\right),$ and $Y$ is $O_{k}\left(N^{-s}\right),$ then $XY$ is $O_{k}\left(N^{-r-s}\right).$

We will finally prove the validity of the following relations:

The $\lambda_{m,A}^{\left(k\right)}$ are real numbers, not random variables, which depend on $A$ only through the type of subset which is taken. For instance, there is only one number (for every $m,k$) if all four indices are taken.

The main point with assuming $\operatorname*{COND}\left(J\right)$ is (2.9). On $A_{J},$ the variables $\mathbf{\phi}^{\left(k\right)}$ are bounded for $k\leq J.$

Assume (4.1) - (4.3) for $k=J,$ and (2.9). Then

a) As $\mathbf{\phi}^{\left(J\right)}$ is $\mathcal{F}_{J-1}$-measurable, and $\mathbf{g}^{\left(J\right)}$ is independent of $\mathcal{F}_{J-1},$ conditionally on $\mathcal{F}_{J-2},$ we get

Using (4.1), (4.2), and the boundedness of the $\phi$’s on $A_{J}$, and $N^{-1}\sum_{i}\phi_{i}^{\left(J\right)2}=1,\ \sum_{i}\phi_{i}^{\left(J\right)}\phi_{i}^{\left(m\right)}=0$ for $m<J$, we get

We split the sum over $\left(s,t\right)$ into the one summand $s=j,t=i$, in $A=\left\{\left(s,s\right):s\neq i,j\right\},\ B=\left\{\left(j,t\right):t\neq i,j\right\},\ C=\left\{\left(s,i\right):s\neq i,j\right\},$ and $D=\left\{\left(s,t\right):\left\{s,t\right\}\cap\left\{i,j\right\}=\emptyset\right\}.$ The one summand $s=j,t=i$ gives $\phi_{i}^{\left(J\right)}\phi_{j}^{\left(J\right)}/N+O_{J-1}\left(N^{-2}\right).$

Because $\left\langle\mathbf{\phi}^{\left(J\right)},\mathbf{\phi}^{\left(m\right)}\right\rangle=0$ for $m<J,$ this is seen to be $O_{J-1}\left(N^{-2}\right).$ The same applies to $\sum_{C}.$

Take e.g. $A=\left\{i,j,s\right\}.$ Then $\lambda_{m,A}^{\left(J\right)}=\lambda_{m,3}^{\left(J\right)}$ with no further dependence of this number on $i,j,s.$ So we get for this part for any summand on $m$ with $m<J$

Using again $\left\langle\mathbf{\phi}^{\left(J\right)},\mathbf{\phi}^{\left(m\right)}\right\rangle=0,$ we get that this is $O_{J-1}\left(N^{-2}\right).$ This applies in the same way to all the parts. Therefore b) follows.

due to the orthogonality of the $\mathbf{\phi}^{\left(m\right)}.$

where the $\mathcal{F}_{J-1}$-measurable coefficients $x_{ij,s}^{\left(J\right)}$ satisfy

The existence of $\mathcal{F}_{J-1}$-measurable coefficients $x_{ij,s}^{\left(J\right)}$ comes from linear algebra.

Therefore, we can replace the $x_{ij,\cdot}^{\left(J\right)}$ by

which satisfy the desired property (4.9).

We keep $i,j$ fixed for the moment and write $x_{s}$ for $x_{ij,s}^{\left(J\right)}.$ The requirement for them is that for all $t$

Due to the orthonormality of the $\mathbf{\phi},$ one gets

Writing $r_{ij}$ for the $O_{J-1}\left(N^{-2}\right)$ error term in (4.5), and for $j=i,$ the $O_{J-1}\left(N^{-1}\right)$ error term in (4.4), we arrive at

In the first summand, we sum now over all $s,$ remarking that we have assumed that $\sum_{s}x_{s}\phi_{s}^{\left(m\right)}=0$ for $m<J.$ The error for not summing over the single $t$ can be incorporated into $r_{tt}.$ We therefore arrive at

Write $\Phi$ for the matrix $\left(N^{-1}\phi_{i}^{\left(J\right)}\phi_{j}^{\left(J\right)}\right)$ and $R$ for $\left(r_{ij}\right).$ Then we have to invert the matrix $\left(I+\Phi+R\right).$ Remark that $\left(I+\Phi\right)^{-1}=I-\Phi/2.$ Therefore

The right hand side, we can develop as a Neumann series:

As $\left(\Phi\mathbf{y}\right)_{i}=O_{J-1}\left(N^{-3}\right),$ we get the desired conclusion. ∎

The summands involving the $x^{\left(J\right)}$ all only give contributions which enter the $O_{J-1}$-terms. Take for instance $s=j,\ t\neq i,j.$ In that case, the claimed $O_{J-1}$-term is $O_{J-1}\left(N^{-3}\right).$ In the last summand of (4.10), there is one summand, namely $u=v=j,$ where the $x^{\left(J\right)}$ are $O_{J-1}\left(N^{-2}\right),$ so this summand is only $O_{J-1}\left(N^{-4}\right).$

The other summands behave similarly. The third and fourth summand in (4.10) behave similarly.

As another case, take $\left\{i,j\right\}\cap\left\{s,t\right\}=\emptyset,$ where we have to get $O_{J-1}\left(N^{-4}\right)$ for the second to fourth summand in (4.10).

We write $m\times n$ for the summand, we get by multiplying the $m$-th summand in the first bracket with the $n$-th in the second. By induction hypothesis, we get

In the $1\times 2$-term, only the multiplication of $g_{ij}^{\left(J\right)}$ with $\xi_{j}^{\left(J\right)}$ counts, the other part giving $O_{J-2}\left(N^{-3}\right).$ Therefore

$2\times 1$ gives the same. In $2\times 2,$ again only the matching of $\xi_{j}^{\left(J\right)}$ with $\xi_{j}^{\left(J\right)}$ counts, so we get

The other parts are easily seen to give $O_{J-1}\left(N^{-3}\right).$ We we have proved that

c) We have here $\left\{i,j\right\}\cap\left\{s,t\right\}=\emptyset.$

The $1\times 1,1\times 2,2\times 1,$ and $2\times 2$-terms are clearly of the desired form, either from induction hypothesis or Lemma 4.2.

For $u=i$ we get for the expectation $\phi_{i}^{\left(J\right)}\phi_{j}^{\left(J\right)}/N+O_{J-1}\left(N^{-2}\right),$ so this is of the desired form. The same applies to $u=j.$ It therefore remains

As $\sum_{u}\phi_{u}^{\left(J\right)}\phi_{u}^{\left(m\right)}=0,$ the whole expression is $O_{J-1}\left(N^{-4}\right).$ The other cases are handled similarly. ∎

Proof of Proposition 2.5

We assume $\operatorname{COND}\left(J\right),$ and (4.1) - (4.3) for $k\leq J.$ By Proposition 4.1 of the last section, this implies (4.1) - (4.3) for $k\leq J+1.$ Using this, we prove now (2.5) and (2.6) for $k=J+1,$ so that we have proved $\operatorname{COND}\left(J+1\right).$ Having achieved this, the proof of Proposition 2.5 is complete.

Remark that under $\operatorname{COND}\left(J\right)$

From $q>\Gamma_{k-1}^{2},$ by (2.5) and (2.6) for $k\leq J,$ and the fact that the $\phi_{j}^{\left(k\right)}$ are uniformly bounded on $A_{J},$ we have

Remark that by Lemma 3.2, we have $\mathbf{g}^{\left(s\right)}\mathbf{M}^{\left(k-1,s-1\right)}=\mathbf{g}^{\left(s\right)}\mathbf{m}^{\left(k\right)}.$ Evidently

This proposition is correct for all $\beta.$ The key point with (2.1) is that the first summand $\left\|\mathbf{M}^{\left(k-1\right)}\right\|\mathbf{\xi}^{\left(k-1\right)}$ disappears for $k\rightarrow\infty$ as $\left\|\mathbf{M}^{\left(k-1\right)}\right\|\simeq\sqrt{q-\Gamma_{k-2}^{2}},$ so that for large $k,$ $\mathbf{\hat{m}}^{\left(k\right)}$ stabilizes to $\operatorname*{Th}\left(\sum\nolimits_{t=1}^{k-2}\gamma_{t}\mathbf{\xi}^{\left(t\right)}\right),$ but above the AT-line $q-\Gamma_{k-2}^{2}$ does not converge to $0.$ Therefore, above the AT-line, in every iteration, new conditionally independent contributions appear.

The above proposition is proved by showing that $\operatorname{COND}\left(J\right)$ implies

As $\operatorname{COND}\left(J\right)$ implies trivially $\operatorname{COND}\left(J^{\prime}\right)$ for $J^{\prime}<J,$ it is then clear that $\operatorname{COND}\left(J\right)$ implies $\mathbf{m}^{\left(k\right)}\approx\mathbf{\hat{m}}^{\left(k\right)}$ for all $k\leq J+1.$ As the $m_{j}^{\left(k\right)}$ are uniformly bounded by $1,$ we get from that

for all $j\leq J+1.$ We will then prove (Lemma 5.3) that

This will prove $\operatorname{COND}\left(J+1\right),$ and therefore, this will have finished the whole induction procedure.

Together with proving (5.3), we also show

for $k=J+1$ which is not evident from (5.3) as the $\xi_{i}^{\left(m\right)}$ are not bounded.

Assume the validity of (2.5)-(2.7) and (5.4) for $k\leq J.$ Then for $s=1,\ldots,J-1$

We prove by induction on $s,\ 1\leq s\leq J-1,$ that

and define $\mathbf{\mu}^{\left(n\right)}$ where $\mathbf{y}$ is replaced by $\mathbf{y}^{\left(n\right)},$ $n=1,\ldots,5.$ Remark that

which prove the desired induction in $s.$

To switch from $\mathbf{\mu}^{\left(0\right)}$ to $\mathbf{\mu}^{\left(1\right)},$ we observe that by the estimates of Lemma 4.3, one has

By choosing $K$ large enough, we get for $1/\sqrt{N}\leq t\leq 1$ by Corollary A.2 a)

For $t\leq 1/\sqrt{N},$ the bound is trivial anyway. This proves (5.7) for $n=1.$ (5.8) follows in the same way using Corollary A.2 b).

on $A_{J}.$ (5.7) for $n=2$ then follows from Corollary A.2 c). As for (5.8), we remark that

We can then again use Corollary A.2 c) remarking that $\exp\left[-Nt/C\right]\leq\exp\left[-Nt^{2}/C\right]$ for $t\leq 1.$

(5.7) for $n=3$ follows from the induction hypothesis (2.7), and Corollary A.2 a). Similarly with (5.8) but here, one has to use part b) of Corollary A.2.

on $A_{k},$ and one uses the induction hypothesis (5.4) for $J$ to get (5.7) for $n=4.$ Remark that actually, one has a bound uniform in $i:$

Therefore, one also gets (5.8) using Corollary A.2. Up to now, we have obtained

and we can therefore replace $\left\langle\mathbf{\xi}^{\left(s\right)},\mathbf{\hat{m}}^{\left(J\right)}\right\rangle\mathbf{\phi}^{\left(s\right)}$ on the right hand side, by $\beta\left(1-q\right)\gamma_{s}\mathbf{\phi}^{\left(s\right)}$ for $s<J-1,$ or $\beta\left(1-q\right)\sqrt{q-\Gamma_{J-2}^{2}}\mathbf{\phi}^{\left(J-1\right)}$ for $s=J-1,$ which is the same as replacing $\mathbf{X}^{\left(J-1,s-1\right)}$ by $\mathbf{X}^{\left(J-1,s\right)}.$ Therefore, the lemma is proved. ∎

We assume $\operatorname{COND}\left(J\right)$.

We condition on $\mathcal{F}_{J-2}.$ Then $\mathbf{\xi}^{\left(J-1\right)}$ is conditionally Gaussian with covariances given in Lemma 4.2 a), b). We can therefore apply Lemma A.3 which gives, conditionally on $\mathcal{F}_{J-2},$ on an event $B_{J-2}\in\mathcal{F}_{J-2}$ which has probability $\geq 1-C\exp\left[-N/C\right],$

Applying now Lemma A.3 successively to $\mathbf{\xi}^{\left(J-2\right)},\mathbf{\xi}^{\left(J-2\right)},\ldots$ , we get

The case $s<J-1$ uses a minor modification of the argument. One first uses Lemma A.3 successively to get

b) This also comes with a modification of the reasoning in a).

In the case $j=J+1,$ the outcome is similar, one only has to replace the second factor by $\operatorname*{Th}\left(\left\|\mathbf{M}^{\left(J\right)}\right\|Z_{J}+\sum\nolimits_{t=1}^{J-1}\gamma_{t}\mathbf{\xi}_{i}^{\left(t\right)}\right).$

The next observation is that by the induction hypothesis, one can replace $\left\|\mathbf{M}^{\left(J\right)}\right\|$ by $\sqrt{q-\Gamma_{J-1}^{2}}$ and we get

The important point is that the factor before $Z_{J}$ is replaced by a constant, which is due to the induction hypothesis. We can now proceed in the same way with $\mathbf{\xi}^{\left(J-1\right)},$ applying again Lemma A.3, conditioned on $\mathcal{F}_{J-2},$ and the induction hypothesis. The final outcome is

For the latter case, the right hand side is simply $q.$ For the case $j\leq J,$ we can rewrite the expression on the right hand side as

Solving, we get $a^{2}+b^{2}=q-\Gamma_{j-2}^{2},$ and

Appendix A Appendix

then $\sup_{i,j\leq N}\left|\alpha_{ij}\right|\leq C/N.$ Therefore, we can represent the $\zeta_{i}$ as

where the $Z_{i}$ are i.i.d. standard Gaussians. Then

By choosing $K$ appropriate, we get the desired estimate.

To prove (A.2), we use the same representation. As

for large enough $K,$ we get the desired conclusion. ∎

Assume $\operatorname{COND}\left(J\right)$ and $k\leq J.$

For any $m\leq k$ there exist $C,K>0$ such that

For any $m,l,$ there exist $C,K>0$ such that

If $Y_{i}$ are $\mathcal{F}_{m-1}$-measurable with

so that we see that it suffices to consider $l=m.$ Then we apply the lemma, part b).

As for c), we have that the conditional distribution of $\sqrt{N}\left\langle\mathbf{\xi}^{\left(m\right)},\mathbf{Y}\right\rangle$, given $\mathcal{F}_{m-1},$ is Gaussian, with bounded variance. So the statement follows. ∎

and there are vectors $\left\{x_{i}^{\left(N\right)}\right\}_{i\leq N},~{}\left\{y_{i}^{\left(r,N\right)}\right\}_{i\leq N,\ r\leq m},\ m$ fixed, which are bounded in all indices, such that

We leave out $N$ in notations, as often as possible. Consider

The constant $K>0$ will be specified below. Then

where $L$ is a bound on the Lipshitz constants for the $F_{N,i},$ and $c$ is a bound of the $\left|y_{i}^{\left(r\right)}\right|.$

We choose $K$ large enough such that the $N\times N$-matrix $\Gamma$ which is $\left(r_{ij}\right)$ off diagonal, and

on the diagonal is positive definite. This is possible as $\left|r_{ij}\right|\leq CN^{-2}.$

Let $\left\{U_{i}\right\}$ be a Gaussian matrix with covariance matrix $\Gamma.$ Then

has the same distribution as $\left\{\eta_{i}^{\prime}\right\}.$ Here we assume that $\left\{U_{i}\right\}$ is independent of the $Z$’s. So, we assume that the $\eta_{i}^{\prime}$ are presented in this way.

We can apply Lemma A.1 to the vector $\left(\sqrt{N}U_{i}\right)_{1\leq i\leq N}$, and (A.3) to the first summand on the right-hand side, obtaining

follows by standard Gaussian isoperimetry (see e.g. ). ∎

References