Making the long code shorter, with applications to the Unique Games Conjecture
Boaz Barak, Parikshit Gopalan, Johan Hastad, Raghu Meka, Prasad Raghavendra, David Steurer
Introduction
Khot’s Unique Games Conjecture [Kho02] (UGC) has been the focus of intense research effort in the last few years. The conjecture posits the hardness of approximation for a certain constraint satisfaction problem, and shows promise to settle many open questions in theory of approximation algorithms. Specifically, an instance of the Unique Games problem with variables and alphabet is described by a collection of constraints of the form where is a permutation over . An assignment to is a mapping from to , and ’s value is the fraction of constraints such that . The Unique Games Conjecture is that for any , there is some finite such that it is NP hard to distinguish between the case that a Unique Games instance with alphabet has an assignment satisfying fraction of the constraints, and the case that every assignment satisfies at most fraction of ’s constraint.
Many works have been devoted to studying the plausibility of the UGC, as well as exploring its implications and obtaining unconditional results motivated by this effort. Tantalizingly, at the moment we have very little evidence for the truth of this conjecture. One obvious reason to believe the UGC is that no algorithm is known to contradict it, though that of course may have more to do with our proof techniques for algorithm analysis than actual computational difficulty. Thus perhaps the strongest evidence for the conjecture comes from results showing particular instances on which certain natural algorithms will fail to solve the problem. However, even those integrality gaps are quantitatively rather weak. For example, while Arora, Barak and Steurer [ABS10] showed a subexponential upper bound on an algorithm for the Unique Games and the related Small-Set Expansion problem, the hardest known instances for their algorithm only required quasipolynomial time [Kol10]. Similarly (and related to this), known integrality gaps for Unique Games and related problems do not rule out their solution by an -round semidefinite hierarchy, an algorithm that can be implemented in quasipolynomial (or perhaps even polynomial [BRS11]) time.
The long code has been a central tool in many of these works. This is the set of “dictator” functions mapping to that have the form for some . Many hardness reductions (especially from Unique Games) and constructions of integrality gap instances use the long code as a tool. However, this is also the source of their inefficiency, as the long code is indeed quite long. Specifically, it has only codewords but dimension , which leads to exponential blowup in many of these applications.In this work, we introduce a different code, which we call the “short code”, that is exponentially more efficient, and can be used in the long code’s place in many of these applications, leading to significant quantitative improvements. In particular, we use our code to show instances on which the [ABS10] algorithm, as well as certain semidefinite hierarchies, require almost sub-exponential time, thus considerably strengthening the known evidence in support of the Unique Games Conjecture. Moreover, our results open up possibilities for qualitative improvements as well, in particular suggesting a new approach to prove the Unique Games Conjecture via an efficient alphabet reduction.
At the heart of the long code’s applications lie its connection with the noisy Hypercube. This is the weighted graph whose vertices are elements in where a random neighbor of is obtained by flipping each bit of independently with probability .This graph is closely related and has similar properties to the unweighted graph where we connect and if their Hamming distance is at most . It is not too hard to show that the codewords of the long code correspond to the top eigenvectors of the noisy hypercube which also give the minimal bisections of the graph, cutting only an fraction of edges. In addition, several converse results are known, showing that bisections (and more general functions) cutting few edges are close to these top eigenvectors (or dictatorships) in some sense. (One such result is the “Majority is Stablest” Theorem of [MOO05].) The inefficiency of the longcode is manifested in the fact that the number of vertices of the noisy cube is exponential in the number of its top eigenvectors.
Another way to describe the long code is that it encodes by a binary vector of length where for every function . This view also accounts for the name “long code”, since one can see that this is the longest possible encoding of without having repeated coordinates. For every subset of functions mapping to , we define the -short code to be the code that encodes by a vector of length where for every . Note that this is a very general definition that encapsulates any code without repeated coordinates. For , we define the -short code to be the the -short code where is the set of all polynomials over of degree at most . Note that the -short code is the Hadamard code, while the -short code is the long code. We use the name “short code” to denote the short code for . Note that the short code has codewords and dimension roughly , and hence only quasipolynomial blowup, as opposed to the exponential blowup of the long code. Our main contribution is a construction of a “derandomized” noisy cube, which is a small subgraph of the noisy cube that enjoys the same relations to the short code (including a “Majority is Stablest” theorem) as the original noisy cube has to the long code. As a result, in many applications one can use the short code and the derandomized cube in place of the long code and the noisy cube, obtaining an exponential advantage. Using this approach we obtain the following results:
Small set expanders with many large eigenvalues
Our first application, and the motivation to this work, is a question of Arora, Barak and Steurer [ABS10]: How many eigenvectors with eigenvalue at least can an -vertex small set expander graph have? We say a graph is a small set expander (SSE) if all sufficiently small subsets of vertices have, say, at least fracton of their neighbors outside the set. [ABS10] showed an upper bound of on the number of large (i.e., greater than ) eigenvalues of a small set expander. Arora et al. then observed that the subspace enumeration algorithm of [KT07, Kol10] for approximating small set expansion in an input graph takes time at most exponential in this number, which they then use to give an algorithm with similar running time for the Unique Games problem. Up to this work, the best lower bound was , with the example being the noisy cube, and hence as far as we knew the algorithm of [ABS10] could solve the small set expansion problem in quasipolynomial time, which in turn might have had significant implications for the Unique Games problem as well. Our derandomized noisy cube yields an example with an almost polynomial number of large eigenvalues:
For every , there is an -vertex small set expander graph with eigenvectors with corresponding eigenvalues at least .
Theorem 1 actually follows from a more general result connecting locally testable codes to small set expanders, which we instantiate with the Reed Muller code. See Section 2 for details.
Efficient integrality gaps
There is a standard semidefinite program (SDP) relaxation for the Unique Games problem, known as the “basic SDP” [KV05, RS09]. Several works have shown upper and lower bounds on the approximation guarantees of this relaxation, and for constant alphabet size, the relation between the alphabet size and approximation guarantee is completely understood [CMM06]. However, for unbounded alphabet, there was still a big gap in our understanding of the relation between the approximation guarantee and the number of variables. Gupta and Talwar [GT06] showed that if the relaxation’s value is , there is an assignment satisfying fraction of constraints. On the other hand, Khot and Vishnoi [KV05] gave an integrality gap instance where the relaxation’s value was Throughout, for any function , denotes a function satisfying . but the objective value (maximum fraction of constraints satisfied by any assignment) was . It was a natural question whether this could be improved (e.g., see [Lee11]), and indeed our short code allows us to obtain an almost exponential improvement:
There is an -variable instance of Unique Games with objective value but for which the standard semidefinite programming (SDP) relaxation has value at least .For functions we write if . That is, if there are constants such that for all sufficiently large , . (Note that we allow , and so does not imply that .) Similarly, we define and write if .
Integrality gaps for SDP hierarchies
Barak, Raghavendra and Steurer [BRS11] (see also [GS11]) showed that for every , rounds of the hierarchy yields a non-trivial improvement over the basic SDP . The unique games conjecture predicts that this is optimal, in the sense that rounds of any hierarchy should not improve the worst-case approximation ratio above the basic SDP.This is under the widely believed assumption that . However, this prediction is far from being verified, with the best lower bounds given by [RS09] (see also [KS09]) who showed instances that require rounds for the hierarchy, and rounds for the hierarchy. Moreover, these instances are known to be solvable in quasipolynomial time [Kol10] and in fact via rounds of the hierarchy [BRS11] . Thus prior work gave no evidence that the unique games problem cannot be solved in quasipolynomial time. In this work we obtain almost-exponentially more efficient integrality gaps, resisting rounds of the hierarchy and rounds of the hierarchy. The latter is the first superlogarithmic SDP hierarchy lower bound for Unique Games for any SDP hierarchy considered in the literature.
For every there is some , such that for every there is an variable instance of Unique Games with alphabet size such that the objective value of is at most , but the value on of both rounds of the hierarchy and rounds of the hierarchy is at least .
Alphabet reduction gadget
Khot, Kindler, Mossel and O’Donnel [KKMO04] used the long code to show an “alphabet reduction” gadget for unique games. They show how to reduce a unique game instance with some large alphabet to an instance with arbitrarily small alphabet. (In particular, they showed how one can reduce arbitrary unique games instances into binary alphabet instances, which turns out to be equivalent to the Max Cut problem.) However, quantitatively their result was rather inefficient, incurring an exponential in blowup of the instance. By replacing the long code with our “short code”, we obtain a more efficient gadget, incurring only a qusipolynomial blowup. One caveat is that, because the short code doesn’t support arbitrary permutations, this reduction only works for unique games instances whose constraints are affine functions over where ; however this class of unique games seems sufficiently rich for many applications.For example, because the multiplicative group of the field is cyclic, one can represent constraints of the form as linear constraints over (i.e., constraints of the form where is an invertible linear map over ).
Once again, our quantitative results are stronger than stated here, and as in [KKMO04], we obtain nearly optimal relation between the alphabet size and the soundness and completeness thresholds. In particular for our results match the parameters of the Max Cut algorithm of Geomans and Williamson. Our alphabet reduction gadget suggests a new approach to proving the unique games conjecture by using it as an “inner PCP”. For example, one could first show hardness of unique games with very large alphabet (polynomial or even subexponential in the number of variables) and then applying alphabet reduction. At the very least, coming up with plausible hard instances for unique games should be easier with a large alphabet.
The long code is also used as a tool in applications that do not involve the unique games conjecture. On a high level, there are two properties that make the long code useful in hardness of approximation: (i) it has a query test obtained from the noisy hypercube and (ii) it has many symmetries, and in particular one can read off any function of from the codeword. Our short code preserves property (i) but (as is necessary for a more efficient code) does not preserve property (ii), as one can only read off low degree polynomials of (also it is only symmetric under affine transformations). We note that if one does not care about property (i) and is happy with a query test, then it’s often possible to use the Hadamard code which is more efficient than the short code (indeed it’s essentially equal to the -short code for ). Thus, at least in the context of hardness of approximation, it seems that the applications the short code will be most useful are those where property (i) is the crucial one.
Despite the name “short code”, our code is not the shortest possible code. While in our applications, dimension linear in the number of codewords is necessary (e.g., one can’t have a graph with more eigenvalues than vertices), it’s not clear that the dimension needs to be polynomial. It is a very interesting open question to find shorter codes that can still be used in the above applications.
Our techniques
To explain our techniques we focus on our first application— the construction of a small set expander with many eigenvalues close to . The best way to view this construction is as a derandomization of the noisy hypercube, and so it will be useful to recall why the noisy hypercube itself is a small set expander.
Recall that the -noisy hypercube is the graph whose vertex set is where we sample a neighbor of by flipping each bit independently with probability . The eigenvectors in are given by the parity functions for subsets and the corresponding eigenvalues are . Thus only depends on the degree of . In particular, the “dictator” functions have eigenvalue and they correspond to balanced cuts (where vertices are partitioned based on the value of ) with edge expansion . As increases, decreases, becoming a constant around .
Given which is the indicator of a set , its Fourier expansion can be viewed as expressing the vector in the eigenvector basis. The edge expansion of is determined by the distribution of its Fourier mass; sets where most of the Fourier mass is on large sets will expand well. Given this connection, small-set expansion follows from the fact that the indicator functions of small sets have most of their mass concentrated on large Fourier coefficients. More precisely a set of measure has most of its Fourier mass on coefficients of degree . This follows from the so-called (2,4)-hypercontractive inequality for low-degree polynomials— that for every degree polynomial ,
for some depending only on . (See Section 4.1 for the proof, though some intuition can be obtained by noting that if is a characteristic function of a set of measure then and and hence Equation (2.1) shows that cannot be an -degree polynomial.)
By a “derandomized hypercube” we mean a graph on much fewer vertices that still (approximately) preserves the above properties of the noisy hypercube. Specifically we want to find a very small subset of and a subgraph of whose vertex set is such that (i) will have similar eigenvalue profile to , and in particular have eigenvalues close to and (ii) will be a a small set expander. To get the parameters we are looking for, we’ll need to have the size of be at most .
A natural candidate is to take to be a random set, but it is not hard to show that this will not work. A better candidate might be a linear subspace that looks suitably pseudorandom. We show that in fact it suffices to choose a subspace whose dual is a sufficiently good locally testable code. (We identify with via the usual map .)
Our construction requires an asymptotic family of linear codes where the distance tends to infinity. The code should have a -query local tester which when given a received word samples a codeword of weight at most from a distribution on and accepts if . The test clearly accepts codewords in , we also require it to reject words that are distance at least from every codeword in with probability . Given such a locally testable code , we consider the Cayley graphCayley graph are usually defined to be unweighted graph. However, the definition can be generalized straightforwardly to weighted graphs. whose vertices are the codewords of the dual code while the (appropriately weighted) edges correspond to the distribution . That is, a vertex of is a codeword , while a random neighbor of is obtained by picking a random from and moving to .
Because is a subspace, it is easy to show that the eigenvectors of are linear functions of of the form for (where if then and are identical on ’s vertices). Moreover, from the way we designed the graph, for every , the corresponding eigenvalue is equal to . This connection between the spectrum of and the local testability of allows us to invoke machinery from coding theory in our analysis.
From this one can deduce that the eigenvalue spectrum of does indeed resemble the hypercube in the range close to . In particular each is a distinct eigenvector with eigenvalue , and gives a bad cut in (where vertices are partitioned based on the value of ). On the other hand for any eigenvector of , choose of minimal weight such that . Now if this means that the distance of from is at least , which using the testing property implies that .
If we can show that indicator functions of small sets have most of their Fourier mass on such eigenvectors (with small eigenvalue), that will imply that small sets have good expansion. For small subsets of the hypercube, recall that this is proved using (2,4)-hpercontractivity for low-degree polynomials. The key observation is that the inequality
still holds for all polynomials of degree . This is because the distance of is , hence the distribution of a random in is -wise independent, which means that the expectation of any polynomial of degree at most is equal over such and over a uniform in . Thus (2.2) follows from (2.1), completing our proof.
We instantiate this approach with using for the Reed Muller code consisting of polynomials in variables over of degree . This is a code of distance . We note that the degree and hence the rate of the code are very high. The graph is over the codewords of that is itself the Reed Muller code of polynomials over of degree . Our basic tester consists of selecting a random minimum weight codeword of .For many applications we amplify the success of this tester by selecting a sum of random such words, this corresponds to taking some power of the basic graph described. Thus our graph has as its vertices the degree polynomials over with an edge between every polynomials such that is a product of linearly-independent affine functions (as those are the minimal weight codewords in the Reed Muller code). We use the optimal analysis of Bhattacharyya, Kopparty, Schoenebeck, Sudan and Zuckerman [BKS+10] to argue about the local testability of which is a high degree Reed Muller code. We should note that this test is very closely related to the Gowers uniformity test that was first analyzed in the work of Kaufman et al. [AKK+05], but our application requires the stronger result from [BKS+10].
We now briefly outline how we use the above tools to obtain more efficient versions of several other constructions such as alphabet reduction gadgets and integrality gaps for unique games and other problems.
To beign with, the graph we construct can be used to prove Theorem 2. That is, a construction of an variable instance of unique games where every assignment can satisfy at most a very small (say ) fraction of the constraints, but for which the standard semidefinite programming (SDP) relaxation has value of at least . The basic idea is to simply take the graph we constructed above, and turn it into an instance of unique games by considering it to be the label extended graph of some unique games instance. We now elaborate a bit below, leaving the full details to Section 7. Recall that a Unique Games instance with variables and alphabet is described by a collection of constraints of the form where is a permutation over . An assignment to is a mapping from to , and ’s value is the fraction of constraints such that . The label extended graph corresponding to is the graph over vertices where for every constraint of the form and we add an edge between and . It is not hard to see that an assignment of value corresponds to a subset containing exactly of ’s vertices with small expansion (i.e., fraction of the edges from leave the set). Thus if is an expander for sets of measure in then there is no nearly satisfying assignment for the unique games instance . In our case, our graph has the degree polynomials over as its vertices, and we transform it into a unique game instance whose variables correspond to degree polynomials without linear terms. The alphabet consists of all linear functions over . We ensure that the graph is the label extended graph of by setting the permutations accordingly: given a polynomial without a linear term, and a function that is a product of affine functions,Actually, to get better parameters, we take some power of , meaning that we consider that is a sum of functions that are products of affine functions. if we write where is the linear part of , then we add a constraint of the form where is the permutation that maps a linear function into . Some not too difficult calculations show that the top eigenvectors of our graph yield a solution for the semidefinite program for (if the top eigenvectors are , our vector solution will associate with each vertex the vector ). By choosing carefully the parameters of the graph , the instance will have SDP value where is the number of variables.
Derandomized Invariance Principle
While hypercontractivity of low degree polynomials suffices for some applications of the long code, other applications require other theorems, and in particular the invariance principle, shown for the hypercube by Mossel, O’Donnel and Oleszkiewicz [MOO05].Roughly speaking their invariance principle says that for “nice” functions on the vertices of the -dimensional noisy hypercube, the distribution of where is a random vertex is close to the distribution of where consists of independent standard Gaussian random variables (appropriately extending to act on ). To obtain more efficient version of these applications, we first show that the same holds even when is a random vertex in our smaller subset of -dimensional strings – the Reed–Muller codewords. Our central tool is a recent result by Meka and Zuckerman [MZ10] which derandomizes the invariance principle of Mossel et al. Our key insight is that taking a random Reed–Muller codeword can in fact be viewed as an instantiation of the Meka-Zuckerman generator, which involves splitting the input into blocks via a pairwise independent hash function, and using independent -wise independent distributions in each block. This allows us to obtain a version of the “Majority is Stablest” theorem for our graph, which is the main corollary of the invariance principle that is used in applications of the longcode. See Section 5 for more details.
Efficient alphabet reduction
With the “Majority of Stablest” theorem in hand, proving Theorem 4 (efficient alphabet reduction for unique games), is fairly straightforward. The idea is to simply replace the noisy hypercube gadget used by [KKMO04] with our derandomized hypercube. This is essentially immediate in the case of alphabet reduction to binary alphabet (i.e., reduction to Max Cut) but requires a bit more work when reducing to a larger alphabet. See Section 6 for more details.
Efficient hierarchy integrality gaps
Our proof Theorem 3 again works by plugging in our short code / derandomized noisy hypercube in place of the long code in the previous integrality gap constructions [KV05, KS09, RS09]. Specifically, these constructions worked by starting with an integrality gap for unique games where the basic SDP yields , and then composing it with an alphabet reduction gadget to obtain a new instance; Raghavendra and Steurer [RS09] showed that the composed instances resist rounds of the hierarchy and rounds of the hierarchy. These constructions used the noisy cube twice— both to obtain the basic unique games gap instance, and to obtain the alphabet reduction gadget. We simply plug in our short code in both usages— using for the basic unique games instance the efficient version obtained in Theorem 2, and for the alphabet reduction gadget the efficient version obtained in Theorem 4. (Luckily, our unique games instance has affine constraints and so is compatible with our alphabet reduction gadget.) The result essentially follows in a blackbox way from the analysis of [RS09]. See Section 8 for details.
Preliminaries
Let be a regular graph with vertex set . For a subset we define the volume of , denoted , to be . We define the expansion of , denoted , to be the probability over a random edge , conditioned on that . Equivalently (since is regular), where is the degree of the graph and is the number of edges going from to . Throughout, we denote the normalized adjacency matrix of a graph also by , and refer to the spectrum of the adjacency matrix as the spectrum of the graph . Note that by definition, every regular graph has maximum eigenvalue . In this paper, we use expectation norms for real-valued functions. That is, for a function and , we let .
Many of the unique games instances that appear in this work belong to a special subclass of unique games, namely instances defined below.
Given a group , an instance consists of a system of linear equations over the group where each equation is of the form for some .
Let be an code, that is, is a -dimensional linear subspace of with minimum distance (). (In this paper, we are mostly interested in the extremely high rate regime when is very small compared to and are happy with being some large constant.) Let denote Hamming distance between . For and a code we define
We say a distribution over is a canonical tester for if every vector in the support of the distribution is a codeword . The query complexity of is the maximum weight of a vector in its support. The tester’s soundness curve is defined as
Similarly, we denote the rejection probability of for a vector by . We let the query probability of a tester be the expected fraction of queried coordinates, that is, . We say that a tester with query probability is smooth if for any coordinate , and we say it is -smooth if in addition, for any two distinct coordinates , .
Finally, the following simple lemma gives some estimates for rejection probabilities of vectors for smooth testers.
If is a smooth canonical tester with query probability , then for every vector . Furthermore, if is -smooth, then for every vector with .
Fix and let . Without loss of generality, we may assume . By renaming coordinates, we may assume and . Then, . On the other hand,
We review the prerequisites for Majority is Stablest and Unique Games related results in the corresponding sections.
Small Set Expanders from Locally Testable Codes
In this section we first use some known properties of hypercontractive norms to give a sufficient condition for graphs to be small set expanders. We then describe a generic way to construct graphs satisfying this condition from locally testable codes, proving Theorem 1.
Let be a subspace of the set of functions from to for some finite set . We denote by the projection operator to the space . For , we define
We now relate this notion to small set expansion. We first show that a subspace with bounded norm cannot contain the characteristic function of a small set:
Let such that then .
Note that if and , then , meaning the projection of onto is small. It is often easier to work with the norm instead of the norm. The following lemma allows us to use a bound on the former to bound the latter:
Let and let . We know that
Dividing by yields the result. ∎
We now conclude that graphs for which the top eigenspace has bounded norm are small set expanders. The lemma can be viewed qualitatively as a generalization of one direction of the classical Cheeger’s inequality relating combinatorial expansion to eigenvalue gap [Che70].
Let be regular graph, and be the span of the eigenvectors of with eigenvalue larger than . Then, for every ,
Let be the characteristic function of , and write where (and so is the projection to the eigenvectors with value at most ). Let . We know that
Plugging this into (4.1) yields the result. ∎
2 Cayley graphs on codes
Motivated by the previous section, we now construct a graph for which the projection operator on to the top eigenspace is hypercontractive, i.e., has small norm, while also having high rank.
By Xoring the results of mulitple tests, one can let the soundness tend to . Hence, if is significantly larger than (for appropriate ), one can obtain a graph with many large eigenvalues such that small enough sets have near-perfect expansion.
Note that if , then the minimum weight representative in is unique. (This uniqueness will allow us later to define low-degree influences of functions, see Section 5.)
We let denote the eigenvalue corresponding to character . The following observation connects the soundness of the canonical tester to the spectrum of :
For any , .
From standard facts about Cayley graphs, it follows that
We use this to show that many dictator cuts in which correspond to characters with degree have eigenvalues close to . We let denote . As noted before, for these are distinct characters.
We have for at least coordinates .
We have . Since for every ,
So we can have for at most coordinates. ∎
Another immediate conseuqence of Lemma 4.5 is that large degree characters have small eigenvalues.
If , then .
Subspace Hypercontractivity
Given a function we can write it (uniquely) as a linear combination of the characters
where is the Fourier transform of (over the abelian group ).
We define the degree of , denoted to be Note that and . The following crucial observation follows immediately from the fact that has minimum distance .
The uniform distribution on is wise independent. That is, for any such that we have .
The proof follows from the following two facts:
This bound on the norm is known to hold for true low degree polynomials under the uniform distribution on the hypercube by the Bonami-Beckner-Gross inequality [O’D08].
Combining the above bound with Lemma 4.3 we get that, if the local tester rejects sufficiently far codewords with high probability, then the resulting graph is a small set expander:
In particular, as tends to , the expansion of small sets tends to . This corollary together with Corollary 4.6 completes the proof of Theorem 4.4.
3 A Canonical Tester for Reed Muller codes
There exists a constant such that for all , and the tester described above has soundness .
For a graph the continuous-time random walk on with parameter is described by the (stochastic) matrix . and have the same eigenvectors and the eigenvalues of are , where is the spectrum of .
If , where is an absolute constant.
For all for some constant , if , .
For , . Therefore, if , . For , by Corollary 4.7, for a universal constant . Therefore, for .
We now prove the second bound. If , we have which implies . Otherwise, if , our assumption implies , hence for all . For , . Hence for all . In this case, we get .
For any , there exists a graph with eigenvalues larger than for and where every set has expansion
Then, for every , , we have . Hence , . Therefore, there are at least eigenvalues which are larger than .
The fact that our graph is a Cayley graph over has a potentially interesting implication for coding theory. By looking at the set of edge labels as the rows of a generating matrix for a code, we know that large Fourier coefficients corresponds to low weight codewords, and hence we get a code of dimension that has an almost exponential (i.e. ) number of codewords of low weight, but yet has small generalized Hamming distance in the sense that every subspace of codimension contains a codeword of fractional Hamming weight . In particular by setting to be a function slowly tending to infinity we can get a linear code for which correcting from an fraction of corruption errors requires an almost exponential list size, but for which one can correct a fraction approaching of erasure errors using a list of constant size. (The code obtained by taking all edges of our graph has an almost exponential blowup, but this can be reduced by subsampling the edges.)
Majority is Stablest over Codes
In this section we show an analogue of the “Majority is Stablest” result of Mossel et al. for the RM graph we constructed in the previous section; this will help us replace the noisy cube with the RM graph in various unique games gadgets.
For , let be the Gaussian noise stability curve defined as follows. For , let be such that . Then, , where is a two-dimensional mean zero Gaussian random vector with covariance matrix \left(\begin{array}[]{cc}1&\rho\\ \rho&1\end{array}\right). We refer the reader to Appendix B in Mossel et al. [MOO05] for a more detailed discussion on .
Let be a -variate multilinear polynomial . Define and we say is -regular if for every , .
Throught this section, we let (we omit when the dimension is clear) denote the noisy hypercube graph with second largest eigenvalue .
The following theorem shows that, in the context of noise stability, a regular function on the hypercube behaves like a function on Gaussian space.
Let be a function with . Suppose for all . Then,
where is the Boolean noise graph with second largest eigenvalue and is the Gaussian noise stability curve.
We will need the following ingredient of the proof of Theorem 5.1 from [MOO05]. For , let be the functional . For a real-valued random variable , the expectation is the -distance of to the set of -valued random variables (over the same probability space as ). We will be interested in the case and . For this case, we abbreviate .
We will need the following corollary of that can handle functions that are not -valued functions.
Let be a function with and . Suppose for all . Then,
where is the Boolean noise graph with second largest eigenvalue and is the Gaussian noise stability curve. (Here, we assume that is small enough.)
Let be the closest $f\lVert f-f^{\prime}\rVert\leqslant\sqrt{\tau}\operatorname{Inf}^{\leqslant 20\log(1/\tau)}_{i}f^{\prime}\leqslant\tau+O(\sqrt{\tau})\ll\tau^{1/3}\operatorname*{\varmathbb{E}}f^{\prime}\leqslant\operatorname*{\varmathbb{E}}f+\sqrt{\tau}\langle f,T_{\rho}f\rangle\leqslant\langle f^{\prime},T_{\rho}f^{\prime}\rangle+O(\sqrt{\tau})f^{\prime}\Gamma_{\rho}(\mu+\sqrt{\tau})\leqslant\Gamma_{\rho}(\mu)+2\sqrt{\tau}$. See Lemma B.3 in [MOO05].) ∎
We remark that although we specialize to Reed–Muller codes in this section, most of the arguments generalize appropriately to arbitrary codes with good canonical testers modulo a conjecture about bounded independence distributions fooling low-degree polynomial threshold functions. We briefly discuss this in Section 5.3.
To state our version of “Majority is Stablest” we first extend the notion of influences to functions over Reed–Muller codes. For , , let be the Reed–Muller code and let be its dual . For the rest of this section we assume that a set of representatives corresponding to the minimum weight codeword in each coset is chosen for the coset space .
where denotes the variance of .
We are now ready to state the main result of this section generalizing the Majority is Stablest result to Reed-Muller codes. Let be the canonical tester for as defined in Section 4.3.
where and is the noise stability curve of Gaussian space.
The proof of the theorem proceeds in three steps. We first show that the eigenvalue profile of the graph is close to the eigenvalue profile of the Boolean noise graph (see Lemma 4.13). We then show an invariance principle for low-degree polynomials (and as a corollary for smoothed functions) showing that they have similar behaviour under the uniform distribution over the hypercube and the uniform distribution over the appropriate Reed–Muller code. Finally, we use the invariance principle to translate the majority is stablest result in the hypercube setting to the Reed–Muller code. The above approach is similar to that of Mossel et al., who translate a majority is stablest result in the Gaussian space to the hypercube using a similar invariance principle.
We first state the invariance principle that we use below (see the next subsection for the proof). Recall the definition of the functional from Section 5.1.
The (somewhat technical) proof of Theorem 5.6 from the above invariance principle closely follows the argument of Mossel et al. and is defered to the appendix – see Section A.
2 Invariance Principles over Reed–Muller Codes
The various invariance principles of Mossel et al. [MOO05] are essentially equivalent (upto some polynomial loss in error estimates) to saying that for any low-degree regular polynomial , the polynomial threshold function (PTF) cannot distinguish between the uniform distribution over the hybercube and the standard multivariate Gaussian distribution .
Ideally, we would like a similar invariance principle to hold even when is chosen uniformly from the codes of the earlier sections instead of being uniform over the hypercube. Such an invariance principle will allow us to analyze alphabet reductions and integrality gaps based on graphs considered in earlier sections (e.g., the RM graph). We obtain such generic invariance principles applicable to all codes modulo certain plausible conjectures on low-degree polynomials being fooled by bounded independence.
For the explicit example of Reed–Muller code we bypass the conjectures and directly show an invariance principle by proving that the uniform distribution over the Reed–Muller code fools low-degree PTFs. To do so, we will use the specific structure of the Reed–Muller code along with the pseudorandom generator (PRG) for PTFs of Meka and Zuckerman [MZ10]. Specifically, we show that the uniform distribution over RM can be seen as an instantiation of the PRG of [MZ10] and then use the latter’s analysis as a blackbox. Call a smooth function -nice if for every .
Choose a random and partition into blocks , with .
Choose independent samples and let be chosen according to an arbitrary distribution independent of .
OutputThe description we give here is slightly different from that of [MZ10] due to the presence of the string . However, the analysis of [MZ10] works without any changes for this case as well.
Meka and Zuckerman show that as above fool (arbitrary) low-degree polynomials. Below we state their result for regular PTFs which suffices for our purposes and gives better quantitative bounds.
For simplicity, in the following discussion we view as generating a vector in and show that the uniform distribution over has the appropriate independence structure as required by Theorem 5.10, albeit with replaced with . This does not the effect the analysis of the generator.
Let and let be the subspace of polynomials of the form
where the polynomials each have degree at most . Note that we can sample a uniformly random element by choosing independent, uniformly random degree at most polynomials for and setting as above. This is because, each collection leads to a unique element of and together they cover all elements of .
Let be a subspace of degree , -variate polynomials such that and together span all degree polynomials. Let be the space of all affine transformations. For , let be defined by and let . It is easy to see that for , the hash functions are almost pairwise independent. Observe that for and , the polynomial is uniformly distributed over all -variate degree polynomials.
Now, fix a polynomial . Then, for a random , we have
where and the polynomials , are independent uniformly random polynomials of degree at most in variables. Let denote the distribution of for a uniformly random polynomial of degree at most in variables. Then, for every fixed and , the distribution of the evaluations of restricted to different buckets are independent of one another. Moreover, within each bucket , the evaluations vector is distributed as , which is -wise independent.
Therefore, for every fixed , the distribution of is the same as the output of as defined in Equation 5.2, where . The theorem now follows from Theorem 5.10. ∎
The invariance principle of Theorem 5.9 combined with the appropriate choice of the smooth function gives us the following corollaries.
Follows from using Theorem 5.9 and an argument as in Theorem 3.19 of [MOO05] who get a similar conclusion for the hypercube starting from an invariance principle for the hypercube to the Gaussian space. ∎
Follows from Theorem 5.9 and Lemma 5.8 in [MZ10]. ∎
3 Invariance Principles over Codes
Our main tool for proving that “majority is stablest” result over Reed–Muller codes, Theorem 5.6, was the invariance principle Theorem 5.7. We conjecture that similar results should hold for any linear code with sufficiently large dual distance so that the codewords have bounded independence. In particular, we conjecture that bounded independence fools arbitrary low-degree polynomial threshold functions (PTFs) over .
The conjecture is known to be true for halfspaces [DGJ+09], degree two PTFs [DKN10] and for Gaussians with bounded independence [Kan11].
For all and , there exists such that the following holds: Let be an -variate multilinear real polynomial with degree . Let be an -wise independent distribution over and let be the uniform distribution over . Then,
Finally, we remark that for the application to “majority is stablest” it suffices to show a weaker invariance principle applicable to the functional.
For all and , there exists and such that the following holds: Let be an -variate multilinear real polynomial with degree . Let be a -wise independent distribution over and let be the uniform distribution over . Suppose that and . Then, .
We show in the appendix that Conjecture 5.13 implies Conjecture 5.14.
Efficient Alphabet Reduction
The long code over a (non-binary) alphabet consists of the set of dictator functions where for all .
A natural -query test for this code was proposed by Khot et al. [KKMO07] and analyzed in Mossel et al. [MOO05]. The queries of the test are associated with the edges of the -noise graph on . In this graph, the weight of an edge is its probability in the following sampling procedure: Sample uniformly at random and resample each coordinate of independently with probability to generate .
In this section, we present a more efficient code that serves as an analogue for the long code over a non-binary alphabet. For , let and let be the Reed–Muller code and let be its dual . Let denote the canonical test set for the code as in Section 4.3.
Let and let . We define the following distribution over (the -fold direct sum of , a subspace of ),
Sample from the test set .
Sample from at random.
Sample by setting
Since the noise graph on is a Cayley graph over the abelian group , the characters of this group form a basis of eigenfunctions. For , let denote the character
The eigenvalue of in the -noise graph on where is the Hamming weight of as a length- string over alphabet . (In this section will always refer to the Hamming weight of strings over alphabet .)
where is the Hamming weight of seen as a length- string over alphabet . (Here, the minimum is over all that lie in the same coset as in .)
We will first prove an upper bound on for the case that . We write with . Let be a string drawn from the distribution . Note that , where and are sampled as in the definition of . Since is a random vector in , we can upper bound ,
Without loss of generality, we may assume that has Hamming weight (as a binary string) at least . By Theorem 4.11, if for sufficiently small , we can upper bound .
Next, we will estimate (from below and above) for . Let be the set of coordinates with . We claim,
We write with . Then, . We refine the event according to the cardinality of . If , then . On the other hand, conditioned on , the event is equivalent to the event with . Since , this event has (conditional) probability . Hence,
which implies the claimed estimate for .
It remains to estimate the distribution of . The argument is similar to the proof of Lemma 3.3. For every coordinate , we have . Thus, . On the other hand, for any two distinct coordinates , we have . Therefore,
Similarly, . We conclude that
(Note that the estimate is only meaningful when .) ∎
For an absolute constant and all , .
Influences
Let . Suppose . (Note that has the same minimum distance as . ) In this case, we will identify with the (unique) codeword of minimum weight in the equivalence class .
(Here, refers to the -th coordinate of the unique minimum-weight representative of the equivalence class .)
1 Majority is Stablest
In this section, we show an analogue of the majority is stablest theorem of [MOO05] on the -noise graph on just as Theorem 5.6 showed an analogue of the majority is stablest theorem over the Boolean noise graph.
where , and is the noise stability curve over Gaussian space.
2 222-Query Test
We will now describe a dictatorship test for functions on , analogous to the -query dictatorship test on -noise graph.
Clearly, the dictator functions are linear functions over , i.e., . This linearity is used to perform the -query test via folding. Note that for each , the constant function for all , belongs to the code . We will fold the function by enforcing that for all , for all .
Completeness
Recall that for a generated from distribution , for each (see Lemma 4.5)
It is easy to see that by construction, this property holds for the distribution also, namely,
Soundness
The probability of acceptance of the -query test is given by,
By applying Theorem 5.6, there is an appropriate choice of such that if for all then the probability of acceptance can be bounded by
Efficient integrality gaps for unique games
In this section, we present constructions of SDP integrality gap instances starting from a code along with a local tester. To this end, we make an additional assumption on the code . Specifically, let us suppose there exists a subcode of with distance . Formally, we show the following result.
Let be an linear code with a canonical tester as described in Definition 3.2. Furthermore, let be a subcode of with distance . Then, there exists an instance of unique games, more specifically a instance, whose vertices are () and alphabet such that:
The optimum value of the natural SDP relaxation for unique games is at least where is the number of queries made by the canonical tester .
Instantiating the above theorem with the Reed–Muller code and its canonical tester we obtain the following explicit SDP integrality gap instance.
For every integer , there exists a instance on vertices such that the optimum value of the SDP relaxation on is while every labelling of satisfies at most fraction of edges.
Fix the code to be the Reed–Muller code of degee over variables. The block length of the code is , while the rate is . This code contains the Hadammard code which is of relative distance .
Let denote the canonical Reed–Muller tester for , and let denote the XOR of -independent tests. Let us fix , thus yielding a canonical tester making queries. By the work of [BKS+10], this tester has a soundness of at least . With , the above soundness is at least . Using Theorem 7.1, the optimum value of the resulting instance is at most . On the other hand, the SDP value is at least
Starting from , we construct an SDP integrality gap instance for unique games as described below.
Here we construct SDP vectors that form a feasible solution to a natural SDP relaxation of unique games [KV05].
For a vector , we will use to denote the vector whose coordinates are given by . For a pair of vectors , we have
For each vertex associate vectors . Notice that for a pair of vectors we have,
Since the distance of the code is , we have
In other words, for every vertex , the corresponding SDP vectors are orthonormal. The objective value of the SDP solution is given by,
where is the number of queries made by the canonical tester for .
Soundness
The expectation of the function is given by,
Since is bounded in the range $$ we have,
Applying Corollary 4.10, we get that for each ,
Hierarchy integrality gaps for Unique Games and Related Problems
This section is devoted to the construction of a integrality gap instance for a hierarchy of SDP relaxations to Unique Games. More specifically, we consider the and SDP hierarchies described in [RS09]. For these SDP hierarchies, we will demonstrate the following integrality gap constructions.
For every , there exists an instance for some positive integer , such that no labelling satisfies more than fraction of edges of while there exists an SDP solution such that,
the SDP solution is feasible for with .
the SDP solution is feasible for with .
the SDP solution has value .
where is the number of vertices in the instance .
Composing the above SDP integrality gap with Unique games based hardness reductions yields corresponding gap instances for several classes of problems like constraint satisfaction problems (CSPs) and ordering CSPs like maximum acyclic subgraph. Specifically, up to rounds of hierarchy or the rounds of the hierarchy can be shown to have te same SDP integrality gap as the simple SDP relaxation for every CSP.For the sake of brevity, we omit a formal statement of this result here.
Towards showing Theorem 8.1, we follow the approach outlined in [RS09]. At a high-level, the idea is to start with an integrality gap instance for a simple SDP relaxation for unique games over a large alphabet. The instance is reduced to an instance of unique games over a smaller alphabet using a reduction similar to Khot et al. [KKMO07]. Moreover, the SDP solution to the simple SDP relaxation of can be translated to a solution for several rounds of SDP hierarchy for .
Let be an instance of over a set of vertices and edges . On every edge , there is a constraint of the form for some . We will reduce to an instance of instance using the two query test described in Section 6.
Notice that Reed-Muller codes are invariant under translation of its coordinates. Therefore, the code and the test distributions are both invariant under translation. Formally, for an , the translation operator is defined by
Given a codeword , we have .
We are now ready to describe the reduction from to an instance of .
Soundness
For all sufficiently small constants and all choices of , there exists such that if no labelling of satisfies more than fraction of edges, then every labelling of satisfies at most fraction of constraints, where .
Due to folding we have for all . Moreover, this implies that . Finally, for a vertex and define,
Clearly, for the functions also we have,
The probability of acceptance of the verifier can be arithmetized in terms of the functions .
Suppose the probability of acceptance of the verifier is at least . By simple averaging, for at least fraction of the vertices we have,
Let us refer to such a vertex as being good.
Fix the parameters to those obtained by applying Theorem 6.4 with parameters . Recall that by (8.1), we have . Applying Theorem 6.4, if for each , then,
This implies that for each good vertex there exists such that . We will use these influential coordinates to decode a labelling for the instance .
For each vertex define the set of influential coordinates as,
Using Lemma A.1, for each of the functions or , there are at most coordinates with influence greater than . Therefore, for each vertex the set is of size at most .
Define an assignment of labels as follows. For each vertex , sample a random and assign .
Fix one good vertex , and a corresponding such that . By definition of this implies that
Hence, for at least a fraction of the neighbours , the coordinate has influence at least on . Therefore, for every good vertex , for at least fraction of its neighbours , the edge is satisfied by the labelling with probability at least . Since there are at least a -fraction of good vertices , the expected fraction of edges satisfied by the labelling is at least .
SDP Solution
We will construct feasible solutions to certain strong SDP relaxations of by appealing to the work of [RS09]. The SDP hierarchies that we consider are referred to as the and hierarchies. Informally, the -level relaxation () consists of the simple SDP relaxation for unique games augmented by local distributions over integral assignments for every set of at most vertices. The local distribution is required to be consistent with the inner products of the SDP vectors. Alternately, this SDP hierarchy can be thought of as, the simple SDP relaxation augmented by every valid constraint on at most vertices.
The hierarchy is a somewhat stronger hierarchy that requires the local distributions to be consistent with each other, namely, and agree on . Alternately, the hierarchy corresponds to the simple SDP relaxation augmented with -rounds of Sherali-Adams LP variables. We refer the reader to [RS09] for formal definitions of the and hierarchies.
Suppose has an SDP solution that of value , then there exists an SDP solution to the instance such that,
the SDP solution is feasible for with .
the SDP solution is feasible for with .
the SDP solution has value on .
This lemma is a direct consequence of Theorem from [RS09].
In [RS09], the authors start with an integrality gap instance for the simple SDP for unique games, and then perform a traditional long code based reduction to obtain an instance .
The crucial observation is the following.
The vertices of are a subset of vertices of – the instance obtained by the traditional -ary long code reduction on .
The vertices of are pairs of the form where and . The codeword can be thought of as a string of length over the alphabet , namely, . The vertices of the instance obtained via a traditional -ary long code reduction is . Hence the observation follows. ∎
In [RS09], the authors construct an SDP solution for the instance that is feasible for relaxation with and for relaxation with . As noted in 8.5, the vertices of are a subset of the vertices of . Therefore, the same SDP solution constructed in [RS09] when restricted to the instance yields a feasible solution for the corresponding and relaxations.
To finish the proof, we need to show that the value of the SDP solution from [RS09] is .
The traditional long code based reduction to get uses the noise stability test as the inner gadget. Namely, to test if a function is a dictator function, the verifier picks uniformly at random, and rerandomizes each coordinate of independently with probability , and then tests if . Composing this noise stability test with the outer unique game yields the instance . The value of the SDP solution constructed for in [RS09] depends only on the expected hamming distance between the queries . More precisely, in Claim of [RS09], the authors show that if the distribution on the queries is chosen to be an arbitrary distribution over , the SDP objective value of the solution is given by
Fix and . By our choice of , we have (see Appendix B in [MOO05] for such asymptotic bounds on ).
Fix depending on and as dictated by Lemma 8.3. Let be the Unique games instance obtained by Corollary 7.2 with the optimal integral value set to . In particular, is a instance that has vertices. Its SDP optimum for the simple Unique games SDP relaxation is at least () for some constant depending on .
Now we apply the reduction to outlined below to obtain an instance . The number of vertices of the instance is . Note that the choice of the degree is a constant (say ) depending on . Hence, the number of points in is given by . Therefore, the number of vertices of is . Equivalently, we have .
By Lemma 8.3, the optimal labelling to satisfies at most fraction of constraints.
By Lemma 8.4, there exists an SDP solution to the instance with value . Since , for large enough choice of , the SDP value is at least .
The SDP solution is feasible for for rounds, where is a constant depending on and . Furthermore, the SDP solution is also feasible for for .
Acknowledgements.
We have had many beneficial conversations on this question with a number of researchers including Venkatesan Guruswami, Ran Raz, Alex Samorodnitsky, Amir Shpilka, Daniel Spielman, Ryan O’Donnell, Subhash Khot and Madhu Sudan. We thank Avi Wigderson for suggesting the name “short code”.
References
Appendix A Missing Proofs
Let and . Then, the graph has the same eigenfunctions as - for with eigenvalues . From the above equation, it is easy to check that, for ,
Further, as the eigenvalues of are each at most , the coordinate influences of are no larger than those of .
Hence, using Equation (A.2) (and the crude bound ),
Let be the set of -vectors corresponding to the Reed–Muller code , that is, for every codeword , the set contains the vector . Then, as is $\mathcal{C}^{\bot}\zeta$ measures distance to bounded random variables, by Equation (A.1),
Since and (cf. Lemma B.3, Corollary B.5 in [MOO05]), it follows from Equations (A.3) and (A.4) that
(Here we used the estimate .) By choosing and for an appropriately large constant , the above expression simplifies to
Since is a non-negative random variable,
The following lemma shows a bound on the sum of influences.
The usual identity for the total (low-degree) influence holds,
Analogous to Theorem 5.7, the following invariance principle can be shown for regular multilinear polynomials.
The proof follows easily from the proof of Theorems 5.9 and 5.7 and the fact that if satisfies the properties of the PRG in [MZ10], then so does . We omit the proof.
The work of Mossel et al. [MOO05] also obtains bounds on noise stability of functions over product spaces of large alphabets namely . The following corollary is a consequence of Theorem in [MOO05]. The proof is analgous to that of Corollary 5.3 from Theorem 5.1.
Let be a function with and . Suppose for all . Then,
where is the noise graph on with second largest eigenvalue and is the Gaussian noise stability curve. (Here, we assume that is small enough.)
Now we are ready to present the proof of the majority is stablest theorem over (Theorem 6.4) using Theorem A.2 and Corollary A.3.
Let and . Then, the graph has the same eigenfunctions as - for with eigenvalues . From the above equation, it is easy to check that, for ,
Hence, using Equation (A.6) (and the crude bound ),
Let be the set of -vectors corresponding to the Reed–Muller code , that is, for every codeword , the set contains the vector . Then, as is $\mathcal{D}^{t}\zeta$ measures distance to bounded random variables, by Equation (A.5),
Since and (cf. Lemma B.3, Corollary B.5 in [MOO05]), it follows from Equations (A.7) and (A.8) that