Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

Introduction

Many machine learning applications, from generative adversarial networks (GANs) to robust reinforcement learning, result in nonconvex-nonconcave constrained minimax problems, which pose notorious difficulties to the scalable first order methods. Indeed, there is no shortage of results illustrating divergent or cycling behavior when going beyond minimization problems (Benaım & Hirsch, 1999; Hommes & Ochea, 2012; Mertikopoulos et al., 2018b; Hsieh et al., 2021).

Traditionally, minimax problems have been studied for more than half a century under the umbrella of the variational inequalities (VIs). The extragradient-type algorithms from the VI literature was recently brought to the awareness of the machine learning community (Mertikopoulos et al., 2018a; Gidel et al., 2018; Böhm et al., 2020), and have provided a principled way of stabilizing training and avoiding Poincaré recursions. However, these results mostly concern the convex-concave setting.

In nonconvex-nonconcave minimax problems, or more generally nonmonotone variational inequalities (VIs), even finding a local solution is in general intractable. This has been made precise through exponential lower bound of the classical optimization type (Hirsch & Vavasis, 1987) and computational complexity results (Papadimitriou, 1994; Daskalakis et al., 2021b). This is in sharp contrast to minimization problems, where only finding a global solution is intractable. The recent result of (Hsieh et al., 2021) provides some intuition behind this difference by showing that the asymptotic limits of most schemes, including extragradient, can converge to attracting limit cycles.

To make progress in lieu of these negative results, Diakonikolas et al. (2021) proposes a simple generalization of extragradient, called (EG+), that can converge to a stationary point even for a class of nonmonotone problems provided that the weak Minty variational inequality (MVI) holds. This problem class is parametrized by a constant $\rho$, which controls the degree of nonconvexity. However, given the range of $\rho$ in Diakonikolas et al. (2021), the new class is still too small to include even the simplest counterexample of Hsieh et al. (2021) for the general Robbins-Monro schemes.

Building on the analysis in Diakonikolas et al. (2021), we propose a new adaptive scheme, called (CurvatureEG+), that converges even in the difficult counter example of Hsieh et al. (2021) as illustrated in Fig. 1. Our main contributions are summarized below.

We propose an adaptive extragradient-type algorithm that converges for a larger range of $\rho$, the parameter in the weak MVI assumption (cf. Item 3) than previously known.

More importantly, we show that convergence is ensured if $2\rho+\gamma_{k}>0$, where $\gamma_{k}$ is the extrapolation stepsize. This is crucial since by selecting $\gamma_{k}$ through a backtracking procedure larger stepsizes are allowed, which in turn implies convergence for more negative values of $\rho$, thus capturing a larger class of problems. In addition, we show that the linesearch eventually passes without triggering any backtrack if initialized based on the Jacobian of $F$ (cf. Section 4).

We present a non-adaptive variant of our algorithm (CEG+), and show that for particular parameter choices (EG+) of Diakonikolas et al. (2021), and when $\rho=0$ the celebrated forward-backward-forward (FBF) algorithm of Tseng (2000) are recovered, thus unifying and generalizing both methods. We improve upon Diakonikolas et al. (2021) by not only relaxing the problem class but also the stepsize range. We show that our results are tight by providing a matching lower bound, thus providing a complete picture of (EG+) under weak MVI.

Related work

The community has resorted to various approaches to make progress for nonconvex-nonconcave minimax problems. One line of work focuses on deriving local convergence results (Mazumdar et al., 2019; Fiez & Ratliff, 2020; Heusel et al., 2017). For global results, the two primary approaches have been to either assume a global oracle for the inner problem (Jin et al., 2019; Davis & Drusvyatskiy, 2018) or assume particular problem structure such as the Polyak-Łojasiewicz condition (Nouiehed et al., 2019; Yang et al., 2020) or concavity for the inner problem (Rafique et al., 2019).

We follow the same tradition of assuming structure, but from the general perspective of operator theory. The idea of studying minimax and related problems through the lens of variational inequality has a long history (Minty, 1962; Rockafellar, 1976; Polyak, 1987; Bertsekas, 1997), with recent renewed interest due to its relevance for minimax formulations (Mertikopoulos et al., 2018a; Gidel et al., 2018; Azizian et al., 2020).

One relaxation of the monotone case for which we have positive results is that of Minty variational inequalities (MVI) (Mertikopoulos et al., 2018a; Song et al., 2021; Zhou et al., 2017; Liu et al., 2021), which includes all quasiconvex-concave and starconvex-concave problems. Diakonikolas et al. (2021) introduced the relaxed condition of weak MVI. In the unconstrained setting they showed non-asymptotic convergence results under a restricted problem constant $\rho$. Similarly to us, Lee & Kim (2021a) extends the regime but under the stronger condition of cohypomonotonicity. They do so by studying a more evolved variant of extragradient building on anchoring techniques. We instead directly improve upon (EG+) and generalize it to new settings.

In the stochastic setting, usually the stepsize for the extrapolation step is diminishing. This is the case in Böhm et al. (2020) where they consider a forward-backward-forward type scheme. However, they remain in the monotone setting, where the limit cycles are non-attracting, as exemplified by a bilinear game. Hsieh et al. (2021) recently showed that a large family of algorithms, which includes the extragradient method with diminishing stepsize, can converge to attracting limit cycles. Going beyond this restriction, prior to Diakonikolas et al. (2021), Hsieh et al. (2020) interestingly considers two separate and diminishing stepsizes under the stronger assumption of MVI.

Problem formulation and preliminaries

holds. The set of all such points is denoted by $\operatorname*{zer}T\coloneqq{\mathopen{}\left\{z\in R^{n}{}\mid{}0\in Tz\right\}\mathclose{}}$. Throughout the paper problem (2.1) is studied under the following assumptions (definitions can be found in Appendix A).

Weak Minty variational inequality (MVI) holds, i.e., there exists a nonempty set $\mathcal{S}^{\star}\subseteq\operatorname*{zer}T$ such that for all $z^{\star}\in\mathcal{S}^{\star}$ and some $\rho\in(-\tfrac{1}{2L},\infty)$

Generally, we do not require the weak Minty assumption to hold at every $z^{\star}\in\operatorname*{zer}T$. In fact, as shown in Theorem 3.1 nonemptiness of $\mathcal{S}^{\star}$ is sufficient for ensuring that the limit points belong to $\operatorname*{zer}T$. Interestingly, despite nonmonotonicity of $F$, global (as opposed to subsequential) convergence can be established when $\mathcal{S}^{\star}=\operatorname*{zer}T$, an assumption that is still weaker than cohypomonotonicity.

VIs provide a convenient abstraction for a range of problems. We mention some central examples below but otherwise defer to the overview in Facchinei & Pang (2007). Subsequently, we provide examples where the weak MVI holds.

Example 1: (minimax optimization). A comprehensive way to capture a wide range of applications in machine learning is to consider structured minimax problems of the form

As it will become clear in the next section (cf. Algorithm 1), the main computations involved in the proposed scheme are evaluations of $F$ and resolvent $J_{A}={\mathopen{}\left({\rm id}+A\right)\mathclose{}}^{-1}$. Recall that the resolvent of a maximally monotone operator is firmly nonexpansive with full domain (cf. (Bauschke & Combettes, 2017, Sect. 23)). If $A=\partial f$ is the subdifferential operator of a convex function $f$, then its resolvent is the proximal mapping. For instance when $A$ is as in Assumption I, then its resolvent is given by $J_{A}(x,y)=(\operatorname{prox}_{g}(x),\operatorname{prox}_{h}(y))$.

The corresponding first order optimality conditions may be written as $Az=(\partial g_{1}(z_{1}),\ldots,\partial g_{N}(z_{N}))$ and Fz=({\nabla}_{z_{1}}\varphi_{1}(\mathchoice{\text{\boldmath{\displaystyle z}}}{\text{\boldmath{\textstyle z}}}{\text{\boldmath{\scriptstyle z}}}{\text{\boldmath{\scriptscriptstyle z}}}),\ldots,{\nabla}_{z_{N}}\varphi_{N}(\mathchoice{\text{\boldmath{\displaystyle z}}}{\text{\boldmath{\textstyle z}}}{\text{\boldmath{\scriptstyle z}}}{\text{\boldmath{\scriptscriptstyle z}}})).

A solution to (2.1) thus returns a candidate for which the first order condition of the above problems is satisfied. In the monotone case these two solution concepts coincide, while in the more general case of weak MVI, we provide examples where this still holds. In particular, we introduce in Section 5 a nonconvex-nonconcave minimax game which additionally exhibits limit cycles for $Fz$. As a consequence most schemes including gradient descent ascent, extragradient and optimistic gradient descent ascent do not converge to a stationary point globally (Hsieh et al., 2021). However, the global Nash equilibrium satisfies Item 3 with $\rho>-\nicefrac{{1}}{{2L}}$, which we show is sufficient for global convergence of (CEG+).

The weak MVI condition is satisfied in certain reinforcement learning settings. Specifically, Diakonikolas et al. (2021); Daskalakis et al. (2021a) considers a two-player zero-sum game where the weak MVI holds, while neither MVI nor cohypomonotonicity holds. Interestingly, the formulation requires constraint—a condition they do not handle. We thus provide the first provable algorithm for this setting. Weak MVI also contains all quasiconvex-concave and starconvex-concave problems. For further examples, the literature on cohypomonotonicity (Bauschke et al., 2020) is relevant since it implies weak MVI, see for instance Lee & Kim (2021b, Example 1).

Generalizing Extragradient+

Our starting point is the Extragradient+ (EG+) algorithm of Diakonikolas et al. (2021) which is identical to extragradient (Korpelevich, 1976) except for the second stepsize being smaller. They only treat the inclusion (2.1) when $A\equiv 0$, and in our notation require $\rho\in(-\nicefrac{{1}}{{8L}},0]$. Specifically,

where they choose $\gamma_{k}=\nicefrac{{1}}{{L}}$ and $\bar{\alpha}_{k}=\nicefrac{{1}}{{2}}$ (Diakonikolas et al., 2021, Thm. 3.2).

We generalize (EG+) in Algorithm 1 to take the operator $A$ into account—consequently we capture constraint and regularized problems as well. In addition, the scheme is adaptive in $\bar{\alpha}_{k}$. We will show that the weaker requirement of $\rho\in(-\nicefrac{{1}}{{2L}},\infty)$ suffices even for the more general inclusion (2.1).

The main convergence results of Algorithm 1 are established in the next theorem. The proof is largely inspired by recent developments in operator splitting techniques in the framework of monotone inclusions (Latafat & Patrinos, 2017; Giselsson, 2021). The key idea lies in interpreting each iteration of the algorithm as a projection onto a certain hyperplane, an interpretation that dates back to Solodov & Tseng (1996); Solodov & Svaiter (1999).

where $\kappa=\liminf_{k\to\infty}\lambda_{k}(2-\lambda_{k})(\delta_{k}+\nicefrac{{\gamma_{k}}}{{2}})^{2}$. Moreover, the following holds

Note that whenever $\limsup_{k\to\infty}\gamma_{k}<\tfrac{1}{L}$, Item 2 may be used to derive a similar inequality in terms of $\|\bar{z}^{k}-z^{k}\|$ by lower bounding $\|H\bar{z}^{k}-Hz^{k}\|$ in (3.1). We also remark that tighter rates may be obtained in the regime $\rho\geq 0$, however, this will not be pursued in this work.

Although we do not incur additional costs for evaluating the adaptive stepsize $\alpha_{k}$ in 1.4, it proves instructive to present a variant with constant stepsize. As a result we compare the range of our stepsizes against Diakonikolas et al. (2021) showing an improvement by a factor of $\nicefrac{{3}}{{2}}$. Moreover, in the monotone case ($\rho=0$), with a certain choice of stepsizes the algorithm reduces to the celebrated forward-backward-forward (FBF) algorithm of Tseng (2000). We remark that the relation of FBF to projection-type algorithms was noted in Tseng (2000), (Giselsson, 2021, Sect. 6.2.1).

To this end, in this subsection consider the following non-adaptive variant of Algorithm 1 that generalizes (EG+). Letting $\bar{\alpha}_{k}\in(0,1+\nicefrac{{2\delta_{k}}}{{\gamma_{k}}})$:

The convergence of this algorithm is an immediate byproduct of Theorem 3.1. To see this, note that the $z^{k+1}$ update in 1.5 may be written as $z^{k+1}{}={}z^{k}{}+{}2\eta_{k}\alpha_{k}(H\bar{z}^{k}-Hz^{k})$, for $\eta_{k}\in(0,1)$. Therefore, convergence is still ensured for any $\bar{\alpha}_{k}<2\alpha_{k}$ as the difference may be absorbed by the relaxation parameter $\eta_{k}$. Note that by $\nicefrac{{1}}{{2}}$-cocoercivity of $H$ (cf. Item 1)

where $\kappa=\bar{\alpha}(1+\tfrac{2\delta}{\gamma}-\bar{\alpha})$. Moreover, the claims of Items 1 and 2 hold true.

The setting of Diakonikolas et al. (2021) in (EG+) involves the stepsizes $\gamma_{k}=\nicefrac{{1}}{{L}}$, $\alpha_{k}=\nicefrac{{1}}{{2}}$. Note that when restricting to $A\equiv 0$, the iterates (CEG+) simplify to this form owing to the fact that $H\bar{z}^{k}-Hz^{k}=\bar{z}^{k}-\gamma F\bar{z}^{k}-Hz^{k}=-\gamma F\bar{z}^{k}$. In comparison, in our setting if $\delta=\rho=-\nicefrac{{1}}{{8L}}$ (the smallest $\rho$ permitted in Diakonikolas et al. (2021)) is selected, then based on our analysis in Corollary 3.2 we may select $\gamma_{k}=\nicefrac{{1}}{{L}}$, and $\bar{\alpha}_{k}\in(0,\nicefrac{{3}}{{4}})$, thus the upper bound for the second stepsize is $\nicefrac{{3}}{{2}}$ times that of Diakonikolas et al. (2021).

In Corollary 3.2 the range of stepsizes $\gamma$, $\bar{\alpha}$ may alternatively be set as \gamma\in\big{(}\lfloor-2\rho\rfloor_{+},\nicefrac{{1}}{{L}}\big{)}, $\bar{\alpha}\in(0,1+\nicefrac{{2\delta}}{{\gamma}}]$. This is due to the fact that if $\gamma<\nicefrac{{1}}{{L}}$ (strictly), then $H$ is strictly $\nicefrac{{1}}{{2}}$-cocoercive. Therefore, in (3.2), $1+\tfrac{2\delta}{\gamma}<2\alpha_{k}$ holds, and thus the stepsize $\bar{\alpha}=1+\tfrac{2\delta}{\gamma}$ is permitted. Although this may appear to be of little practical significance, by setting $\gamma\in(0,\nicefrac{{1}}{{L}})$, $\delta=\rho=0$, and $\bar{\alpha}=1$ in (CEG+), we obtain $z^{k+1}=\bar{z}^{k}+\gamma Fz^{k}-\gamma F\bar{z}^{k}$, which is the forward-backward-forward (FBF) algorithm of Tseng (2000), (Bauschke & Combettes, 2017, Thm. 26.17)). ∎

2 Lower bounds

We show that the result in Corollary 3.2 is tight by providing a matching lower bound when $A\equiv 0$. We do so by fixing $\bar{\alpha}_{k}$ and showing a stepsize dependent lower bound. In particular, note that if $\bar{\alpha}_{k}=\nicefrac{{1}}{{2}}$ as in Diakonikolas et al. (2021, Thm. 3.2), then Theorem 3.4 implies a lower bound of $\rho>-\nicefrac{{1}}{{4L}}$ for the (EG+) scheme. The lower bound is contextualized in Fig. 2 by relating it to our convergence results and existing results in the literature.

Adaptively taking larger stepsizes using local curvature

The proposed scheme involves a backtracking linesearch that uses the local curvature for its initial guess. The reason being that this will immediately pass, close enough to the solution $z^{\star}$, by argument of continuity. More precisely, we will set the initial guess to something slightly smaller than $\|JF(z^{k})\|^{-1}$, where $JF(z)$ denotes the Jacobian of $F$ at $z$ and $\|\cdot\|$ is the spectral norm. Note that, despite the use of second order information, the scheme remains efficient since $\|JF({}z{})[{}{}{}]\|$ only requires one eigenvalue computation performed through Jacobian-vector product (Pearlmutter, 1994).

Given an initial point $z^{0}=z^{\rm init}$ and $\nu\in(0,1)$, the final scheme which we denote (CurvatureEG+) proceeds for $k=0,1,\dots$ as follows:

The linesearch terminates in finite time with $\gamma\geq\min\{\gamma^{\rm init},\nicefrac{{\nu\tau}}{{L}}\}$;

The convergence results for (CurvatureEG+) are deduced based of the above lemma and Theorem 3.1 and are provided in Corollary B.1 in Section B.2. We illustrate the behavior of (CurvatureEG+) in Fig. 1 and in Section 6.

Constructing toy examples

When Item 3 holds for negative $\rho$, limit cycles of the underlying operator $Fz$ can emerge. We illustrate this with simple polynomial examples for which all the properties of interest can be computed in closed form.

A PolarGame denotes a two-player game whose associated operator $F$ has limit cycles at $\|z\|_{2}=c_{i}$ for all $i\in[k]$ where $c_{i}\neq 0$.

This turns out to be particularly easy to construct in polar coordinates as the name suggests (see Section C.1). Apart from introducing arbitrary number of limit cycles it also gives us control over $\rho$. This is illustrated in the following instantiations capturing three important cases.

Example 3: (PolarGame). Consider $Fz={\mathopen{}\left(\psi(x,y)-y,\psi(y,x)+x\right)\mathclose{}}$ where $\|z\|_{\infty}\leq\nicefrac{{11}}{{10}}$ and $\psi(x,y)=\frac{1}{16}ax(-1+x^{2}+y^{2})(-9+16x^{2}+16y^{2})$. We have the following three cases:

(i) $a=1$ then $\rho\in(-\frac{1}{L},-\frac{1}{2L})$ (ii) $a=\frac{3}{4}$ then $\rho\in(-\frac{1}{2L},-\frac{1}{3L})$ (iii) $a=\frac{1}{3}$ then $\rho\in(-\frac{1}{8L},-\frac{1}{10L})$

where $L$ denotes the Lipschitz constant of $F$ restricted to the constraint set. For all cases $F$ exhibits limit cycles at $\|z\|=1$ and $\|z\|=\nicefrac{{3}}{{4}}$. Proof is deferred to Section C.2.

Example 4: (minimax). In the particular case of constrained minimax problem we introduce the following polynomial game:

where $\psi(z)=\frac{2z^{6}}{21}-\frac{z^{4}}{3}+\frac{z^{2}}{3}$. We provide proof of the following properties in Section C.3:

There exists a repellant limit cycle and an attracting limit cycle of $F$.

$z^{\star}=(0,0)$ is a global Nash equilibrium for which Item 3 holds inside the constraint with $\rho>-\nicefrac{{1}}{{2L}}$, where $L$ denotes the Lipschitz constant of $F$ restricted to the constraint set.

Experiments

The algorithms considered in the experiments include the adaptive Algorithm 1, (CurvatureEG+), and constant stepsize methods that can be seen as instances of (CEG+) for various choices of $\gamma_{k}$ and $\bar{\alpha}_{k}$. When $\gamma_{k}=\nicefrac{{1}}{{L}}$ and $\bar{\alpha}_{k}=1$ we recover a constrained variant of extragradient, which we denote CEG. When $\bar{\alpha}_{k}=\nicefrac{{1}}{{2}}$ we denote the scheme CEG+, which is the direct generalization to the constraint setting of the (EG+) scheme studied in Diakonikolas et al. (2021, Thm. 3.2). Note that this choice of $\bar{\alpha}_{k}$ restricts the problem class for which we otherwise can have guaranteed convergence according to Corollary 3.2. When $\bar{\alpha}_{k}$ is chosen adaptively according to Algorithm 1 we refer to it as AdaptiveEG+. Finally, when $\gamma_{k}$ is additionally chosen adaptively we use the name (CurvatureEG+).

In the stochastic setting, when $\gamma_{k}=\nicefrac{{1}}{{k}}$ and $\alpha_{k}=1$, effectively both stepsizes diminish, and we recover a constrained variant of the popular stochastic extragradient scheme (see e.g. Hsieh et al. (2021, Algorithm 3)), which we refer to as SEG. We also consider a heuristic variant where $\gamma_{k}=\nicefrac{{1}}{{L}}$ and only $\alpha_{k}$ is decreasing, which we refer to as SEG+.

Conclusion

This paper introduced an EG-type algorithm for a class of nonconvex-nonconcave minimax problems that satisfy the weak Minty variational inequality (MVI). The range of parameter in the weak MVI was extended compared to EG+ of Diakonikolas et al. (2021), and tightness of our results were demonstrated through construction of a counter example. In addition, EG+ (Diakonikolas et al., 2021), as well as the forward-backward-forward algorithm (Tseng, 2000) were all shown to be special cases of our scheme. Furthermore, (CurvatureEG+) was proposed that performs a backtracking linesearch on the extrapolation stepsize $\gamma_{k}$ allowing for larger stepsizes and relaxes the condition $\rho>\tfrac{1}{2L}$ to $\rho>\nicefrac{{-\gamma_{k}}}{{2}}$ which is often a much weaker condition. More importantly, it is shown that asymptotically the linesearch always passes with $\gamma_{k}=\nu\|JF({}z^{k}{})[{}{}{}]\|^{-1}$ for any $\nu\in(0,1)$, thus ratifying the name (CurvatureEG+). Future direction include exploring applications of the proposed algorithm in particular in the setting of GANs. It is also interesting to develope a variance reduced variant of the algorithm for finite sum minimax problems.

Acknowledgments and disclosure of funding

We would like to especially thank Yu-Guan Hsieh for providing valuable feedback and discussion. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement n° 725594 - time-data). This work was supported by the Swiss National Science Foundation (SNSF) under grant number 200021_205011. The work of the second and third author was supported by the Research Foundation Flanders (FWO) postdoctoral grant 12Y7622N and research projects G081222N, G0A0920N, G086518N, and G086318N; Research Council KU Leuven C1 project No. C14/18/068; Fonds de la Recherche Scientifique – FNRS and the Fonds Wetenschappelijk Onderzoek – Vlaanderen under EOS project no 30468160 (SeLMA); European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 953348. The work of Olivier Fercoq was supported by the Agence National de la Recherche grant ANR-20-CE40-0027, Optimal Primal-Dual Algorithms (APDO).

References

Appendix A Preliminary definitions

Notationally we will use $\lfloor x\rfloor_{+}\coloneqq\max\{0,x\}$ throughout. We additionally recall some standard definitions and results and refer to Bauschke & Combettes (2017); Rockafellar (1970)) for further details.

and it is said to be $\rho$-comonotone if for all $(x,y),(x^{\prime},y^{\prime})\in\operatorname{gph}A$

The operator $A$ is said to be maximally (co)monotone if its graph is not strictly contained in the graph of another (co)monotone operator.

We say that $A$ is monotone if it is -monotone. When $\rho<0$, $\rho$-comonotonicity is also referred to as $|\rho|$-cohypomonotonicity.

Moreover, $A$ is said to be nonexpansive if it is $1$-Lipschitz continuous, and firmly nonexpansive if it is $1$-cocoercive.

The following lemma plays an important role in our convergence analysis.

$A$ is $1$-Lipschitz if and only if $T={\rm id}-A$ is $\nicefrac{{1}}{{2}}$-cocoercive.

The first claim follows directly from (Bauschke & Combettes, 2017, Prop.4.11). That $T$ is strongly monotone is a consequence of the Cauchy Schwarz inequality and Lipschitz continuity of $A$:

In turn, the last claim follows from the Cauchy-Schwarz inequality. ∎

Appendix B Proofs and further results

Let $H={\rm id}-\gamma_{k}F$. By 1.3 $Hz^{k}\in\bar{z}^{k}+\gamma_{k}A\bar{z}^{k}$. Therefore,

In what follows we will show that Algorithm 1 is equivalent to taking a forward-backward step followed by a correction step. Consider the updates

where in the inequality Item 1 was used. Hence, by (B.3) the stepsize $\alpha_{k}$ is positive and bounded away from zero. Moreover, if $z^{k}\in\mathcal{D}_{k}$, then from (B.3) we may conclude that $\|H\bar{z}^{k}-Hz^{k}\|\leq 0$ which implies that the generated sequence remains constant and $\bar{z}^{k}\in\operatorname*{zer}T$ (cf. (B.1)).

The projection onto $\mathcal{D}_{k}$ for any $v\notin\mathcal{D}_{k}$ is given by

Moreover, (B.1) together with Item 3 at $\bar{z}^{k}$ yields

thus ensuring $z^{\star}\in\mathcal{S}^{\star}\subseteq\mathcal{D}_{k}$. The projection onto $\mathcal{D}_{k}$ is then given by $\operatorname{\Pi}_{\mathcal{D}_{k}}(z^{k})=z^{k}+\alpha_{k}(H\bar{z}^{k}-Hz^{k}),$ where $\alpha_{k}$ is as in 1.4.

The proof of convergence was already given prior to the statement of the corollary. It remains to derive (3.3). By Item 3 and owing to $\nicefrac{{1}}{{2}}$-cocoercivity of $H$ (cf. Item 1)

Therefore, provided that $\bar{\alpha}>0$ we have

Telescoping the above inequality yields the claimed inequality. ∎

B.2 Convergence results and proofs of Section 4

The convergence results for (CurvatureEG+) are provided in the next corollary where $\rho$ in Item 3 is allowed to take potentially larger values provided that $\rho>-\nicefrac{{\gamma_{k}}}{{2}}$. Note that owing to the lower bound on $\gamma_{k}$ (cf. Item 1), the weak MVI assumption in the corollary is always satisfied if $\rho\in(-\nicefrac{{\nu\tau}}{{2L}},\infty)$, however, in practice $\gamma_{k}$ may take larger values.

Moreover, if $z^{k},\bar{z}^{k}\to z^{\star}\in\operatorname*{zer}T$ (as is the case in 3), and $F$ is continuously differentiable, then eventually the backtrack will never be invoked.

Observe that in the proof of Theorem 3.1 $1$-Lipschitz continuity of $\gamma_{k}F$ is only used at the generated points $\bar{z}^{k}$ and $z^{k}$ (see (B.3)), and is thus ensured by the linesearch Algorithm 2. Therefore, it is easy to see that $\alpha_{k}$ is positive and bounded away from zero provided that $\rho>-\nicefrac{{\gamma_{k}}}{{2}}$ , see (B.3). Moreover, since $\gamma_{k}\|F\bar{z}^{k}-Fz^{k}\|\leq\nu\|\bar{z}^{k}-z^{k}\|$, arguing as in Item 2 we obtain $\|H\bar{z}^{k}-Hz^{k}\|\geq(1-\nu)\|\bar{z}^{k}-z^{k}\|$. Hence, it follows from (B.5) that

1: Since $F$ is $L$-Lipschitz continuous the linesearch would terminate in finite steps. Either $\gamma^{\rm init}$ satisfies the condition, or else the backtrack procedure is invoked, which in turn implies the previous candidate $\gamma/\tau$ should have violated the condition leading the the claimed lower bound.

B.3 Proofs of Section 3.2

To prove the lower bound we introduce the following unconstrained bilinear minimax problem with an unstable critical point.

Example 5: Consider the following minimax problem:

The associated operator of Section B.3 can easily be computed,

where $z=(x,y)$. In this particular case, both $L$ and $\rho$ turn out to be constants. By simple calculation we have,

where $\|\cdot\|$ is the spectral norm. Since the norm of the Jacobian is constant it equates the global Lipschitz constant, $L=\|JF(z)\|$.

By linearity of $F$, one step of (EG+) is conveniently also a linear operator. Specifically,

We know that a linear dynamical system is globally asymptotically stable if and only if the spectral radius of the linear mapping is strictly less than 1.

Let $\lambda_{1},\lambda_{2}$ be the eigenvalues of $T$. Then the spectral radius is the largest absolute value of the eigenvalues. For $T$ this becomes,

Equation (B.13) provides a specification for Section B.3. As long as (B.12) is satisfied, (EG+) is guaranteed to converge for $\gamma_{k}=\nicefrac{{1}}{{L}}$. On the other hand, since (B.10) is a linear system, we simultaneously learn that picking $c$ any larger would imply non-convergence through $\max_{i}|\lambda_{i}|\geq 1$ (given $z^{0}\neq 0$). We can trivially embed problem (B.10) into a higher dimension to generalize the result. Noting that $c=-\rho L$ completes the proof. ∎

We provide Mathematica code to verify each step of the above proof.The supplementary code can be found at https://github.com/LIONS-EPFL/weak-minty-code/.

Appendix C Toy examples

In the following appendix, $L$ denotes the Lipschitz constant of $F$ restricted to the constraint set and $\rho$ is the parameter of the weak MVI (Item 3) when restricted to the constraint set. This restriction of the definitions is warranted, since $z^{k}$ remains within the constraint set in all simulations, while $\bar{z}^{k}$ is guaranteed to stay within by definition of 1.3 in Algorithm 1 (and likewise for all other considered method treating problem (2.1)).

All computer-assisted calculations can be found in the supplementary code.1

with $a,b\neq 0$. Transforming this dynamics into cartesian coordinates yields the desired vectorfield, $F$, while subsequently integrating with respect to $x$ and $y$ yields the two potentials associated with the two players. Note that the roots ${\mathopen{}\left\{-c_{i}{}\mid{}{}\right\}\mathclose{}}_{i=1}^{k}$ for the polynomial defining $\dot{r}$ are not strictly necessary for showing existence of limit cycles, but leads to a simpler form for $Fz$. We illustrate the construction in Fig. 5.

Let $Fz=(\dot{x},\dot{y})$ be the evolution in cartesian coordinates of the associated vectorfield in polar coordinates defined by (C.1). Then the only stationary point of $F$ is at the origin $(0,0)$ and there exists a limit cycle at $r=c_{i}$ for all $i\in[k]$.

Let $r=\sqrt{x^{2}+y^{2}}$. It is easy to see from (C.1) that the only stationary point is at $r=0$. By construction, $\dot{r}$ is a polynomial with roots $c_{i}$ for all $i\in[k]$, so any trajectory starting on the circle defined by $r=c_{i}$ remains in that set. However, $\dot{\theta}$ is strictly nonzero. As a consequence $Fz$ is nonzero, so $r=c_{i}$ must define a limit cycle, which proofs the claim. ∎

C.2 Proof for properties of Definition 1

This can easily be verified by a change of variables. From Proposition 1 it then follows, that there must exist a limit cycle at $\|z\|=1$ and $\|z\|=\nicefrac{{3}}{{4}}$. To verify the conditions on $\rho$ we compute the closed form solution to $\rho$ and $L$ in Mathematica:

For $a=1$ we have $\rho=-\frac{50176}{1050977}$ and $L=\frac{\sqrt{2538096\sqrt{704424929}+70246989617}}{20000}$

For $a=\nicefrac{{3}}{{4}}$ we have $\rho=-\frac{602112}{16798825}$ and $L=\frac{\sqrt{7614288\sqrt{6383574361}+635022906553}}{80000}$

For $a=\nicefrac{{1}}{{3}}$ we have $\rho=-\frac{150528}{9439585}$ and $L=\frac{\sqrt{2538096\sqrt{754424929}+73446989617}}{60000}$

It can easily be verified that the stated conditions for $\rho$ in Definition 1 are met for the values above. This completes the proof.

We provide Mathematica code verifying the construction of $F$ and the closed form solutions to $L$ and $\rho$.

C.3 Proof for properties of Definition 1

Under the definitions of $\rho$ and $L$ in Appendix C, we claim that the origin $(0,0)$ in (GlobalForsaken) is a global Nash equilibrium and satisfies Item 3 with $\rho>-\nicefrac{{1}}{{2L}}$.

To verify that $(0,0)$ is indeed a global Nash equilibrium we need to check that the solution cannot be unilaterally improved. In other words, the solution should coincide with $(x^{\star},y^{\star})$ where

We can easily verify this with Minimize in Mathematica, since the functions are polynomial for which a closed form solutions to the global optimization problem will be returned.

To find $\rho$ for $z^{\star}=(0,0)$ we solve the global minimization problem,

for which a closed form solution can be found with Mathematica, which when numerically evaluated is approximately $-0.119732$.

We need to compute $L$ to ensure $\rho>-\nicefrac{{1}}{{2L}}$. In our case of convex constraints, $\mathcal{C}$, we have that $L=\sup_{z\in\mathcal{C}}\|JF(z)\|$ where $\|\cdot\|$ denotes the spectral norm (Rockafellar & Wets, 2009, Thm. 9.2 and 9.7). Under our constraint $\|z\|_{\infty}\leq\nicefrac{{4}}{{3}}$, this can similarly be computed in closed form, yielding $L=\nicefrac{{\sqrt{\frac{1}{2}{\mathopen{}\left(9409\sqrt{59721901}+74125591\right)\mathclose{}}}}}{{2835}}$. So $-\frac{1}{2L}\approx-0.165432$ which satisfy the condition $\rho>-\frac{1}{2L}$. This completes the proof.

Let $F$ be the associated operator of $\phi$ in (GlobalForsaken) defined as $Fz=({\nabla}_{x}\phi(x,y),-{\nabla}_{y}\phi(x,y))$. Define the radius as $r=\|z\|$. Then, $Fz$ has a stable critical point at the origin $(0,0)$, at least one attracting limit cycle in the region defined by $\sqrt{\nicefrac{{3}}{{2}}}<r<2$ and at least one repellant limit cycle within $r\leq\sqrt{\nicefrac{{3}}{{2}}}$.

We follow a similar argument as in Hsieh et al. (2021, D.2). We can compute the associated operator $F$,

With a change of variables into polar coordinates $(r,\theta)$ we get that $r=\sqrt{x^{2}+y^{2}}$ evolves as,

When $r=\sqrt{\nicefrac{{3}}{{2}}}$ this reduces to $\dot{r}=\frac{3\cos(4\theta)+5}{56\sqrt{6}}$ and we observe that $\dot{r}>0$ for any $\theta$. Likewise for $r=2$, we have that $\dot{r}=-\frac{4}{21}(22\cos(4\theta)+25)$ which implies $\dot{r}<0$. Since there is no stationary point in the region $\mathcal{S}={\mathopen{}\left\{(r,\theta):\sqrt{\nicefrac{{3}}{{2}}}<r<2{}\mid{}{}\right\}\mathclose{}}$ it then follows from the Poincaré-Bendixson theorem (Teschl, 2012, Thm. 7.16) that there must exist at least one attracting limit cycle in $\mathcal{S}$. Further, it is easy to see that $(0,0)$ is a critical point and that it is stable by inspection of the Jacobian $JF(z)$. Since $\mathcal{S}$ is trapping, it follows from Poincaré–Hopf index theorem, that there must exist a repellant limit cycles in the region defined by $r<\sqrt{\nicefrac{{3}}{{2}}}$. This completes the proof. ∎

C.4 Proof of properties for (Hsieh et al., 2021, Example 5.2)

Example 6: (Hsieh et al., 2021, Example 5.2)

where $\psi(z)=\frac{1}{4}z^{2}-\frac{1}{2}z^{4}+\frac{1}{6}z^{6}$.

By using Mathematica, we can obtain a closed form solution of the Lipschitz constant $L$ of $F$ restricted to the constraint set, which we find to be $L=\frac{1}{80}\sqrt{\frac{1}{2}{\mathopen{}\left(1089\sqrt{801761}+993841\right)\mathclose{}}}$. Mathematica can solve approximately for the critical point, yielding $z^{\star}=(0.0780267,0.411934)$. To find $\rho$ we want to globally minimize $\rho(z):=\frac{{\mathopen{}\left\langle Fz,z-z^{\star}\right\rangle\mathclose{}}}{\|Fz\|^{2}}$ for $z\in\mathcal{D}$. Mathematica finds the candidate $z^{\prime}=(-1.01236,-0.104749)$ for which $\rho(z^{\prime})=-0.477761$. So $\rho$ must be at least this small, i.e. $\rho<-0.477761$. Since $-\nicefrac{{1}}{{2L}}\approx-0.04$, this implies that $\rho<-\nicefrac{{1}}{{2L}}$. See Forsaken.nb for Mathematica-assisted computations.

This rules out convergence guarantees for both (CEG+) and AdaptiveEG+ (Algorithm 1), which is supported by the simulation in Figure 8. However, as observed, (CurvatureEG+) converges in the simulations.