Last Iterate is Slower than Averaged Iterate in Smooth Convex-Concave Saddle Point Problems

Introduction

In this paper we study the following saddle-point problem:

where the function $f$ is smooth, convex in ${\mathbf{x}}$, and concave in ${\mathbf{y}}$. This problem is equivalent ([facchinei_finite-dimensional_2003]) to finding a global saddle point of the function $f$, i.e., a point $({\mathbf{x}}^{*},{\mathbf{y}}^{*})$ such that: