Stein's method of exchangeable pairs for the Beta distribution and generalizations

Introduction

Since its introduction in in 1972 Stein’s method has become a famous and useful tool for proving distributional convergence. One of its main advantages over other techniques is that it automatically yields concrete error bounds on various distributional distances. Being first only developed for normal approximation it was observed by several authors that Stein’s idea of linking a characterizing operator for the target distribution to a differential equation, the Stein equation, carries over to many other absolutely continuous and discrete distributions, where, in the discrete case, the differential equation has to be replaced by a suitable difference equation. Among those other distributions, to which Stein’s method has been successfully extended, are the Poisson distribution (see e.g. , or ), the Gamma distribution (see or ), the exponential distribution (see e.g. , and ), the Laplace distribution and, more generally, the class of Variance-Gamma distributions . Stein’s method for the Beta distribution has been developed independently in the paper as well as in the preprint .

Although in both works and a rate of convergence for the relative number of drawn red balls in a Pólya urn model was derived using Stein’s method for the Beta distribution, the actual approaches were quite different. In the authors developed a useful and widely applicable technique to find a whole class of characterizing operators for a discrete distribution, whose probability mass function is known explicitly, and compared one of these operators to the Stein operator of the limiting Beta distribution. In contrast, the preprint built on a coupling approach by developing a new version of the exchangeable pairs approach of Stein’s method for a rather large class of absolutely continuous distributions on the real line. This new version of the exchangeable pairs approach differs from that in the framework of the density method as developed in and , since it allows for a modification of the Stein equation, which is adapted to a given exchangeable pair and does not necessarily rely on the characterization by the density method. Recently, in , a nice generalization of the density method, which does not necessarily assume absolute continuity of the given distribution, was given and, as an application, it was shown, how the situation of the Pólya urn example from may be fitted into this framework.

The main purpose of the present paper is to give a more easily readable account of the method and ideas from by keeping the class of Beta distributions on $$ and the Pólya urn model as a running example. In addition, we derive new numerical bounds on the solution to the Stein equation for the Beta distribution and, for smooth test functions, also on its first order derivative. For Lipschitz-continuous test functions, these bounds complement those given in in the sense that they are neither uniformly worse nor uniformly better in the parameters of the Beta distribution. Furthermore, we use a new iterative procedure to obtain uniform bounds for derivatives of any order of the solution to the Beta Stein equation with sufficiently smooth right hand side. Incidentally, this is the first paper to give bounds on higher order derivatives of the solution to the Beta Stein equation. It should be mentioned that, generally, obtaining bounds on higher order derivatives of the solution to the Stein equation is quite a difficult problem, because the explicit representations of those derivatives become more and more complicated. Hence, to date bounds on higher order derivatives of the solution are still quite rare in Stein’s method. For instance, the paper obtains sharp bounds on higher order derivatives in the context of the normal and exponential distributions by exploiting very peculiar identities and facts about these distributions, which are not available for more general absolutely continuous distributions. Also, if one succeeds in deriving a tractable generator representation of the solution to the Stein equation as suggested in , one can usually use this form of the solution to obtain bounds on higher order derivatives. This has been used for the multivariate normal and for the Gamma distribution . However, in contrast to the bounds from , these bounds usually do not exhibit the smoothness property of the inverse of the corresponding Stein operator. In the case of the multivariate normal distribution with non-singular covariance matrix, one can combine the generator representation with a partial integration to obtain bounds on higher order derivatives, which demand one fewer order of smoothness from the test function than the bounds from . This has been accomplished independently in and . The recent paper combines bounds obtained from the generator representation with the iterative method from the present article in order to obtain new bounds on derivatives of arbitrary order of the solution to the Gamma Stein equation, whose dependence on the shape parameter of the Gamma distribution is superior to previous bounds.

We also indicate, how our iterative method can be applied to obtain bounds for the solution to a Stein equation for the exponential distribution, which are better than those previously obtained. We thus suggest that exploiting this iterative procedure can become a fruitful technique for a larger class of distributions.

The remainder of this paper is structured as follows: In Section 2 the general approach is motivated by means of a natural exchangeable pair in the context of the Pólya urn model and it is stressed by means of this example that the framework of exchangeable pairs within the density approach as developed in and is not always suitable and why one might want to use a different Stein characterization. Furthermore, our main application, Theorem 2.1, a quantitative distributional limit theorem for the relative number of drawn red balls is stated. Then, motivated by this example, in Section 3 a general version of Stein’s method for a large class of absolutely continuous distributions adapted to a given exchangeable pair is developed. In Section 4 the theory from Section 3 is specialized to the class of Beta distributions and Theorem 2.1 is proved. Finally, in Section 5 several proofs for statements from Sections 3 and 4 are given.

Acknowledgements

Most parts of the research which led to this article have been accomplished during the authors PhD studies and, hence, there is a certain overlap with the author’s PhD thesis , see also the unpublished paper . The author was supported by the DFG via SFB/TR 12 during this time. We also refer to Appendix A of for a version of de l’Hôpital’s rule which covers (locally) absolutely continuous functions and which is general enough to justify all invocations of this famous tool within this article. Finally, Appendix B of contains some identities about the Gibbs sampling procedure, which are generally useful in the exchangeable pairs version of Stein’s method and which will be used in the present paper. I am grateful to an anonymous referee whose detailed and valuable comments and suggestions helped me improve the presentation of my results.

The Pólya urn model and motivation of our general approach

or, with $a:=\frac{r}{c}$ and $b:=\frac{w}{c}$,

where, for a real number $x$ and a nonnegative intger $m$, we define the generalized binomial coefficient by

Here, $B(a,b)$ denotes the Euler Beta function $B(a,b)=\int_{0}^{1}x^{a-1}(1-x)^{b-1}dx$ which is related to the Gamma function $\Gamma(t)=\int_{0}^{\infty}x^{t-1}e^{-x}dx$ via

where the constants $C(\cdot,\cdot)$ are defined in (47) and (48) below and $\lVert h^{\prime\prime}\rVert_{\infty}$ denotes the minimum Lipschitz constant of $h^{\prime}$.

Recall that a pair $(X,X^{\prime})$ of random elements on a common probability space is called exchangeable, if

Representation (6) for $W$ suggests constructing another random variable $W^{\prime}$ such that $W$ and $W^{\prime}$ make up an exchangeable pair using a Gibbs sampling procedure. Noticing that also the random variables $X_{1},\dotsc,X_{n}$ are exchangeable, the construction of $W^{\prime}$ can be simplified to the following: Observe $X_{1}=x_{1},\dotsc,X_{n}=x_{n}$ and construct $X_{n}^{\prime}$ according to the distribution $\mathcal{L}(X_{n}|X_{1}=x_{1},\dotsc,X_{n-1}=x_{n-1})$. Then, letting

the pair $(W,W^{\prime})$ is exchangeable. Note that $\lvert W-W^{\prime}\rvert\leq\frac{1}{n}$ is small which suggests that the exchangeable pair $(W,W^{\prime})$ be beneficial for a Stein’s method approach to the proof of weak convergence of $\mathcal{L}(W_{n})$ to $Beta(a,b)$. From the exchangeable pairs approach within normal approximation (see e.g. , or ) and for non-normal approximation (see and ) we know that exchangeability of $(W,W^{\prime})$ is not enough to guarantee distributional closeness of $W$ and of $Z\sim Beta(a,b)$ but that a further regression property has to be satisfied.

The exchangeable pair $(W,W^{\prime})$ satisfies the regression property

where \gamma_{a,b}(x)=(a+b)\bigl{(}\frac{a}{a+b}-x\bigr{)} and $\lambda=\lambda_{n}=\frac{1}{n(a+b+n-1)}$.

We have $W^{\prime}-W=\frac{X_{n}^{\prime}}{n}-\frac{X_{n}}{n}$ and by exchangeability of $X_{1},\ldots,X_{n}$ it clearly holds that $E[X_{n}|W]=E[X_{n}|S_{n}]=\frac{1}{n}S_{n}=W$. Also, by the definition of $X_{n}^{\prime}$ and since $X_{n}^{\prime}$ only assumes the values and $1$ we have for any $x_{1},\ldots,x_{n-1}\in\{0,1\}$

Thus, since $\sigma(W)\subseteq\sigma(X_{1},\ldots,X_{n})$, we obtain

From the theory developed in and in we know that if a given exchangeable pair $(W,W^{\prime})$ satisfies a regression property of the form

where $\lambda>0$ is a typically small constant and $R$ is negligible in size, then $\mathcal{L}(W)$ can be approximated by the absolutely continuous distribution whose density has logarithmic derivative $\psi$, if and only if the following additional condition is satisfied: It must be the case that

which is often paraphrased as that the term on the left hand side in (9) must satisfy a law of large numbers in order for the approximation to be accurate. Comparing (8) to the statement of Proposition 2.2 we see that according to the theory from or the only possibility would be to approximate the distribution of $W$ by a distribution whose density has logarithmic derivative equal to (a constant multiple) of

for $x$ in the support of this density, which should be equal to $$in this case. Since the logarithmic derivative$\psi_{a,b}$of the density$p_{a,b}$of$Beta(a,b)$ is given by

we conclude by way of contradiction that the law of large numbers (9) cannot hold. Indeed, we will see in Proposition 2.3 below that that the term on the left hand side of (9) is close to the non-constant random quantity $W(1-W)$ rather than to the constant $1$. From Proposition 2.2 and some experience with the exchangeable pairs approach within Stein’s method we conclude that it would be desirable to have a Stein operator $L$ of the form

for the Beta distribution $Beta(a,b)$. Indeed, in Section 4 we will see that a random variable $Z\sim Beta(a,b)$ satisfies the Stein identity

for all $g$ in a suitable class of functions, i.e. we can let $\eta_{a,b}(x)=\eta(x)=x(1-x)$. Evidently, the Stein identity (12) was first found in and it was also used in . The statement of the following Proposition will make it possible to exploit the above constructed exchangeable pair $(W,W^{\prime})$ in connection with the Stein identity (12) in Section 4.

For the above constructed exchangeable pair $(W,W^{\prime})$ we have

From general facts about Gibbs sampling (see e.g. Appendix B in ) it is known that

Since $X_{n}^{2}=X_{n}$ we have from the proof of Proposition 2.2 that

where we have used $E[X_{n}|W]=W$ again. Finally, we compute

Putting pieces together, we eventually obtain

The last assertion easily follows from (2) and from $\lambda=\frac{1}{n(a+b+n-1)}$. ∎

One main aspect of the theoretical contribution of this article is to emphasize that it is no coincidence that

but that this is a natural replacement of condition (9) from the density approach to our class of Stein operators of the form (15) below. We end this motivational section by an abstraction of the ideas in the context of the Pólya urn model and the limiting Beta distribution above. Suppose we are given a sequence of random variables $W=W_{n}$ of which we know that, as $n\to\infty$, it converges in distribution to a random variable $Z$ with an absolutely continuous distribution and density $p$ with respect to the Lebesgue measure. We will also assume that $p$ itself is absolutely continuous (on each compact subinterval of its support $\overline{(a,b)}$, where $-\infty\leq a<b\leq\infty$ are extended real numbers). Suppose also that we can naturally construct a random variable $W^{\prime}$, a small random perturbation of $W$, such that $(W,W^{\prime})$ is an exchangeable pair, $\lvert W-W^{\prime}\rvert$ is small in a certain sense and that a regression property of the form

holds, where $\gamma$ is a certain function on the support of $\mathcal{L}(Z)$, $\lambda>0$ is constant and $R$ is a negligible remainder term. The goal is to compute a rate of convergence for the distributional convergence $W\rightarrow Z$ by Stein’s method of exchangeable pairs for $\mathcal{L}(Z)$. By the above reasoning it would be beneficial to have a characterizing Stein operator $L$ for $Z$ of the form

where $\eta$ is a function that still has to be found. One might suppose that, in order that $L$ characterizes $\mathcal{L}(Z)$, given the density $p$ of $Z$ and the function $\gamma$ the function $\eta$ is unique but we will see that this is only so up to a constant multiple of $p^{-1}$. Note that by exchangeability

where the first approximation is by the assumption that $R$ is of negligible order and the second is by the fact that $W$ converges to $Z$ in distribution. Hence, it is natural to assume from the outset that $E[\gamma(Z)]=0$. In particular, we should assume that $E\lvert\gamma(Z)\rvert<\infty$. A natural question is, given $p$ and $\gamma$, if there is a general formula for the function $\eta$. In the preprint the first order linear differential equation

was found by making, for a given test function $h$, the ansatz $g_{h}(x)=\alpha(x)f_{h}(x)$ for the solutions $g_{h}$ of the Stein equation

belonging to the operator (15) and $f_{h}$ of the Stein equation

corresponding to the density approach. Here, again $\psi$ denotes the logarithmic derivative of $p$. In this paper we follow a different, more direct reasoning. If $\eta$ is such that (15) is characterizing $\mathcal{L}(Z)$, then, for suitable functions $g$ by partial integration:

Thus, if we want this expression to equal

then from (2) we conclude that $\eta$ must satisfy (17). Of course, (17) can be solved by the method of variation of the constant and it turns out that

is a particular solution which even satisfies $(\eta p)(a+)=(\eta p)(b-)=0$ whenever $E[\gamma(Z)]=0$ and, hence, the boundary conditions

hold for each regular enough, say e.g. bounded, function $g$. Also note that every other solution to (17) has the form

Hence, in all these cases, using the density approach implicitly entails choosing $\eta_{\kappa}$ with $\kappa=p(a+)$. When developing the general theory in Section 3 we restrict ourselves to the solution $\eta$ given by (21), i.e to $\kappa=0$. We thus already mention at this point that the density approach for $p$ is included in the theory presented in Section 3 if and only if

However, at least if $\gamma(x)=c(E[Z]-x)$, it turns out that in many cases $\eta$ given by (21) has a neat analytical representation, e.g. it is given by a polynomial of degree at most $2$, whereas the choice $\kappa\not=0$ would introduce a complicated coefficient into (18) originating from the term $p(x)^{-1}$. For instance, if $Z\sim N(0,1)$ is standard normally distributed and $\gamma(x)=-x$, then (21) yields $\eta\equiv 1$, whereas the general expression is $\eta_{\kappa}(x)=1+\kappa e^{x^{2}/2}$, which is difficult to handle in practice. Furthermore, if $p$ is not bounded away from zero, then $\kappa\not=0$ gives an unbounded function $\eta_{\kappa}$, whereas $\eta$ given by (21) usually is bounded, at least if $a>-\infty$ and $b<\infty$ (see, e.g. Proposition 3.5 below). In the next section we will see that under certain mild conditions on the density $p$ of $Z$ and on the coefficient $\gamma$ which, of course, needs not originate from an exchangeable pair, the operator $L$ given by (15) is indeed characterizing $\mathcal{L}(Z)$ and prove bounds on the corresponding Stein equation (18) for suitable test functions $h$. Finally, we want to propose a strategy of how to proceed, if, contrarily to the above reasoning, we do not know the limiting density $p$ from the outset but are only given an exchangeable pair $(W,W^{\prime})$ such that (14) holds and also

and, hence, for $x\in J$ and $x_{0}\in J$ an arbitrary point, we have

Here, of course, $K=p(x_{0})\eta(x_{0})$ is the normalization constant. Formula (2) shows that $p$ is uniquely determined by $\gamma$ and $\eta$. Furthermore, in Theorem 3.22 we will give precise criteria for $\gamma$ and $p$ defined by (2) to satisfy

and for $\eta$ to satisfy (21) so that the results of the theory developed in Section 3 can in fact be applied. This, together with Proposition 3.19 and Remark 3.20 (iii), suggests the approximation of $\mathcal{L}(W)$ by the distribution with density $p$, if the exchangeable pair $(W,W^{\prime})$ satisfies (14) and (23). Note that this idea yields a certain extension of the methodology proposed in , where only Stein characterizations from the density approach are put to use.

The general approach

Motivated by Section 2 in this section we develop a general version of Stein’s method for a random variable $Z$ with an absolutely continuous distribution with respect to the Lebesgue measure. This version is useful for those distributions, which allow for a tractable first order linear Stein operator. This class covers many of the standard absolutely continuous distributions. However, it should not be left unmentioned that certain distributions, like the Laplace , the Variance-Gamma and the PRR distribution fall outside the scope of this approach, as they only possess a second order linear Stein operator with tractable coefficients.

For some extended real numbers $-\infty\leq a<b\leq\infty$ the density $p$ is positive and locally absolutely continuous on the interval $(a,b)$.

$\gamma$ is Borel-measurable and not identically equal to zero,

$\gamma$ is decreasing on $\overline{(a,b)}$,

$E\lvert\gamma(Z)\rvert=\int_{a}^{b}|\gamma(t)|p(t)dt<\infty$ and in fact $E[\gamma(Z)]=\int_{a}^{b}\gamma(t)p(t)dt=0$.

Henceforth, we will always assume that Conditions 3.1 and 3.2 are satisfied. Note that by Condition 3.2 there exists a point $x_{0}\in(a,b)$ such that

though it might not be unique. For definiteness, we choose

and by the positivity of $p$ on $(a,b)$ we can define the function $\eta$ on $(a,b)$ by

The following proposition lists some properties of the function $I$.

Under Conditions 3.1 and 3.2 the function $I$ has the following properties:

$I$ is locally absolutely continuous on $\overline{(a,b)}$.

$I$ is increasing on $\overline{(a,x_{0}]}$ and decreasing on $\overline{[x_{0},b)}$ and, hence, attains its global maximum at $x_{0}$.

Of course, (a) follows from the fundamental theorem of calculus for Lebesgue integration and the second part of (b) is immediate from item (iii) of Condition 3.2. Finally, (c) and the first part of (b) follow from the second part of (b) and (25). ∎

If $a>-\infty$ and/or $b<\infty$, then it is of interest to know under what circumstances it is possible to extend $\eta$ to a continuous function on $\overline{(a,b)}$ because we would like to have $\eta(W)$ make sense, even if $W$ assumes one of the boundary values $a$ and $b$ with positive probability. We will see that in most cases we indeed have $\eta(a+)=0$ or $\eta(b-)=0$ if $a>-\infty$ or if $b<\infty$, respectively. The following Mills ratio condition is satisfied by most absolutely continuous distributions and will in fact turn out to be equivalent to the asserted boundary behaviour of $\eta$. From now on, we will denote by $F$ the distribution function corresponding to the density $p$.

The density $p$ of $Z$ satisfies all the properties from Condition 3.1 and also the following:

If $a>-\infty$, then $\lim_{x\downarrow a}\frac{F(x)}{p(x)}=0$.

If $b<\infty$, then $\lim_{x\uparrow b}\frac{1-F(x)}{p(x)}=0$.

Assume that Conditions 3.1 and 3.2 hold for $p$ and $\gamma$, respectively. Then, the function $\eta$ vanishes at the finite end points of the support $\overline{(a,b)}$ of $\mathcal{L}(Z)$, i.e. $\eta(a+)=0$ whenever $a>-\infty$ and $\eta(b-)=0$ whenever $b<\infty$, if and only if Condition 3.4 is satisfied. Thus, in this case we can extend $\eta$ to a continuous function on $\overline{(a,b)}$ vanishing at the finite end points of this interval.

Not every density $p$ satisfies Condition 3.4 as is clarified by the following example.

Let $\delta_{n}\in(0,1)$, $n\geq 1$, be such that $\sum_{n\geq 1}\delta_{n}=1$ and define $x_{n}:=1-\sum_{j=1}^{n-1}\delta_{j}=\sum_{j=n}^{\infty}\delta_{j}$ and $I_{n}:=[x_{n+1},x_{n}]$, $n\geq 1$. Furthermore let $q$ be the unique continuous function, which is linear on each interval $I_{n}$ and such that $q(x_{2n})=\delta_{2n}^{2}$ and $q(x_{2n+1})=\delta_{2n}$ for $n\geq 1$ and $q(1):=\delta_{1}$. Define $p$ to be the probability density which is a constant multiple of $q$. Then, $p$ satisfies Condition 3.1 with $a=0$ and $b=1$ but Condition 3.4 does not hold: We have $\lim_{n\to\infty}x_{2n}=0$ but

Note that $p$ satisfies $\lim_{x\to 0}p(x)=0$, so that this does not only happen because $p(0+)$ might not exist.

The counterexample given in Example 3.6 is quite artificial. Indeed, the following proposition lists mild assumptions on the density $p$ which guarantee that Condition 3.4 is satisfied. In practice, at least one of these assumptions is usually met. In particular, note that by part (f) of Proposition 3.7 the Mills ratio limits from Condition 3.4 at finite boundary points $a$ or $b$ are always zero, whenever they exist.

Assume $a>-\infty$. In either of the following cases $\lim_{x\downarrow a}\frac{F(x)}{p(x)}=0$.

The density $p$ is bounded away from zero in a suitable neighbourhood of $a$.

We have $p(a+)=0$ and there is a $\delta>0$ such that $p$ is increasing on $(a,a+\delta)$.

We have $p(a+)=0$ and there is a $\delta>0$ such that $p$ is convex on $(a,a+\delta)$.

We have $p(a+)=0$ and there is a $\delta>0$ such that $p$ is concave on $(a,a+\delta)$.

The limit $\lim_{x\downarrow a}\frac{F(x)}{p(x)}$ exists.

Of course, similar conditions guarantee that $\lim_{x\uparrow b}\frac{1-F(x)}{p(x)}=0$ if $b<\infty$.

if $h$ has a right limit at $a$. Note that $\gamma$ has a right limit at $a$ since it is decreasing. Similarly,

If $a>-\infty$ and $h$ has a right limit at $a$, then $g_{h}$ can be extended continuously to $a$ by letting $\displaystyle g_{h}(a):=\frac{h(a+)-E[h(Z)]}{\gamma(a+)}$.

If $b<\infty$ and $h$ has a left limit at $b$, then $g_{h}$ can be extended continuously to $b$ by letting $\displaystyle g_{h}(b):=\frac{h(b-)-E[h(Z)]}{\gamma(b-)}$.

The success of Stein’s method within applications considerably depends on good bounds on the solutions $g_{h}$ and their lower order derivatives, generally uniformly over some given class of test functions $h$. The next step will be to prove such bounds. It has to be mentioned that we cannot expect to derive concrete good bounds in full generality, but that sometimes further conditions have to be imposed either on the density $p$ or on the coefficient $\gamma$. Nevertheless, we will derive bounds involving functional expressions which can be simplified, computed or further bounded a posteriori for concrete distributions. So our abstract viewpoint will pay off. Moreover, some of our general bounds will already be explicit. In what follows, we denote by $g_{h}$ the standard solution to Stein’s equation (18) on $\overline{(a,b)}$, implicitly assuming that $h$ satisfies the assumptions of Proposition 3.8. Furthermore, for a function $f$ we denote by $\lVert f\rVert_{\infty}$ its essential supremum norm on $\overline{(a,b)}$. Note that this implies for $f$ a Lipschitz-continuous function on $\overline{(a,b)}$ that $\lVert f^{\prime}\rVert_{\infty}$ is just its minimum Lipschitz constant. First we give bounds for bounded and measurable test functions $h$.

The proof is deferred to Section 5. The following corollary specializes this result to the case that $\gamma(x)=-c(x-E[Z])$ and that $\mathcal{L}(Z)$ is symmetric with respect to its mean $E[Z]$, i.e. $Z-E[Z]\stackrel{{\scriptstyle\mathcal{D}}}{{=}}E[Z]-Z$. Then, it is also clear that $m=E[Z]$.

In this case we clearly have $I(m)=\frac{c}{2}E[\lvert Z-m\rvert]$ which implies the result by Proposition 3.9. ∎

In the case that $Z\sim N(0,1)$ and $c=1$ this result specializes to the well known bound $\lVert g_{h}\rVert_{\infty}\leq\sqrt{\frac{\pi}{2}}\lVert h-E[h(Z)]\rVert_{\infty}$ (see or , e.g.).

In the statement of Proposition 3.9 it might suprise that there is no bound mentioned for $\lVert g_{h}^{\prime}\rVert_{\infty}$. This is because, in general, a bound of the form $\lVert g_{h}^{\prime}\rVert_{\infty}\leq C\lVert h-E[h(Z)]\rVert_{\infty}$ with a finite constant $C$ does not exist in this setup. For instance, for $z>0$ and $Z$ having the exponential distribution with mean one, consider the Stein equation

Identity (3.3) from shows that for $x>z$ the solution $g_{z}$ to (33) satisfies

proving that such a constant $C$ in general cannot exist. Note also that this is contrary to the density approach, where one usually has such a bound (see or ).

The Kolmogorov distance between a given random variable $W$ and $Z$ is induced by the class of test functions $h_{z}:=1_{(-\infty,z]}$, where $z\in(a,b)$. In this situation it is easy to verify that the standard solution $g_{z}:=g_{h_{z}}$ to (18) is given by

By using de l’Hôpital’s rule it is not hard to check that always $\sup_{z\in(a,b)}S(z)<\infty$. Furthermore, $g_{z}$ is Lipschitz-continuous and on $(a,b)\setminus\{z\}$ it is infinitely often continuously differentiable with

where the functions $H$ and $G$ are defined in Proposition 3.13. From the negative example of (a) we already know that, in general, there is no finite constant $C$ such that

Nevertheless, even in such a situation, one may use the uniform bound on $S$ and a $z$-dependent bound on $\lVert g_{z}^{\prime}\rVert_{\infty}$ as well as particular properties of $W$ to prove accurate bounds on the Kolmogorov distance. This was done in for the exponential distribution. Incidentally, in the case of the Beta distribution, the function $S$ will be bounded for a different purpose in the proof of Proposition 4.2.

Proposition 3.9 is already sufficient to prove that the operator $L$ given by (15) characterizes the distribution of $Z$. The proof is given in Section 5.

A random variable $X$ with values in $\overline{(a,b)}$ has the same distribution as $Z$ if and only if for each continuous function $f$ on $\overline{(a,b)}$, which is locally absoulutely continuous on $(a,b)$ and which satisfies $E\lvert\eta(Z)f^{\prime}(Z)\rvert=\int_{a}^{b}\lvert f^{\prime}(x)\rvert I(x)dx<\infty$ we have

In particular, in this case both expected values exist.

Next, we will turn to Lipschitz continuous test functions $h$. In contrast to bounded measurable test functions, there we will also be able to prove useful bounds for $g_{h}^{\prime}$. In order that $E[h(Z)]$ exists for Lipschitz continuous test functions $h$ we need to assume that $E\lvert Z\rvert<\infty$. The following two result, which are also proved in Section 5, include optimal bounds for both, $g_{h}$ and $g_{h}^{\prime}$, when $h$ is Lipschitz.

$\displaystyle\lvert g_{h}(x)\rvert\leq\lVert h^{\prime}\rVert_{\infty}\frac{F(x)E[Z]-\int_{a}^{x}yp(y)dy}{I(x)}=\lVert h^{\prime}\rVert_{\infty}\frac{\int_{a}^{x}(E[Z]-y)p(t)dt}{I(x)}\,$;

$\displaystyle\lvert g_{h}^{\prime}(x)\rvert\leq\lVert h^{\prime}\rVert_{\infty}\frac{\int_{a}^{x}F(s)dsG(x)+\int_{x}^{b}(1-F(s))dsH(x)}{p(x)\eta(x)^{2}}$.

Here, for $x\in\overline{(a,b)}$, the positive functions $H(x)$ and $G(x)$ are defined by

Moreover, these bounds are optimal among all bounds involving the factor $\lVert h^{\prime}\rVert_{\infty}$.

If $a>-\infty$ and $b<\infty$, then it follows by an application of de l’Hôpital’s rule that the function $S(x):=\frac{\int_{a}^{x}(E[Z]-y)p(t)dt}{I(x)}$ is bounded on $(a,b)$. Indeed, if $a>-\infty$, for instance, we have that

However, in general $S(x)$ is unbounded, if $\lvert\gamma(x)\rvert$ does not grow at least linearly with $x$. For instance, if $Z\sim N(0,1)$ and $\gamma(t)=-\operatorname{sign}(t)$, then we have for positive $x$ that

The bound for $\lvert g_{h}(x)\rvert$ in part (a) of Proposition 3.13 can be written as

where $\tau$ is the so-called Stein factor or Stein kernel of $Z$ given by

i.e. $\tau$ is the function $\eta$ which belongs to the choice $\gamma(x)=E[Z]-x$. The Stein kernel $\tau$ appeared first in Lecture $6$ of and it has turned out to be a fundamental object in Stein’s method for one-dimensional absolutely continuous distributions (see, e.g. , and ).

$\displaystyle\lVert g_{h}\rVert_{\infty}\leq\frac{\lVert h^{\prime}\rVert_{\infty}}{c}\,$;

$\displaystyle\lvert g_{h}^{\prime}(x)\rvert\leq\frac{2\lVert h^{\prime}\rVert_{\infty}}{c}\frac{H(x)G(x)}{\eta(x)^{2}p(x)}=2c\lVert h^{\prime}\rVert_{\infty}\frac{\int_{a}^{x}F(s)ds\int_{x}^{b}(1-F(t))dt}{\eta(x)^{2}p(x)}$.

In the case of the normal distribution (via its classical Stein equation) the bound given in Corollary 3.15 (a) reduces to $\lVert g_{h}\rVert_{\infty}\leq\lVert h^{\prime}\rVert_{\infty}$. Formally, this bound is a special instance of a general bound given in Lemma 3.1 of for the multivariate standard normal distribution (see also Lemma 2.6 in ). However, this lemma is stated under the additional assumption that $h$ has three bounded derivatives, which is stronger than being Lipschitz-continuous. Yet, as has been pointed out to me by the referee, one can use the generator representation of the solution to the Stein equation to obtain the same bound as in Corollary 3.15 (a) for once differentiable test functions $h$ with bounded first derivative by applying the well-known consequences of the dominated convergence theorem on differentiating under the integral sign. Then, using smoothing techniques, this result could be extended to the class of Lipschitz-continuous test functions, yielding an alternative proof of this bound. Nevertheless, in the context of Stein’s method for the univariate normal distribution, the best bound mentioned on $g_{h}$ for a Lipschitz test function $h$ is $\lVert g_{h}\rVert_{\infty}\leq 2\lVert h^{\prime}\rVert_{\infty}$ (see, e.g. or ). Hence, we believe that Corollary 3.15 (a) is the first result that rigorously proves the aforementioned bound, although, as described above, it can also be proved by means of existing techniques from the generator framework.

For concrete distributions the ratio appearing in the bounds for $g_{h}^{\prime}(x)$ may be bounded uniformly in $x$ by some constant which can sometimes also be computed explicitely. For instance, this is performed for the Beta distribution in Section 4. Furthermore, for the situation of Corollary 3.15, in the authors give mild conditions for the existence of a finite constant $k$ such that $\lVert g_{h}^{\prime}\rVert_{\infty}\leq k\lVert h^{\prime}\rVert_{\infty}$ for any Lipschitz-continuous $h$. In practice, these conditions are usually met. However, there is no hope of estimating the constant $k$ by their method of proof. Thus, for concrete distributions and explicit constants it might therefore by useful to work with our bounds from Corollary 3.15 (b) or from Proposition 3.13.

Now, we show how we can use the above results and the density formula (2) to give bounds on higher order derivatives of $g_{h}$, if $h$ itself is smooth enough. First note that the constant $K$ from (2) is given by

Formula (35) is a more general version of formula (3.14) in and is also derived in . Now, if the coefficient $\gamma$ is also absolutely continuous, by differentiating Stein’s equation (18), we obtain for $h$ Lipschitz

for the distribution $\operatorname{Exp}(\alpha)$, if $h$ is continuously differentiable on $[0,\infty)$ and both $h$ and $h^{\prime}$ are Lipschitz:

These bounds are better than those derived in and, additionally, since we do not have to assume that $h^{\prime}(0)=0$ for the bound on $\lVert g_{h}^{\prime\prime}\rVert_{\infty}$ to be valid, one term in the bounds of Theorems 1.1 and 1.2 from would drop off, if instead our bounds were used.

Next, we introduce the approach of exchangeable pairs satisfying the regression properties (14) and (23) in our general framework. As was observed in for the normal distribution, in case of univariate distributional approximations, one does not need the full strength of exchangeability, but equality in distribution of the random variables $W$ and $W^{\prime}$ is sufficient. This may allow for a greater choice of admissible couplings in several situations, or at least, relaxes the verification of asserted properties. Thus, let $W,W^{\prime}$ be real-valued random variables defined on the same probability space such that $W\stackrel{{\scriptstyle\mathcal{D}}}{{=}}W^{\prime}$. We will assume, that the random variables $W$ and $W^{\prime}$ only have values in an interval $(a,b)\subseteq J\subseteq\overline{(a,b)}$ where both functions $\eta$ and $\gamma$ are defined (recall that it might be the case that $\eta$ can only be defined on $(a,b)$). However, from Proposition 3.5 we know that we can let $J=\overline{(a,b)}$ if Condition 3.4 holds.

where $\lVert f^{\prime\prime}\rVert_{\infty}$ denotes the minimum Lipschitz constant of $f^{\prime}$.

The bound (3.19) can only be small, if $S$ and $R$ are of negligible order.

The proof shows, that Proposition 3.19 can easily be generalized to the situation, where there is a sub-$\sigma$-algebra $\mathcal{F}$ with $\sigma(W)\subseteq\mathcal{F}$ and the more general regression properties

hold for some $\mathcal{F}$-measurable remainder terms $R$ and $S$.

If $\mathcal{H}$ is some class of test functions, such that there are finite, positive constants $c_{0}$, $c_{1}$ and $c_{2}$ with $\lVert g_{h}\rVert_{\infty}\leq c_{0}$, $\lVert g_{h}^{\prime}\rVert_{\infty}\leq c_{1}$ and $\lVert g_{h}^{\prime\prime}\rVert_{\infty}\leq c_{2}$ for each $h\in\mathcal{H}$, then (3.19) immediately yields a bound on the distance

Finally, in our general framework, we readdress the last issue discussed in Section 2. Namely, we suppose that we are given two functions $\gamma$ and $\eta$, such that for some $-\infty\leq a<b\leq\infty$ the function $\gamma$ is defined on $\overline{(a,b)}$, $\eta$ is defined at least on $(a,b)$ and the following properties hold.

The function $\gamma$ is decreasing and such that $0<\gamma(a+)\leq\infty$ and $-\infty\leq\gamma(b-)<0$. Again, we define $x_{0}\in(a,b)$ by $x_{0}:=\sup\{x\in(a,b)\,:\,\gamma(x)>0\}$.

The function $\eta$ is positive and locally absolutely continuous on $(a,b)$.

The function $\gamma/\eta$ is locally integrable on $(a,b)$ and, if we define

Note that by definition we have $Q(x)\leq 0$ for all $x\in(a,b)$, if Condition 3.21 is satisfied. Now, we define the density $p$ by relation (2) with $K$ being a suitable normalizing constant. The existence of $K$ follows from the fact that, by Condition 3.21, for each $c\in(a,x_{0})$ there is a finite constant $L>0$ such that $L\gamma(x)\geq 1$ for each $x\in(a,c)$. Thus,

for each $d\in(x_{0},b)$. Hence, $p$ can be suitably normalized. Now, let $Z$ be a random variable with probability density function $p$. The next result is a generalization of Lemma 3, Lecture 6 in .

If Condition 3.21 is satisfied, then the density $p$ defined by (2) is such that

In particular, the theory developed in this section can be applied in this framework.

Thus, $E[\gamma(Z)]=0$. The second claim follows from

Stein’s method for the Beta distribution

In this section we specialize the theory from Section 3 to the family $Beta(a,b)$, $a,b>0$, of Beta distributions as defined in Section 2. Let us fix $a,b>0$ and from now on assume that $Z\sim Beta(a,b)$. Motivated by the Pólya urn example, the above constructed exchangeable pair $(W,W^{\prime})$ and by Proposition 2.2 we define the function $\gamma:=\gamma_{a,b}$ as in Proposition 2.2 and observe that

It is thus easy to see that $\gamma$ satisfies all assumptions of Condition 3.2 and also that the Beta density $p:=p_{a,b}$ given by (4) satisfies Conditions 3.1 and 3.4, the latter either directly or by Proposition 3.7. We claim that the function $\eta$ defined by (21) is given by

which easily follows from differentiating both sides of (43) and using (10). Thus, from Proposition 3.12 we immediately obtain the following Stein characterization for the Beta distribution. This result substantially extends Theorem 1 in in the case of the Beta distribution, which is weaker as it only characterizes the Beta distribution among the class of absolutely continuous distributions with finite second moment.

A random variable $X$ with values in $$has the distribution$Beta(a,b)$if and only if for each continuous function$f$on$$, which is locally absolutely continuous on$(0,1)$such that$E\lvert Z(1-Z)f^{\prime}(Z)\rvert<\infty$, we have

For the Beta distribution and a mesaurable function $h$ with $E\lvert h(Z)\rvert<\infty$, the Stein equation (18) is given by

and the standard solution (3) has the form

if $h$ has a right limit at and a left limit at $1$ by Proposition 3.8. We mention that the same Stein equation (44) has already been considered in , and in .

From Proposition 3.9 and Corollary 3.15 we can derive the following bounds for the solution (45) to (44). The proof is given in Section 5.

If $h$ is bounded, then $\displaystyle\lVert g_{h}\rVert_{\infty}\leq\frac{\lVert h-E[h(Z)]\rVert_{\infty}}{2m(1-m)p(m)}$, where $m$ is the median of $Beta(a,b)$.

If $h$ is Lipschitz, then $\displaystyle\lVert g_{h}\rVert_{\infty}\leq\frac{\lVert h^{\prime}\rVert_{\infty}}{a+b}$ and $\lVert g_{h}^{\prime}\rVert_{\infty}\leq C(a,b)\lVert h^{\prime}\rVert_{\infty}$, where $C(a,b)$ is given by (47) and (48).

If $h$ is continuously differentiable with Lipschitz derivative $h^{\prime}$, then $g_{h}^{\prime}$ is Lipschitz and $\displaystyle\lVert g_{h}^{\prime\prime}\rVert_{\infty}\leq C(a+1,b+1)\lVert h^{\prime\prime}\rVert_{\infty}+(a+b)C(a+1,b+1)C(a,b)\lVert h^{\prime}\rVert_{\infty}$.

More generally, if $m\geq 1$ is an integer and $h$ is at least $(m-1)$-times differentiable such that $h^{(j)}$ is Lipschitz-continuous for $j=0,\dotsc,m-1$, then $\lVert g_{h}^{(m-1)}\rVert_{\infty}$ is Lipschitz and

where we define an empty product to be equal to $1$.

It is worthwhile to compare our bound for $\lVert g_{h}^{\prime}\rVert_{\infty}$ from Proposition 4.2 (b) to the bound $\lVert g_{h}^{\prime}\rVert_{\infty}\leq(b_{0}+b_{1})\lVert h^{\prime}\rVert_{\infty}$ given in . One can show that if $a=b$, then our bound is uniformly better than theirs. However, if $a\not=b$, then there are regions for $(a,b)$ where our constant $C(a,b)$ is smaller and other ones, where their $b_{0}+b_{1}$ is smaller. For instance, if $0<a,b\leq 1$, then, again, $C(a,b)\leq b_{0}+b_{1}$. But, if $1<b<2$ is fixed and $a$ tends to zero, then $C(a,b)$ goes to infinity while their $b_{0}+b_{1}$ tends to $12$. In any case, neither our bound nor the bound from seem to be optimal for $\lVert g_{h}^{\prime}\rVert_{\infty}$.

Form Corollary 3.15 (b) we know that for Lipschitz $h$ and $x\in(0,1)$

By an application of de l’Hôpital’s rule, one can show that

We conjecture that if $\min(a,b)<1$, then

i.e. that $B$ assumes its maximum value at the boundary of $(0,1)$. However, if $\min(a,b)>1$, then we believe that there is always an $x_{1}\in(0,1)$ such that

If $a=b$, then the median of $Beta(a,a)$ equals $1/2$ and the bound in (a) has the explicit form $\lVert g_{h}\rVert_{\infty}\leq B(a,a)2^{a+b-1}\lVert h-E[h(Z)]\rVert_{\infty}$. Unfortunately, for $a\not=b$ there is no closed from expression for the median of $Beta(a,b)$. In such a case one could use known inequalities about the median $m$ in order to get bounds on $\lVert g_{h}\rVert_{\infty}$. Since one would have to distinguish several cases according to the values of $a$ and $b$ and, hence, to the shape of the density $p$, we omit the details, here.

From Proposition 3.19, Remark 3.20 (ii) and the bounds from Proposition 4.2 we obtain the following plug-in result, which bounds a certain distance to the Beta distribution by terms related to a given exchangeable pair.

Let $W$ and $W^{\prime}$ be identically distributed random variables on a common probability space $(\Omega,\mathcal{A},P)$ and let $\mathcal{F}\subseteq\mathcal{A}$ be a sub-$\sigma$-algebra of $\mathcal{A}$ such that $\sigma(W)\subseteq\mathcal{F}$ and

where the constants $C(\cdot,\cdot)$ are defined by (47) and (48).

Now we are in a position to prove Theorem 2.1.

The claim immediately follows from Theorem 4.4, Propositions 2.2, 2.3 and the fact that in this case

Proofs

Suppose, that $a>-\infty$ and choose $y\in(a,x_{0})$. Then $\gamma(y)>0$ and, by the nonnegativity of $I$ and the monotonicity of $\gamma$, for $a<x<y$ we have

so that $\lim_{x\downarrow a}\eta(x)=0$. Conversely, if $\eta(a+)=0$, then, again by (5),

The calculation for finite $b$ is similar by using the representation $I(x)=-\int_{x}^{b}\gamma(t)p(t)dt$ and is therefore omitted. ∎

That item (a) is sufficient is clear. If (b) holds, then the claim follows from the inequality

valid for $x\in(a,a+\delta)$. Under Condition (c) we obtain a continuous and convex function on $[a,a+\delta)$ by letting $p(a):=0$. Now, let $a<x<y<a+\delta$. Then, there exists a $\lambda\in(0,1)$ with $x=\lambda a+(1-\lambda)y$ and by convexity we have:

Thus, the assumptions of (b) are satisfied. If (d) holds, then again letting $p(a):=0$ we obtain a continuous and concave function on $[a,a+\delta)$. Thus, there exists a decreasing function $f$ on $[a,a+\delta)$ such that

If there was a sequence $(x_{n})_{n\geq 1}$ in $[a,a+\delta)$ such that $x_{n}\downarrow a$ and $f(x_{n})\leq 0$ for each $n\geq 1$, then for each $x\in(a,a+\delta)$ and large enough $n$ we would have

for all $x\in(a,a+r)$ and hence, by de l’Hôpital’s rule,

In order to prove (f) we show that always

if $p$ satisfies Condition 3.1. To show this, define the function $G(x):=\log F(x)$ for $x\in(a,b)$. Then, $G$ is increasing and continuously differentiable on $(a,b)$ and satisfies $G(a+)=-\infty$ and $G(b-)=0$. If (50) did not hold, then

Hence, choosing $\delta>0$ such that $G^{\prime}(x)\leq c+1$ for all $x\in(a,a+\delta]$ we would obtain

which would contradict $G(a+)=-\infty$. ∎

which exists in $[0,\infty)$ by Condition 3.2. Here, we used the convention $\frac{1}{\infty}=0$. Moreover,

again by Condition 3.2 and by Proposition 3.3. Furthermore, we have

for each $x\in(a,b)$ since by the positivity of $p$ and because $\gamma$ is decreasing

Hence, $M$ is increasing and, thus, for each $x\in(a,m]$ we have

The same bound can be proved for $x\in(m,b)$ by using the representation

and the fact that also $1-F(m)=\frac{1}{2}$. ∎

The following two lemmas, which are quite standard in Stein’s method, will be needed for the proof of Proposition 3.13. For proofs we refer to , for instance.

Suppose that $p$ satisfies Condition 3.1 and that $\int_{a}^{b}\lvert x\rvert p(x)dx<\infty$. Then, for each $x\in\overline{(a,b)}$ we have:

$\int_{a}^{x}F(t)dt=xF(x)-\int_{a}^{x}sp(s)ds\,$;

$\int_{x}^{b}(1-F(t))dt=\int_{x}^{\infty}sp(s)ds-x(1-F(x))$.

Suppose that $p$ satisfies Condition 3.1 and that $E\lvert Z\rvert=\int_{a}^{b}\lvert x\rvert p(x)dx<\infty$. Then, for each Lipschitz function $h$, the following assertions hold true:

For each $x\in\overline{(a,b)}$ we have $\int_{a}^{x}(h(y)-E[h(Z)])p(y)dy=-(1-F(x))\int_{a}^{x}F(s)h^{\prime}(s)ds-F(x)\int_{x}^{b}(1-F(s))h^{\prime}(s)ds$.

First, we prove (a). Recall the representation

By Lemmas 5.2 and 5.1 we thus obtain that

implying (a). Now, we turn to the proof of (b). By Stein’s equation (18) we obtain for $x\in(a,b)$

From (52) we already know that $H$ is nonnegative on $(a,b)$. Similarly we prove the nonnegativity of $G$ on $(a,b)$: Since $p$ is positive and $\gamma$ is decreasing, for $x$ in $(a,b)$ we have

which reduces to the bound asserted in (b). Optimality of the bound in (a) follows from choosing $h(x)=x$ and observing that the above inequalities are in fact equalities, in this case. To see that also the bound in (b) is optimal, for given $x\in(a,b)$ choose a $1$-Lipschitz function $h$ such that $h^{\prime}(s)=1$ for all $s\in(a,x)$ and $h^{\prime}(s)=-1$ for all $s\in(x,b)$. Then, from (5) and the nonnegativity of $H$ and $G$, we see that equality holds in (5). ∎

Claim (a) follows from Proposition 3.13 (a) and the observation that in this case we have

Part (b) follows from Proposition 3.13 (b) and Lemma 5.1 by observing that in this case

and, similarly, $G(x)=c\int_{x}^{b}(1-F(s))ds$. ∎

We first prove necessity. Let $f$ be given as in the proposition. First we show that $E\lvert\gamma(Z)f(Z)\rvert<\infty$. We have

Repeating essentially the same calculation without absolute value signs and using $E[\gamma(Z)]=0$ yields

To prove sufficiency it is clearly enough to show that

holds for each bounded and continuous function $h$. Let $g_{h}$ be the standard solution of the Stein equation (18) corresponding to $h$. Then, from Proposition 3.9 we know that $\lVert g_{h}\rVert_{\infty}<\infty$. Also, $g_{h}$ is continuous on $\overline{(a,b)}$ and continuously differentiable on each compact subinterval of $\overline{(a,b)}$. Furthermore, since $I(x)=\eta(x)p(x)$ and $g_{h}$ solves (18) we have

By the hypothesis of Proposition 3.12 we can thus conclude that

To show finiteness of the first integral in (56) note that since $\gamma$ is decreasing, by Fubini’s theorem

since $h$ is Lipschitz. Similarly, one shows that

Since $h^{\prime}$ is bounded, to show that the second integral in (56) is finite, it suffices to prove that

since $E\lvert\gamma(Z)\rvert<\infty$ and $E\lvert Z\gamma(Z)\rvert<\infty$ and similarly one shows that

Using $\eta p=I$, $I^{\prime}=\gamma p$ and $I(a+)=I(b-)=0$, from Fubini’s theorem we obtain that the left hand side of (59) equals

Similarly, using $\gamma(x_{0})=0$, the definition of $g_{h}$ in (3) and Fubini’s theorem again, we have that the right hand side of (59) equals

where we have used $E[\gamma(Z)]=0$ for the last equality. Thus, from (5) and (5) we conclude that (59) holds. Thus, the standard solution $f=f_{h_{2}}$ to (38) is well-defined and given by

Now, first suppose that $a>-\infty$. Since $g_{h}$ solves the Stein equation (18) we know that

Hence, from (63), (66) and (64) we conclude that $g_{h}^{\prime}=f$ is the standard solution to (38). Similarly, one obtains this result if $b<\infty$. Finally assume that $g_{h}^{\prime}$ is bounded. Since $\lim_{x\downarrow a}\eta(x)^{2}p(x)=0$ we conclude from (64) and (63) that

Hence, by distributional equality, we obtain

and $\int_{0}^{1}s(1-s)ds=\frac{1}{6}$ the bound (3.19) now easily follows from (5) and the properties of $f$. ∎