A new dynamical approach of Emden-Fowler equations and systems

Marie-Françoise Bidaut-Véron, Hector Giacomini

Introduction

where $Q,p,q,\delta,\mu,s,m,$ $a,b$ are real parameters, $Q,p,q\neq 1,$ and $\varepsilon_{1}=\pm 1,$ $\varepsilon_{2}=\pm 1.$ These problems are the subject of a very rich litterature, either in the case of source terms $(\varepsilon_{1}=\varepsilon_{2}=1)$ or absorption terms $(\varepsilon_{1}=\varepsilon_{2}=1)$ or mixed terms $(\varepsilon_{1}=-\varepsilon_{2}).$ In the sequel we are concerned by the radial solutions, except at Section 9 where the solutions may be nonradial.

In this article we we give a new way of studying the radial solutions. In Section 2 we reduce system $(G)$ to a quadratic autonomous system:

This system is of Kolmogorov type. The reduction is valid for equations and systems with source terms , absorption terms , or mixed terms . It is remarkable that in the new system, $p$ and $q$ appear only as simple coefficients, which allows to treat any value of the parameters, even $p$ or $q<1,$ and $s,m,\delta$ or $\mu<0$ .

In Section 3 we revisit the well-known scalar case (1.1), where $(G)$ becomes two-dimensional. We show that the phase plane of the system gives at the same time the behaviour of the two equations

which is a kind of unification of the two problems, with source terms or absorption terms. For the case of source term ( $\varepsilon_{1}=1),$ we find again the results of , , showing that the new dynamical approach is simple and does not need regularity results or energy functions. Moreover it gives a model for the study of system $(G)$ . Indeed if $p=q$ , $a=b$ and $\delta+s=\mu+m,$ system $(G)$ admits solutions of the form $(u,u),$ where $u$ is a solution of (1.1) with $Q=\delta+s.\vskip 6.0pt plus 2.0pt minus 2.0pt$

In the sequel of the article we study the case of source terms, i.e. $\left(G\right)=(S),$ where

This system has been studied by many authors, in particular the Hamiltonian problem $s=m=0,$ in the linear case $p=q=2,$ see for example , , , , , , and the potential system where $\delta=m+1$ , $\mu=s+1$ and $a=b,$ see , , ; the problem with general powers has been studied in , , , in the linear case and , , in the quasilinear case, see also , , .

Here we suppose that $\delta,\mu>0$ , so that the system is always coupled, $s,m\geqq 0,$ and we assume for simplicity

We say that a positive solution $(u,v)$ in $(0,R)$ is regular at if $u,v$ $\in C^{2}\left(0,R\right)\cap C(\left[0,R)\right)$ . Condition $\min(p+a,q+b)>0$ guaranties the existence of local regular solutions. Then $u,v\in C^{1}(\left[0,R)\right)$ . when $a,b>-1,$ and $u^{\prime}(0)=v^{\prime}(0)=0$ . The assumption $D>0$ is a classical condition of superlinearity for the system.

We are interessed in the existence or nonexistence of ground states, called G.S., that means global positive $(u,v)$ in $\left(0,\infty\right)$ and regular at $0.$ We exclude the case of ”trivial” solutions, $(u,v)=\left(0,C\right)$ or $\left(C,0\right),$ where $C$ is a constant, which can exist when $s>0$ or $m>0.$

In Section 4 we give a series of local existence or nonexistence results concerning system $(S)$ , which complete the nonexistence results found in the litterature. They are not based on the fixed point method, quite hard in general, see for example , . We make a dynamical analysis of the linearization of system $(M)$ near each fixed point, which appears to be performant, even for the regular solutions. For a better exposition, the proofs are given at Section 10.

In Section 5 we study the global existence of G.S. This problem has been often compared with the nonexistence of positive solutions of the Dirichlet problem in a ball, see , , , . Here we use a shooting method adapted to system $(M)$ , which allows to avoid questions of regularity of system $(S).$ We give a new way of comparison, and improve the former results:

(i) Assume $s<\frac{N(p-1)+p+pa}{N-p}$ and $m<\frac{N(q-1)+q+qb}{N-q}.$ If system $(S)$ has no G.S., then

(i) there exist regular radial solutions such that $X(T)=\frac{N-p}{p-1}$ and $Y(T)=\frac{N-q}{q-1}$ for some $T>0,$ with $0<X<\frac{N-p}{p-1}$ and $0<Y<\frac{N-q}{q-1}$ on $(-\infty,T).$

(ii) there exists a positive radial solution $(u,v)$ of the Dirichlet problem in a ball $B(0,R)$ .

This result is a key tool in the next Sections for proving the existence of a G.S. It gives also new existence results for the Dirichlet problem, see Corollary 5.3. We also give a complementary result:

Assume $s\geqq\frac{N(p-1)+p+pa}{N-p}$ and $m\geqq\frac{N(q-1)+q+qb}{N-q}.$ Then all the regular radial solutions are $G.S.$

In Section 6 we study the radial solutions of the well known Hamiltonian system

corresponding to $p=q=2<N,$ $s=m=0,$ $a>-2,$ which is variational. In the case $a=b=0,$ a main conjecture was made in :

System $(SH)$ with $a=b=0$ admits no (radial or nonradial) G.S. if and only if $(\delta,\mu)$ is under the hyperbola of equation

The question is still open; it was solved in the radial case in , , then partially in , , and up to the dimension $N=4$ in , see references therein. Here we find again and extend to the case $a,b\neq 0$ some results of relative to the G.S., with a shorter proof. We also give an existence result for the Dirichlet problem improving a result of .

Let $\mathcal{H}_{0}$ be the critical hyperbola in the plane $(\delta,\mu)$ defined by

(i) System $(SH)$ admits a (unique) radial G.S. if and only if ( $\delta,\mu)$ is above $\mathcal{H}_{0}$ or on $\mathcal{H}_{0}$ .

(ii) The radial Dirichlet problem in a ball has a solution if and only if ( $\delta,\mu)$ is under $\mathcal{H}_{0}$ .

(iii) On $\mathcal{H}_{0}$ the G.S. has the following behaviour at $\infty:$ assuming for example $\delta>\frac{N+a}{N-2},$ then $\lim_{r\rightarrow\infty}r^{N-2}u(r)=\alpha>0,$ and

Our proofs use a Pohozaev type function; in terms of the new variables $X,Y,Z,W$ , it contains a quadratic factor

As observed in () the G.S. can present a non-symmetric behaviour. This non-symmetry phenomena has to be taken in account for solving conjecture (1.3).

In Section 7 we consider the radial solutions of a nonvariational system:

where $p=q=2<N,$ $a=b>-2$ and $m=s>0.$ For small $s$ it appears as a perturbation of system $(SH).$ In the litterature very few results are known for such nonvariational systems. Our main result in this Section is a new result of existence of G.S. valid for any $s$ :

Consider the system $(SN),$ with $N>2,$ $a>-2.$ We define a curve $\mathcal{C}_{s}$ in the plane $(\delta,\mu)$ by

located under the hyperbola defined by (1.6). If $(\delta,\mu)$ is above $\mathcal{C}_{s},$ system $(SN)$ admits a G.S.

This result is obtained by constructing a new type of energy function which contains two terms in $X^{2},Y^{2}$ :

In Section 8 we consider the radial solutions of the potential system

where $\delta=m+1,\mu=s+1$ and $a=b,$ which is variational, see , . Using system ( $M)$ we deduce new results of existence:

Let $\mathcal{D}$ be the critical line in the plane $(m,s)$ defined by

(i) System $(SP)$ admits a radial G.S. if and only if ( $m,s)$ is above or on $\mathcal{D}.\vskip 6.0pt plus 2.0pt minus 2.0pt$

(ii) On $\mathcal{D}$ the G.S. has the following behaviour: suppose for example $q\leqq p.$ Let $\lambda^{\ast}=N+a-(s+1)\frac{N-p}{p-1}-m\frac{N-q}{q-1}.$ Then $\lim_{r\rightarrow\infty}r^{\frac{N-p}{p-1}}u(r)=\alpha>0,$ and

In particular (1.10) holds if $p=q,$ or $q\leqq m+1.$

(iii) The radial Dirichlet problem in a ball has a solution if and only if ( $m,s)$ is under $\mathcal{D}$ .

In that case we use the following energy function, which deserves to be compared with the one of Section 6 , since it has also a quadratic factor:

Finally in Section 9 we deduce a nonradial result for the potential system in the case of two Laplacians:

Our result proves a conjecture proposed in , showing that in the subcritical case there exists no G.S.:

then system $(SL)$ admits no (radial or nonradial) G.S.

Our proof uses the estimates of , which up to now are the only extensions of the results of to systems. It is based on the construction of a nonradial Pohozaev function extending the radial one given at (1.13) for $p=q=2$ , different from the energy function used in .

The case of the system $(G)$ with absorption terms ( $\varepsilon_{1}=\varepsilon_{2}=-1)$ or mixed terms ( $\varepsilon_{1}=-\varepsilon_{2}=1)$ , studied in , , will be the subject of a second article. Our approach also extends to a system with gradient terms,

Acknoledgment The authors are grateful to Raul Manasevich whose stimulating discussions encouraged us to study system $(G)$ .

Reduction to a quadratic system

Here we consider the radial positive solutions $r\mapsto(u(r),v(r))$ of system $(G)$ on any interval $\left(R_{1},R_{2}\right)$ , that means

Near any point $r$ where $u(r)\neq 0,u^{\prime}(r)\neq 0$ and $v(r)\neq 0,$ $v^{\prime}(r)\neq 0$ we define

This sytem is quadratic, and moreover a very simple one, of Kolmogorov type: it admits four invariant hyperplanes: $X=0,Y=0,Z=0,W=0$ . As a first consequence all the fixed points of the system are explicite. The trajectories located on these hyperplanes do not correspond to a solution of system $(G);$ they will be called nonadmissible.

We suppose that the discriminant of the system

Then one can express $u,v$ in terms of the new variables:

Since system $(M)$ is autonomous, each admissible trajectory $\mathcal{T}$ in the phase space corresponds to a solution $(u,v)$ of system $(G)$ unique up to a scaling: if $(u,v)$ is a solution, then for any $\theta>0$ , $r\mapsto(\theta^{\gamma}u(\theta r),\theta^{\xi}v(\theta r))$ is also a solution.

2 Fixed points of system (M)

System $(M)$ has at most $16$ fixed points. The main fixed point is

corresponding to the particular solutions

when they exist, depending on $\varepsilon_{1},\varepsilon_{2}.$ The values of $A$ and $B$ are given by

3 First comments

This formulation allows to treat more general systems with signed solutions by reducing the study on intervals where $u$ and $v$ are nonzero. Consider for example the problem

0n any interval where $uv>0,$ the couple $(\left|u\right|,\left|v\right|)$ is a solution of $(G).$ On any interval where $u>0>v,$ the couple $(u,\left|v\right|)$ satisfies $(G)$ with $(\varepsilon_{1},\varepsilon_{2})$ replaced by $(-\varepsilon_{1},-\varepsilon_{2}).$

There is another way for reducing the system to an autonomous form: setting

It extends the well-known transformation of Emden-Fowler in the scalar case when $p=2,$ used also in for general $p,$ see Section 3. When $p=q=2$ we obtain

which was extended to the nonradial case and used for Hamiltonian systems ( $s=m=0),$ with source terms in ( $\varepsilon_{1}=\varepsilon_{2}=1)$ and absorption terms in ( $\varepsilon_{1}=\varepsilon_{2}=-1)$ . Our system is more adequated for finding the possible behaviours: unlike system (2.8)it has no singularity, since it is polynomial, also its fixed points at $\infty$ are not concerned when we deal with solutions $u,v>0.$

It has been used in for studying the Hamiltonian system $(SH)$ . Even in that case we will show at Section 6 that system $(M)$ is more performant, because it is of Kolmogorov type.

Assume $p=q$ and $a=b.$ Setting $t=k\hat{t}$ and $\left(\hat{X},\hat{Y},\hat{Z},\hat{W}\right)=k(X,Y,Z,W)$ , we obtain a system of the same type with $N,a$ replaced by $\hat{N},\hat{a},$ with

From (2.3) and (2.4), we get $\hat{\gamma}/\gamma=\hat{\xi}/\xi=k=\frac{p+\hat{a}}{p+a}.$ There is one free parameter. In particular

1) we get a system without power ( $\hat{a}=0),$ by taking

2) we get a system in dimension $\hat{N}=1,$ by taking

The scalar case

We first study the signed solutions of two scalar equations with source or absorption:

with $\varepsilon=\pm 1,$ $1<p<N,$ $Q\neq p-1$ and $p+a>0.\vskip 6.0pt plus 2.0pt minus 2.0pt$

We cannot quote all the huge litterature concerning its solutions, supersolutions or subsolutions, from the first studies of Emden and Fowler for $p=2,$ recalled in ; see for example and , for any $p>1,$ and references therein. We set

From Remark 2.4 we could reduce the system to the case $a=0$ , in dimension $\hat{N}=p(N+a)/(p+a).$ However we do not make the reduction, because we are motivated by the study of system $(G)$ , and also by the nonradial case.

Near any point $r$ where $u(r)\neq 0$ (positive or negative), and $u^{\prime}(r)\neq 0$ setting

with $t=\ln r,$ we get a 2-dimensional system

and then $\left|u\right|\mathbf{=}r^{-\gamma}\mathbf{(}\left|Z\right|\left|X\right|^{p-1})^{1/(Q+1-p)}.$ This change of unknown was mentioned in in the case $p=2,\varepsilon=1$ and $N=3.$ It is remarkable that system $(M_{scal})$ is the same for the two cases $\varepsilon=\pm 1$ , the only difference is that $X(t)Z(t)$ has the sign of $\varepsilon:\vskip 6.0pt plus 2.0pt minus 2.0pt$

The equation with source $(\varepsilon=1)$ is associated to the 1st and $3^{rd}$ quadrant. It is well known that any local solution has a unique extension on $\left(0,\infty\right).$ The 1st quadrant corresponds to the intervals where $\left|u\right|$ is decreasing, which can be of the following types $\left(0,\infty\right),(0,R_{2})$ , $\left(R_{1},\infty\right)$ , $\left(R_{1},R_{2}\right)$ , $0<R_{1}<R_{2}<\infty.$ The $3^{rd}$ quadrant corresponds to the intervals $\left(R_{1},R_{2}\right)$ where $\left|u\right|$ is increasing.

The equation with absorption $(\varepsilon=-1)$ is associated to the $2^{nd}$ and $4^{th}$ quadrant. It is known that the solutions have at most one zero, and their maximal interval of existence can be $(0,R_{2}),(R_{1},\infty),\left(R_{1},R_{2}\right)$ or $(0,\infty).$ The $2^{nd}$ quadrant corresponds to the intervals $\left(R_{1},R_{2}\right)$ where $\left|u\right|$ is increasing. The $4^{th}$ quadrant corresponds to the intervals $(0,R_{2})$ or $\left(R_{1},\infty\right)$ where $\left|u\right|$ is decreasing.

In particular $M_{0}$ is in the 1st quadrant whenever $\gamma<\frac{N-p}{p-1},$ equivalently $Q>Q_{1},$ and in the $4^{th}$ quadrant whenever $Q<Q_{1}$ . It corresponds to the solution

where $A=\left(\varepsilon\gamma^{p-1}(N-p-\gamma(p-1))\right)^{1/(Q-p+1)}.$

2 Local study

We examine the fixed points, where for simplicity we suppose $Q\neq Q_{1},$ and we deduce local results for the two equations:

$\bullet$ Point $(0,0):$ it is a saddle point, and the only trajectories that converge to $(0,0)$ are the separatrix, contained in the lines $X=0,Y=0,$ they are not admissible.

$\bullet$ Point $N_{0}:$ it is a saddle point: the eigenvalues of the linearized system are $\frac{p}{p-1}$ and $-N$ . the trajectories ending at $N_{0}$ at $\infty$ are located on the set $Z=0,$ then there exists a unique trajectory starting from $-\infty$ at $N_{0}$ ; it corresponds to the local existence and uniqueness of regular solutions, which we obtain easily.

$\bullet$ Point $A_{0}:$ the eigenvalues of the linearized system are $\frac{N-p}{p-1}$ and $\frac{N-p}{p-1}(Q_{1}-Q)$ . If $Q<Q_{1},$ $A_{0}$ is an unstable node. There is an infinity of trajectories starting from $A_{0}$ at $-\infty;$ then $X(t)$ converges exponentially to $\frac{N-p}{p-1},$ thus $\lim_{r\rightarrow 0}$ $r^{\frac{N-p}{p-1}}u=\alpha>0.$ The corresponding solutions $u$ satisfy the equation with a Dirac mass at $0.$ There exists no solution converging to $A_{0}$ at $\infty.$ If $Q>Q_{1},$ $A_{0}$ is a saddle point; the trajectories starting from $A_{0}$ at $-\infty$ are not admissible; there is a trajectory converging at $\infty,$ and then $\lim_{r\rightarrow\infty}$ $r^{\frac{N-p}{p-1}}u=\alpha>0.$

$\bullet$ Point $M_{0}:$ the eigenvalues $\lambda_{1},\lambda_{2}$ of the linearized system are the roots of equation

For $\varepsilon=1$ , $M_{0}$ is defined for $Q>Q_{1};$ the eigenvalues are imaginary when $X_{0}=Z_{0},$ equivalently $\gamma=(N-p)/p,$ $Q=Q_{2}$ . When $Q<Q_{2},$ $M_{0}$ is a source, there exists an infinity of trajectories such that $\lim_{r\rightarrow 0}r^{\gamma}u=A$ . When $Q>Q_{2},$ $M_{0}$ is a sink, and there exists an infinity of trajectories such that $\lim_{r\rightarrow\infty}r^{\gamma}u=A$ . When $Q=Q_{2},$ $M_{0}$ is a center, from For $\varepsilon=-1,$ $M_{0}$ is defined for $Q<Q_{1},$ it is a saddle-point. There exist two trajectories $\mathcal{T}_{1},\mathcal{T}_{1}^{\prime}$ converging at $\infty,$ such that $\lim_{r\rightarrow\infty}r^{\gamma}u=A$ and two trajectories $\mathcal{T}_{2},$ $\mathcal{T}_{2}^{\prime},$ converging at $0,$ such that $\lim_{r\rightarrow 0}r^{\gamma}u=A.$

3 Global study

System $(M_{scal})$ has no limit cycle for $Q\neq Q_{2}$ . It is evident when $\varepsilon=-1.$ When $\varepsilon=1,$ as noticed in , it comes from the Dulac’s theorem: setting $X_{t}=f(X,Z),\quad Z_{t}=g(X,Z),$ and

then $M=KB$ with $K=(Q_{2}-Q)\gamma(N-p)/p,$ thus $M$ has no zero for $Q\neq Q_{2}.$

Then from the Poincaré-Bendixson theorem, any trajectory bounded near $\pm\infty$ converges to one of the fixed points. Thus we find again global results:

$\bullet$ Equation with source $\left(\varepsilon=1\right)$ . If $Q<Q_{1},$ there is no G.S.: the regular trajectory $\mathcal{T}$ issued from $N_{0}$ cannot converge to a fixed point. Then $X$ tends to $\infty$ and the regular solutions $u$ are changing sign, there is no G.S..

If $Q_{1}<Q<Q_{2},$ the regular trajectory $\mathcal{T}$ cannot converge to $M_{0};$ if it converges to $A_{0},$ it is the unique trajectory converging to $A_{0}$ ; the set delimitated by $\mathcal{T}$ and $X=0,Z=0$ is invariant, thus it contains $M_{0};$ and the trajectories issued from $M_{0}$ cannot converge to a fixed point, which is contradictory. then again $X$ tends to $\infty$ on $\mathcal{T}$ and the regular solutions $u$ are changing sign.. The trajectory ending at $A_{0}$ converges to $M_{0}$ at $-\infty;$ then there exist solutions $u>0$ such that $\lim_{r\rightarrow 0}r^{\gamma}u=A$ and $\lim_{r\rightarrow 0}$ $r^{\frac{N-p}{p-1}}u=\alpha>0.$

If $Q>Q_{2},$ the only singular solution at is $u_{0},$ and the regular solutions are G.S., with $\lim_{r\rightarrow\infty}r^{\gamma}u=A.$ Indeed $M_{0}$ is a sink; the trajectory ending at $A_{0}$ cannot converge to $N_{0}$ at $-\infty$ , thus $X$ converges to $0,$ and $Z$ converges to $\infty,$ then $u$ cannot be positive on $(0,\infty).$ The trajectory issued from $N_{0}$ converges to $M_{0}.$

$\bullet$ Equation with absorption $\left(\varepsilon=-1\right)$ . If $Q>Q_{1},$ all the solutions $u$ defined near are regular; indeed the trajectories cannot converge to a fixed point.

If $Q<Q_{1},$ we find again easily a well known result: there exists a positive solution $u_{1},$ unique up to a scaling, such that $\lim_{r\rightarrow 0}r^{\frac{N-p}{p-1}}u_{1}=\alpha>0,$ and $\lim_{r\rightarrow\infty}r^{\gamma}u_{1}=A.$ Indeed the eigenvalues at $M_{0}$ satisfy $\lambda_{1}<0<\lambda_{2}$ . There are two trajectories $\mathcal{T}_{1},\mathcal{T}_{1}^{\prime}$ associated to $\lambda_{1},$ and the eigenvector $(X_{0}+\left|\lambda_{1}\right|,-\frac{X_{0}}{p-1}).$ The trajectory $\mathcal{T}_{1}$ satisfies $X_{t}>0>Z_{t}$ near $\infty,$ and $X>\frac{N-p}{p-1},$ since $Z_{0}<0,$ and $X$ cannot take the value $\frac{N-p}{p-1}$ because at such a point $X_{t}<0;$ then $\frac{N-p}{p-1}<X<X_{0}$ and $X_{t}>0$ as long as it is defined; similarly $Z_{0}<Z<0$ and $Z_{t}<0;$ then $\mathcal{T}_{1}$ converge to a fixed point, necessarily $A_{0},$ showing the existence of $u_{1}.$ The trajectory $\mathcal{T}_{1}^{\prime}$ corresponds to solutions $u$ such that $\lim_{r\rightarrow\infty}r^{\gamma}u=A$ and $\lim_{r\rightarrow R}u=\infty$ for some $R>0.$ There are two trajectories $\mathcal{T}_{2},$ $\mathcal{T}_{2}^{\prime},$ associated to $\lambda_{2},$ defining solutions $u$ such that $\lim_{r\rightarrow 0}r^{\gamma}u=A$ and changing sign, or with a minimum point and $\lim_{r\rightarrow R}u=\infty$ for some $R>0.$ The regular trajectory starts from $N_{0}$ in the $2^{nd}$ quadrant, it cannot converge to a fixed point, then $\lim_{r\rightarrow R}u=\infty$ for some $R>0.$

$\bullet$ Critical case $Q=Q_{2}:$ it is remarkable that system $(M_{scal})$ admits another invariant line, namely $A_{0}N_{0},$ given by

It precisely corresponds to well-known solutions of the two equations

where $K^{2}=c^{Q-p+1}(N+a)^{-1}\left((N-p)/(p-1)\right)^{1-p}.$

The global results have been obtained without using energy functions. The study of was based on a reduction of type of Remark 2.2, using an energy function linked to the new unknown. Other energy functions are well-known, of Pohozaev type:

with $\sigma=\frac{N-p}{p},$ satisfying $\mathcal{F}_{\sigma}^{\prime}(r)=r^{N-1+a}\left(\frac{N+a}{Q+1}-\frac{N-p}{p}\right)\left|u\right|^{Q+1},$ or with $\sigma=\frac{N+a}{Q+1}$ , leading to $\mathcal{F}_{\sigma}^{\prime}(r)=r^{N-1}\left(\frac{N+a}{Q+1}-\frac{N-p}{p}\right)\left|u^{\prime}\right|^{p}.$ In the critical case $Q=Q_{2}$ , all these functions coincide and they are constant, in other words system $(M_{scal})$ has a first integral. We find again the line (3.3): the G.S. are the functions of energy $0.$

Local study of system (S)𝑆(S)

In all the sequel we study the system with source terms: $(G)=(S)$ . Assumption (1.5) is the most interesting case for studying the existence of the G.S.

We first study the local behaviour of nonnegative solutions $(u,v)$ defined near or near $\infty.$ It is well known that any solution $(u,v)$ positive on some interval $\left(0,R\right)$ satisfies $u^{\prime},v^{\prime}<0$ on $\left(0,R\right).$ Any solution $(u,v)$ positive on $(R,\infty),$ satisfies $u^{\prime},v^{\prime}<0$ near $\infty.$ We are reduced to study the system in the region $\mathcal{R}$ where $X,Y,Z,W>0,$ and consider the fixed points in $\mathcal{\bar{R}}.$ Then

and ( $X,Y,Z,W)$ is a solution of system $(M)$ in $\mathcal{R}$ if and only if $(u,v)$ defined by

is a positive solution with $u^{\prime},v^{\prime}<0.$ Among the fixed points, the point $M_{0}$ defined at (2.6) lies in $\mathcal{R}$ if and only if

The local study of the system near $M_{0}$ appears to be tricky, see Remark 4.2. A main difference with the scalar case is that there always exist a trajectory converging to $M_{0}$ at $\pm\infty:$

(Point $M_{0})$ Assume that (4.3) holds. Then there exist trajectories converging to $M_{0}$ as $r\rightarrow$ $\infty,$ and then solutions $(u,v)$ being defined near $\infty,$ such that

There exist trajectories converging to $M_{0}$ as $r\rightarrow 0$ , and thus solutions $(u,v)$ being defined near such that

The eigenvalues are the roots $\lambda_{1},\lambda_{2},\lambda_{3},\lambda_{4},$ of equation

From (1.5) we have $H>0,$ then $\lambda_{1}\lambda_{2}\lambda_{3}\lambda_{4}<0.$ There exist two real roots $\lambda_{3}<0<\lambda_{4},$ and two roots $\lambda_{1},\lambda_{2}$ , real with $\lambda_{1}\lambda_{2}>0$ , or complex. Therefore there exists at least one trajectory converging to $M_{0}$ at $\infty$ and another one at $-\infty.$ Then (4.4) and (4.5) follow from (4.2). Moreover the convergence is monotone for $X,Y,Z,W.$

There exist imaginary roots, namely $\operatorname{Re}\lambda_{1}=\operatorname{Re}\lambda_{2}=0,$ if and only if there exists $c>0$ such that $f(ci)=0,$ that means $Ec^{2}-G=0,$ and $c^{4}-Fc^{2}-H=0,$ equivalently

(i) either $Z_{0}=X_{0}$ and $W_{0}=Y_{0},$ i.e.

in other words $(\delta,\mu)=(\frac{q\left(N(p-1-s)+p(1-s+a\right))}{p(N-q)},\frac{p\left(N(q-1-m)+q(1-m+b\right))}{q(N-p)}).\vskip 6.0pt plus 2.0pt minus 2.0pt$

(ii) or $(p-1-s)(q-1-m)>0$ and $\left(\gamma,\xi\right)$ satisfies

This gives in general 0,1 or 2 values of $\left(\gamma,\xi\right)$ . For example, in the case $\frac{m}{q-1}=\frac{s}{p-1}\neq 1,$ and $(p-2)(q-2)>0$ and $N>\frac{pq-p-q}{p+q-2}$ we find another value, different from the one of (4.7) for $p\neq q:$

Moreover the computation shows that it can exist imaginary roots with $E,G\neq 0.$

In the case $p=q=2$ and $s=m$ the situation is interesting:

Assume $p=q=2$ and $s=m<\frac{N}{N-2},$ with $\delta+1-s>0,\mu+1-s>0.$ In the plane $(\delta,\mu),$ let $\mathcal{H}_{s}$ be the hyperbola of equation

equivalently $\gamma+\xi=N-2.$ Then $\mathcal{H}_{s}$ is contained in the set of points $(\delta,\mu)$ for which the linearized system at $M_{0}$ has imaginary roots, and equal when $s\leqq 1$ .

Proof. The assumption $D>0$ imply $\delta+1-s>0$ and $\mu+1-s>0$ ; condition $E=G=0$ implies $s<N/(N-2)$ and reduces to condition (4.10). Moreover if $s\leqq 1,$ all the cases are covered. Indeed $2G=(s-1)E\left[Y_{0}Z_{0}+X_{0}W_{0}\right],$ hence $GE\leqq 0.\vskip 6.0pt plus 2.0pt minus 2.0pt$

Next we give a summary of the local existence results obtained by linearization around the other fixed points of system $(M)$ proved in Section 10. Recall that $t\rightarrow-\infty$ as $r\rightarrow 0$ and $t\rightarrow\infty$ as $r\rightarrow\infty.$

(Point $N_{0})$ A solution $(u,v)$ is regular if and only if the corresponding trajectory converges to $N_{0}$ when $r\rightarrow 0$ . For any $u_{0},v_{0}>0,$ there exists a unique local regular solution $(u,v)$ with initial data $(u_{0},v_{0}).\vskip 6.0pt plus 2.0pt minus 2.0pt$

(Point $A_{0})$ If $s\frac{N-p}{p-1}+\delta\frac{N-q}{q-1}>N+a$ and $\mu\frac{N-p}{p-1}+m\frac{N-q}{q-1}>N+b,$ there exist (admissible) trajectories converging to $A_{0}$ when $r\rightarrow\infty$ . If $s\frac{N-p}{p-1}+\delta\frac{N-q}{q-1}<N+a$ and $\mu\frac{N-p}{p-1}+m\frac{N-q}{q-1}<N+b,$ the same happens when $r\rightarrow 0$ . In any case

If $s\frac{N-p}{p-1}+\delta\frac{N-q}{q-1}<N+a$ or $\mu\frac{N-p}{p-1}+m\frac{N-q}{q-1}<N+b,$ there exists no trajectory converging when $r\rightarrow\infty;$ if $s\frac{N-p}{p-1}+\delta\frac{N-q}{q-1}>N+a$ or $\mu\frac{N-p}{p-1}+m\frac{N-q}{q-1}>N+b,$ there exists no trajectory converging when $r\rightarrow 0.$

(Point $P_{0})$ 1) Assume that $q>m+1$ and $q+b<\frac{N-p}{p-1}\mu<N+b-m\frac{N-q}{q-1}.$ If $\gamma<\frac{N-p}{p-1}$ there exist trajectories converging to $P_{0}$ when $r\rightarrow\infty$ (and not when $r\rightarrow 0).$ If $\gamma>\frac{N-p}{p-1}$ the same happens when $r\rightarrow 0$ (and not when $r\rightarrow\infty).$

2) Assume that $q<m+1$ and $q+b>\frac{N-p}{p-1}\mu>N+b-m\frac{N-q}{q-1}$ and $q\frac{N-p}{p-1}\mu+m(N-q)\neq N(q-1)+(b+1)q.$ If $\gamma<\frac{N-p}{p-1}$ there exist trajectories converging to $P_{0}$ when $r\rightarrow 0$ (and not when $r\rightarrow\infty).$ If $\gamma>\frac{N-p}{p-1}$ there exist trajectories converging when $r\rightarrow$ $\infty$ (and not when when $r\rightarrow$ $0).$

In any case, setting $\kappa=\frac{1}{q-1-m}(\frac{N-p}{p-1}\mu-(q+b)),$ there holds

This result improves the results of existence obtained by the fixed point theorem in in the case of system $(RP)$ with $p=q=2,a=0,N=3,$ $2s+m\neq 3.$ The proof is quite simpler..

(Point $I_{0})$ If $\frac{N-p}{p-1}s>N+a$ and $\frac{N-q}{q-1}\mu>N+b,$ there exist trajectories converging to $I_{0}$ when $r\rightarrow\infty,$ and then

For any $s,m\geqq 0,$ there is no trajectory converging when $r\rightarrow$ $0.\vskip 6.0pt plus 2.0pt minus 2.0pt$

(Point $G_{0})$ Suppose $\frac{N-p}{p-1}\mu<N+b.$ If $q+b<\frac{N-p}{p-1}\mu$ and $N+a<\frac{N-p}{p-1}s,$ there exist trajectories converging to $G_{0}$ when $r\rightarrow$ $\infty.$ If $\frac{N-p}{p-1}\mu<q+b$ and $\frac{N-p}{p-1}s<N+a,$ the same happens when $r\rightarrow 0$ . In any case

(Point $C_{0})$ Suppose $N+b<\frac{N-q}{q-1}m$ (hence $q<m+1)$ with $m\neq\frac{N(q-1)+(b+1)q}{N-q},$ and $\delta>\frac{(N+a)(m+1-q)}{q+b}$ . Then there exist trajectories converging to $C_{0}$ when $r\rightarrow$ $\infty$ (and not when $r\rightarrow$ $0),$ and then

(Point $R_{0})$ Assume that $N+b<\frac{N-q}{q-1}m$ (hence $q<m+1)$ with $m\neq\frac{N(q-1)+b+bq}{N-q},$ and $\delta<\frac{(N+a)(m+1-q)}{q+b}.$ If $\frac{(p+a)(m+1-q)}{q+b}<\delta,$ there exist trajectories converging to $R_{0}$ when $r\rightarrow\infty$ (and not when $r\rightarrow 0)$ . If $\delta<\frac{(p+a)(m+1-q)}{q+b},$ there exist trajectories converging when $r\rightarrow 0$ (and not when $r\rightarrow\infty)$ , and then (4.15) holds again.

We obtain similar results of convergence to the points $Q_{0},J_{0},H_{0},D_{0},S_{0}$ by exchanging $p,\delta,s,a$ and $q,\mu,m,b$ . There is no admissible trajectory converginf to $0,K_{0},L_{0},$ see Remark 10.1.

Global results for system (S)𝑆(S)

We are concerned by the existence of global positive solutions. First we find again easily some known results by using our dynamical approach.

Assume that system $(S)$ admits a positive solution $(u,v)$ in $(0,\infty).$ Then the corresponding solution $(X,Y,Z,W)$ of system $(M)$ stays in the box

where $C_{1}=(N+a)(\frac{N-p}{p-1})^{p-1},C_{2}=(N+b)(\frac{N-q}{q-1})^{q-1},$ and

As a consequence if $s\leqq p-1$ or $m\leqq q-1,$ we have

with $K_{1}=C_{1}^{(q-1-m)/D}C_{2}^{\delta/D},K_{2}=C_{1}^{\mu/D}C_{2}^{(p-1-s)/D}.$

To any $(x,y)\in B(0,\rho)\backslash\left\{0\right\}$ we associate the unique trajectory $\mathcal{T}_{x,y}$ in $\mathcal{V}_{u}$ going through this point. If $T^{\ast}$ is the maximal interval of existence of a solution on $\mathcal{T}_{x,y}$ , then $\lim_{t\rightarrow T^{\ast}}(X(t)+Y(t))=\infty.$ Indeed $Z,$ and $W$ satisfy $0<Z<N+a,$ $0<W<N+b$ as long as the solution exists, because at a time $T$ where $Z(T)=N+a,$ we have $Z_{t}<0.$ If there exists a first time $T$ such that $X(T)=\frac{N-p}{p-1}$ or $Y(T)=\frac{N-q}{q-1},$ then $T<T^{\ast}.$ We consider the open rectangle $\mathcal{N}$ of submits

Let $\mathcal{U}=\left\{(x,y)\in B(0,\rho):x,y>0\right\}$ ; then $\mathcal{U=S}_{1}\cup\mathcal{S}_{2}\cup\mathcal{S}_{3}\cup\mathcal{S},$ where

Any element of $\mathcal{S}$ defines a G.S. Assume $s<\frac{N(p-1)+p+pa}{N-p}.$ Let us show that $\mathcal{S}_{1}$ is nonempty. Consider the trajectory $\mathcal{T}_{\bar{x},0}$ on $\mathcal{V}_{u}$ associated to $(\bar{x},0),$ with $\bar{x}\in\left(0,\rho\right),$ going through $\bar{M}=(\bar{x},0,$ $\varphi(\bar{x},0),\psi(\bar{x},0));$ it is not admissible for our problem, since it is in the hyperplane $Y=0$ : it satisfies the system

which is not completely coupled. The two equations in $X,Z$ corresponds to the equation

The regular solutions of (5.6) are changing sign, since $s$ is subcritical, see Section 3. Consider the solution $(\bar{X},\bar{Y},\bar{Z},\bar{W})$ of system $(M),$ of trajectory $\mathcal{T}_{\bar{x},0}$ , going through $\bar{M}$ at time $0;$ it satisfies $\bar{Y}=0,$ and $\bar{X}(t)>0,$ $\bar{Z}(t)>0$ tend to $\infty$ in finite time $T^{\ast}$ , then for any given $C\geqq\frac{N-p}{p-1},$ there exist a first time $T<T^{\ast}$ such that $\bar{X}(T)=C$ , and $\bar{Y}(T)=0$ . We have $\lim_{t\rightarrow-\infty}\bar{W}=N+b,$ and necessarily $0<\bar{W}<N+b,$ in particular $0<\bar{W}(T)<N+b;$ and $\bar{W}_{t}$ is bounded on $\left(-\infty,T^{\ast}\right),$ then $\bar{W}$ has a finite limit at $T^{\ast}.\$ The field at time $T$ is transverse to the hyperplane $X=\frac{N-p}{p-1}$ : we have $\bar{X}_{t}\geqq C\frac{Z(T)}{p-1}>0,$ since $\bar{Z}(T)>0$ . From the continuous dependance of the initial data at time $0,$ for any $\varepsilon>0,$ there exists $\eta>0$ such that for any $(x,y)\in B((\bar{x},0),\eta)$ and for any $(X,Y,Z,W)$ on $\mathcal{T}_{x,y},$ there exists a first time $T_{\varepsilon}$ such that $X(T_{\varepsilon})=C,$ and $\left|Y(t)\right|\leqq\varepsilon$ for any $t\leqq T_{\varepsilon}$ , in particular for any $(x,y)$ $\in B((\bar{x},0),\eta)$ with $y>0,$ and then $0<Y(t)\leqq\varepsilon$ for any $t\leqq T_{\varepsilon}.$ Let us take $C=\frac{N-p}{p-1}.$ Then $(x,y)\in$ $\mathcal{S}_{1}$ . The same arguments imply that $\mathcal{S}_{1}$ is open. Similarly assuming $m<\frac{N(q-1)+q+qb}{N-q}$ implies that $\mathcal{S}_{2}$ is nonempty and open. By connexity $\mathcal{S}$ is empty if and only if $\mathcal{S}_{3}$ is nonempty.

(ii) Here the difficulty is due to the fact that the zeros of $u,v$ correspond to infinite limits for $X,Y,$ and then the argument of continuous dependance is no more available. We can write $\mathcal{U=M}_{1}\cup\mathcal{M}_{2}\cup\mathcal{M}_{3}\cup\mathcal{S},$ where

In other words, $\mathcal{M}_{1}$ is the set of $(x,y)\in\mathcal{U}$ such that for any $(X,Y,Z,W)$ on $\mathcal{T}_{x,y},$ there exists a $T^{\ast}$ such that $\lim_{t\rightarrow T^{\ast}}X(t))=\infty,$ and $Y(t)$ stays bounded on $\left(-\infty,T^{\ast}\right),$ that means the set of $(x,y)\in\mathcal{U}$ such that for any solution $(u,v)$ corresponding to $\mathcal{T}_{x,y},$ $u$ vanishes before $v;$ similarly for $\mathcal{M}_{2}.$ Otherwise $\mathcal{M}_{3}$ is the set of $(x,y)\in\mathcal{U}$ such that there exists a $T^{\ast}$ such that $\lim_{t\rightarrow T^{\ast}}X(t)=\lim_{t\rightarrow T^{\ast}}Y(t)=\infty,$ that means $(u,v)$ vanish at the same $R^{\ast}=e^{T^{\ast}}$ . In that case, from the Höpf Lemma, $\lim_{r\rightarrow R}\frac{u^{\prime}}{(r-R)u}=1,$ then $\lim_{t\rightarrow T^{\ast}}\frac{X}{Y}=1.$

We are lead to show that $\mathcal{M}_{1}$ is nonempty and open for $s<\frac{N(p-1)+p+pa}{N-p}$ . We consider again the trajectory $\mathcal{\bar{T}}$ and take $C$ large enough: $C=2(\frac{N-p}{p-1}+\frac{N+\left|b\right|}{q-1}).$ Let $\varepsilon\in\left(0,\frac{C}{2}\right).$ For any $(x,y)\in B((\bar{x},0),\eta)$ with $y>0,$ and any $(X,Y,Z,W)$ on $\mathcal{T}_{x,y},$ there is a first time $T_{\varepsilon}$ such that $X(T_{\varepsilon})=C,$ and $0<Y(t)\leqq\varepsilon$ for any $t\leqq T_{\varepsilon}.$ And $X$ is increasing and $X_{t}\geqq X(X-C),$ thus there exists $T^{\ast}$ such that $\lim_{t\rightarrow T^{\ast}}X(t)=\infty.$ Setting $\varphi=X/Y,$ we find

then $\varphi_{t}(T_{\varepsilon})>0$ . Let $\theta=\sup\left\{t>T_{\varepsilon}:\varphi_{t}>0\right\};$ suppose that $\theta$ is finite; then $\varphi(\theta)>\varphi\left(T_{\varepsilon}\right)=C/\varepsilon>2$ and $X\left(\theta\right)\leqq Y\left(\theta\right)+C<X\left(\theta\right)/2+C,$ which is contradictory. Then $\varphi$ is increasing up to $T^{\ast};$ if $\lim_{t\rightarrow T^{\ast}}Y(t)=\infty,$ then $\lim_{t\rightarrow T^{\ast}}\varphi=1,$ which is impossible. Then $(x,y)\in\mathcal{M}_{1},$ thus $\mathcal{M}_{1}$ is nonempty. In the same way $\mathcal{M}_{1}$ is open. Indeed for any $(\bar{x},\bar{y})\in\mathcal{M}_{1}$ there exists $M>0$ such that $0<\bar{Y}(t)\leqq M/2$ on $\mathcal{T}_{\bar{x},\bar{y}}$ . To conclude we argue as above, with $(\bar{x},0)$ replaced by $(\bar{x},\bar{y}),$ and $C$ replaced by $C+M.$

Proof of Proposition 1.2. Assume $s\geqq\frac{N(p-1)+p+pa}{N-p}.$ Consider the Pohozaev type function

From our assumption, $\mathcal{F}$ is decreasing, and $Z>0,$ thus $X<\frac{N-p}{p-1}.$ Then $\mathcal{S}_{1},\mathcal{S}_{3}$ are empty. If moreover $m\geqq\frac{N(q-1)+q+qb}{N-q}$ then $\mathcal{S}_{2}$ is empty, therefore $\mathcal{S}=\mathcal{U}.$

Let us only assume that $s\geqq\frac{N(p-1)+p+pa}{N-p}.$ If one function has a first zero, it is $v$ . Indeed if there exists a first value $R$ where $u(R)=0,$ and $v(r)>0$ on $\left[0,R\right),$ then $\mathcal{F}(R)=\frac{R^{N}}{p^{\prime}}\left|u^{\prime}(R)\right|^{p}>0.$

As a first consequence we obtain existence results for the Dirichlet problem. It solves an open problem in the case $s>p-1$ or $m>q-1,$ and extends some former results of and . Our proof, based on the shooting method differs from the proof of , based on degree theory and blow-up technique. Our results extend the ones of [3, Theorem 2.2] relative to the case $p=q=2,$ obtained by studying the equation satisfied by a suitable function of $u,v.$

system $(S)$ admits no G.S. and then there is a radial solution of the Dirichlet problem in a ball in any of the following cases:

(i) $p<s+1,q<m+1,$ and $\min(s\frac{N-p}{p-1}+\frac{N-q}{q-1}\delta-(N+a),\frac{N-p}{p-1}\mu+m\frac{N-q}{q-1}-(N+b))\leqq 0;$

(ii) $p<s+1,$ $q>m+1$ and $s\frac{N-p}{p-1}+\frac{N-q}{q-1}\delta-(N+a)\leqq 0$ or $\gamma-\frac{N-p}{p-1}>0;$

(iii) $p>s+1,q>m+1$ and $\max(\gamma-\frac{N-p}{p-1},\xi-\frac{N-q}{q-1})\geqq 0;$

(iv) $p\geqq s+1,q\geqq m+1$ and $\max(\gamma-\frac{N-p}{p-1},\xi-\frac{N-q}{q-1})>0.$

Proof. From Theorem 1.1, we are reduced to prove the nonexistence of G.S.

(i) Assume $p<s+1,$ and $s\frac{N-p}{p-1}+\frac{N-q}{q-1}\delta-(N+a)<0.$ We have $-\Delta_{p}u\geqq Cr^{a-\frac{N-q}{q-1}\delta}u^{s}$ for large $r.$ From [6, Theorem 3.1], we find $u=O(r^{-(p+a-\frac{N-q}{q-1}\delta)/(s+1-p)})$ , and then $s\frac{N-p}{p-1}+\frac{N-q}{q-1}\delta-(N+a)\geqq 0,$ from (5.4), which contradicts our assumption. In case of equality, we find $-\Delta_{p}u\geqq Cr^{-N}$ for large $r,$ which is impossible. Then there exists no G.S. This improves ythe result of where the minimum is replaced by a maximum.

(ii) Assume $p<s+1,$ $q>m+1$ and $\gamma-\frac{N-p}{p-1}>0;$ then $u=O(r^{-\gamma}),$ which contradicts (5.4). If $\gamma-\frac{N-p}{p-1}=0,$ then $\lim r^{\frac{N-p}{p-1}}u=\alpha>0,$ and $\xi>\frac{N-q}{q-1}.$ Hence $-\Delta_{q}v\geqq Cr^{b-\frac{N-p}{p-1}\mu}v^{m}$ for large $r,$ then $v\geqq Cr^{(q+b-\frac{N-p}{p-1}\delta)/(q-1-m)}=Cr^{-\xi}.$ There exists $C_{1}>0$ such that $C_{1}\leqq$ $r^{\xi}v\leqq 2C_{1}$ for large $r,$ from [6, Theorem 3.1] and (5.5), then $-\Delta_{p}u\geqq Cr^{-N}$ for some $C>0,$ which is again contradictory.

(iii) (iv) The nonexistence of G.S is obtained by extension of the proof of to the case $a,b\neq 0.$ Moreover (iii) implies the nonexistence of positive solution $(u,v)$ , radial or not, in any exterior domain $(R,\infty)\times(R,\infty),R>0$ from .

Assume (4.3) with $p=q=2.$ If $\delta+s\geqq\frac{N+2+2a}{N-2}$ and $\mu+m\geqq\frac{N+2+2b}{N-2},$ then system $(S)$ admits a G.S.

Proof. It was shown in , by the moving spheres method that the Dirichlet problem has no radial or nonradial solution. Then Theorem 1.1 applies again.

We aso extend and improve a result of nonexistence of for the case $p=q=2,a=0,s>1$ :

Assume $s+1>p$ or $\gamma>\frac{N-p}{p},$ and

Then system $(S)$ admits no $G.S.$ and then there is a solution of the Dirichlet problem. The same happens by exchanging $p,s,\delta,a,\gamma$ with $q,m,\mu,b,\xi.$

Proof. Consider the function $\mathcal{F}$ defined at (5.7). Suppose that there exists a G.S. Then from (5.1) and (5.9) we find

From (5.8), we deduce that $\mathcal{F}$ is nondecreasing. First suppose $s+1>p.$ From (5.3) and (5.4),it follows that $u=O(r^{-k})$ at $\infty,$ with $k=(p+a-\delta\frac{N-q}{q-1})/(s-p+1).$ In turn $r^{N-p}u^{p}=O(r^{(N-p)-kp})=o(1)$ from (5.9), then $\mathcal{F}(r)=o(1)$ near $\infty.$ Next assume $s+1\leqq p$ and $\gamma>\frac{N-p}{p}$ . Then $r^{N-p}u^{p}=O(r^{N-p-\gamma p}),$ hence $\mathcal{F}(r)=o(1)$ near $\infty$ . In any case we get a contradiction.

The Hamiltonian system

where $p=q=2<N,$ $s=m=0,$ $a>b>-2,$ and $D=\delta\mu-1>0.$ For this case we find

The particular solution ( $u_{0}(r),v_{0}(r))=(Ar^{-\gamma},Br^{-\xi})$ exists for $0<\gamma<N-2,$ $0<\xi<N-2.$ Here $X,Y,Z,W$ are defined by

This system has a Pohozaev type function, well known at least in the case $a=b=0$ , given at (1.7):

It can also be found by a direct computation, and $\mathcal{E}_{H}$ satisfies

We define the critical case as the case where $(\delta,\mu)$ lie on the hyperbola $\mathcal{H}_{0}$ given by

In this case $\gamma=$ $\frac{N+b}{\mu+1},\xi=\frac{N+a}{\delta+1},$ and $\mathcal{E}_{H}^{\prime}(r)\equiv 0$ . It corresponds to the existence of a first integral of system $(M),$ which can also be expressed in the variables U $=r^{\gamma}u,$ V $=r^{\xi}v$ of Remark 2.2:

The supercritical case is defined as the case where $(\delta,\mu)$ is above $\mathcal{H},$ equivalently $\gamma+\xi<N-2$ and the subcritical case corresponds to $(\delta,\mu)$ under $\mathcal{H}$ .

The energy $\mathcal{E}_{H,0}$ of the particular solution associated to $M_{0}$ is always negative, given by $\mathcal{E}_{H,0}=-\frac{D}{\left(\mu+1\right)(\delta+1)}r^{N-2-\gamma-\xi}X_{0}Y_{0}(Z_{0}X_{0})^{(\mu+1)/D}(W_{0}Y_{0})^{(\delta+1)/D}.$

Next consider the critical and supercritical cases. When $a=b=0,$ there exists no solution if $\Omega$ is starshaped, see . Here we show the existence of G.S. for general $a,b$ . The existence in the critical case with $a=b=0$ was first obtained in , then in the supercritical case in , and uniqueness was proved in , . The proofs of are quite long due to regularity problems, when $\delta$ or $\mu<1,$ which play no role in our quadratic system.

The particular case $\delta=\mu$ and $a=b$ is easy to treat. Indeed in that case $u=v$ is a solution of the scalar equation $\Delta u+\left|x\right|^{a}\left|u\right|^{\delta-1}u=0,$ for which the critical case is given by $\delta=(N+2+2a)/(N-2).$ Moreover if system $(SH)$ admits a G.S., or a solution of the Dirichlet problem in a ball, it satisfies $u=v,$ from . Then we are completely reduced to the scalar case. In particular, in the critical case, the G.S. are given explicitely by: $u=v=c(K+r^{(2+a)})^{(2-N)/(2+a)},$ where $K=c^{\delta-1}/(N+a)(N-2);$ in other words they satisfy (3.3) with $X=Y$ and $Z=W,$ i.e.

Near $\infty,$ the G.S. is (obviously) symmetrical: it joins the points $N_{0}$ and $A_{0}.$

Consider the case $\delta=1,$ $a=b=0,$ which is the case of the biharmonic equation

Recall that it is the only case where the conjecture (1.3) was completely proved by Lin in . In the critical case $\mu=(N+4)/(N-4),$ the G.S. are also given explicitely, see :

They satisfy the relation $XY=\frac{N-Z}{2}X+\frac{N-4}{2N}(N-W)Y,$ and moreover we find that they are on an hyperplane, of equation

Observe also that the G.S. is not symmetrical near $\infty$ : $u$ behaves like $r^{4-N}$ and $v$ behaves like $r^{2-N}.$ The trajectory in the phase space joins the points $N_{0}$ and $Q_{0}=(N-4,N-2,2,0).$

Proof of Theorem 1.4. 1) Existence or nonexistence results:

$\bullet$ In the supercritical or critical case we apply any of the two conditions of Theorem 1.1: Here $\mathcal{E}_{H}(0)=0,$ and $\mathcal{E}_{H}$ is nonincreasing; there does not exist solutions of $(M)$ such that at some time $T,$ $X(T)=Y(T)=N-2$ , because at the time $T,$

since $W>0,Z>0,$ thus $\mathcal{E}_{H}(e^{T})>0,$ which is impossible. Otherwise there exists no solution of the Dirichlet problem in a ball $B(0,R),$ because $\mathcal{E}_{H}(R)=R^{N}u^{\prime}(R)v^{\prime}(R)>0$ from the Höpf Lemma. Then there exists a G.S. The uniqueness is proved in .

$\bullet$ In the subcritical case there is no radial G.S.: it would satisfy $\mathcal{E}_{H}(0)=0,$ and $\mathcal{E}_{H}$ is nondecreasing, $\mathcal{E}_{H}(r)\leqq Cr^{N-2-\gamma-\xi}$ from (5.1), and $\gamma+\xi>(N-2),$ then $\lim_{r\rightarrow\infty}\mathcal{E}_{H}(r)=0$ . From Theorem 1.1, there exists a solution of the Dirichlet problem.

2) Behaviour of the $G.S.$ in the critical case.

It is easy to see that the condition (1.6) implies $\mu>\frac{2+b}{N-2}$ and $\delta>\frac{2+a}{N-2},$ and that $\delta\leqq\frac{N+a}{N-2}$ and $\mu\leqq\frac{N+b}{N-2}$ cannot hold simultaneously. One can suppose that $\delta>\frac{N+a}{N-2}.$ Let $\mathcal{T}$ be the unique trajectory of the G.S.. Then $\mathcal{E}_{H}(0)=0,$ thus $\mathcal{T}$ lies on the variety $\mathcal{V}$ of energy , defined by

From (5.2) $\mathcal{T}$ starts from the point $N_{0},$ and from (5.1) $\mathcal{T}$ stays in

(i) Suppose that $\mathcal{T}$ converges to a fixed point of the system in $\mathcal{\bar{R}}$ . Then the only possible points are $A_{0},P_{0},Q_{0}$ which are effectively on $\mathcal{V}$ . Indeed $I_{0},$ $J_{0},G_{0},H_{0}\not\in\mathcal{V}.$ But $Q_{0}=((N-2)\delta-(2+a),N-2,N+a-(N-2)\delta,0)\not\in\mathcal{\bar{R}}$ , since $\delta>\frac{N+a}{N-2}$ . And $P_{0}\in\mathcal{\bar{R}}$ if and only if $\mu\leqq\frac{N+b}{N-2}$ .

If $\mu>\frac{N+b}{N-2},$ then $\mathcal{T}$ converges to $A_{0}$ . If $\mu<\frac{N+b}{N-2},$ no trajectory converges to $A_{0},$ from Proposition 4.5, thus $\mathcal{T}$ converges to $P_{0}$ . If $\mu\neq\frac{N+b}{N-2}$ the convergence is exponential, thus the behaviour of $u,v$ follows. If $\mu=\frac{N+b}{N-2},$ then $\mathcal{T}$ converges converges to $A_{0,}=P_{0};$ the eigenvalues given by (10.3) satisfy $\lambda_{1}=\lambda_{2}=N-2,$ $\lambda_{3}=N+a-\delta(N-2)<0$ and $\lambda_{4}=0;$ the projection of the trajectory on the hyperplane $Y=N-2$ satisfies the system

which presents a saddle point at $(N-2,0)$ , thus the convergence of $X$ and $Z$ is exponential, in particular we deduce the behaviour of $u.$ The trajectory enters by the central variety of dimension $1,$ and by computation we deduce that $Y-(N-2)=-t^{-1}+O(t^{-2+\varepsilon})$ near $\infty,$ and the behaviour of $v$ follows.

(ii) Let us show that $\mathcal{T}$ converges to a fixed point. We eliminate $W$ from (6.2) and we get a still quadratic system in $(X,Y,Z):$

We have $X_{t}\geqq 0,$ and $Y_{t}\geqq 0$ near $-\infty.$ Suppose that $X$ has a maximum at $t_{0}$ followed by a minimum at $t_{1}.$ At these times $X_{tt}=XZ_{t}$ , thus we find $Z_{t}(t_{0})<0<Z_{t}(t_{1}).$ There exists $t_{2}\in\left(t_{0},t_{1}\right)$ such that $Z_{t}(t_{2})=0,$ and $t_{2}$ is a minimum. At this time $Z(t_{2})=N+a-\delta Y(t_{2}),$ $Z_{tt}(t_{2})=-\delta(ZY_{t})(t_{2})$ hence

and $X_{t}(t_{2})<0,$ hence ( $X+Z)(t_{2})<N-2,$ and

but $X(t_{2})<X(t_{0})<\delta(N-2)-(2+a),$ which is contradictory. Then $X$ has at most one extremum, which is a maximum, and then it has a limit in $\left(0,N-2\right]$ at $\infty.$ In the same way, by symmetry, $Y$ has at most one extremum, which is a maximum, and has a limit in $\left(0,N-2\right]$ at $\infty.$ Then $Z$ has at most one extremum, which is a minimum. Indeed at the points where $Z_{t}=0,$ $-Z_{tt}$ has the sign of $Y_{t}$ . Thus $Z$ has a limit in $\left[0,N+a\right)$ , similarly $W$ has a limit in $\left[0,N+b\right).\vskip 6.0pt plus 2.0pt minus 2.0pt$

Open problems: 1) For the case $\delta=\mu,$ in the critical case it is well known that there exist solutions $(u,v)$ of system $(SH)$ of the form $(u,u),$ such that $r^{\gamma}u$ is periodic in $t=\ln r.$ They correspond to a periodic trajectory for the scalar system $(M_{scal})$ with $p=2,$ and it admits an infinity of such trajectories. If $\delta\neq\mu,$ does there exist solutions $(u,v)$ such that $(r^{\gamma}u,r^{\xi}v)$ is periodic in $t,$ in other words a periodic trajectory for system $(MH)?\vskip 6.0pt plus 2.0pt minus 2.0pt$

2) In the supercritical case, we cannot prove that the regular trajectory $\mathcal{T}$ converges to $M_{0},$ that means $\lim_{r\rightarrow\infty}r^{\gamma}u=A,$ $\lim_{r\rightarrow\infty}r^{\xi}v=B.$ Here $\mathcal{E}_{H}(0)=0,$ $\mathcal{E}_{H}$ is nonincreasing, then $\mathcal{E}_{H}$ is negative. The only fixed points of negative energy are $M_{0},$ $G_{0},H_{0},$ but a G.S. satisfies (5.5), then it tends to $(0,0)$ at $\infty,$ hence $\mathcal{T}$ cannot converge to $G_{0}$ or $H_{0}$ from Proposition 4.9; but we cannot prove that $\mathcal{T}$ converges to some fixed point.

A nonvariational system

Here we consider system $(S)$ with $p=q=2,a=b$ and $s=m\neq 0.$

where $D=\delta\mu-(1-s)^{2}$ $>0.$ In order to prove Theorem we can reduce the system to the case $a=0,$ by changing $N$ into $\hat{N}=\frac{2(N+a)}{2+a},$ from Remark 2.4; thus we assume $a=0$ in this Section. Here

We have chosen this system because it is not variational, and different hyperbolas in the plane $(\delta,\mu)$ :

$\bullet$ the hyperbola $\mathcal{H}_{s}$ for which the linearized system at $M_{0}$ has two imaginary roots, given by

whenever $s<\frac{N}{N-2},$ and $\delta+1-s>0,$ $\mu+1-s>0,$ from Proposition 4.3;

$\bullet$ the hyperbola $\mathcal{H}_{0}$ defined by

it was shown in that above $\mathcal{H}_{0}$ there exists no solution of the Dirichlet problem;

$\bullet$ an hyperbola $\mathcal{Z}_{s}$ introduced in in case $s<\frac{N}{N-2},$ and $\min(\delta,\mu)>\left|s-1\right|:$

$\bullet$ we introduce the new curve $\mathcal{C}_{s}$ defined for any $s>0$ by

We first extend and complete the results of and :

(i) Assume $s<\frac{N}{N-2},$ and $\delta+1-s>0,$ $\mu+1-s>0.$ Under the hyperbola $\mathcal{Z}_{s},$ system $(SN)$ admits no G.S., and then there is a solution of the Dirichlet problem in a ball.

(ii) Above $\mathcal{H}_{0}$ there exists no solution of the Dirichlet problem. Thus there exists a G.S.

Proof. (i) We consider an energy function with parameters $\alpha,\beta,\sigma,\theta:$

Taking $\alpha=\frac{1}{\mu+1},\beta=\frac{1}{\delta+1},$ we find

If there exists a G.S., from (5.1) it satisfies $X,Y<N-2,$ hence

Taking $\theta=\frac{N-(N-2)s}{\mu+1},\sigma=\frac{N-(N-2)s}{\delta+1},$ we deduce that $\mathcal{E}_{N}^{\prime}>0$ under $\mathcal{Z}_{s}$ . Moreover $\mathcal{Z}_{s}$ is under $\mathcal{H}_{s},$ thus $\gamma+\xi>N-2.$ Then $\mathcal{E}_{N}(r)=O(r^{N-2-\gamma-\xi})$ tends to at $\infty,$ which is contradictory.

(ii) Taking $\alpha=\frac{1}{\mu+1}=\frac{\theta}{N},\beta=\frac{1}{\delta+1}=\frac{\sigma}{N},$ it comes from (7.6)

hence $\mathcal{E}_{N}^{\prime}<0$ when (7.1) holds. At the value $\ R$ where $u(R)=v(R)=0,$ we find $\mathcal{E}_{N}(R)=R^{N}u^{\prime}(R)v^{\prime}(R)>0,$ which is a contradiction.

(i)When the four curves are simultaneously defined, they are in the following order, from below to above: $\mathcal{Z}_{s},\mathcal{H}_{s},\mathcal{C}_{s},\mathcal{H}_{0}.$ They intersect the diagonal $\delta=\mu$ repectively for

(ii) For $\delta=\mu,$ system $(SN)$ has a G.S. for $\delta\geqq\frac{N+2}{N-2}-s.$ Indeed it admits solutions of the form $(U,U)$ , where $U$ is a solution of equation $-\Delta U=U^{s+\delta}.$ Suppose moreover $s\leqq\delta.$ If $1-s<\delta<\frac{N+2}{N-2}-s,$ then there exists no G.S; indeed all such solutions satisfy $u=v,$ from [3, Remark 3.3]. Then the point $P_{s}=\left(\frac{N+2}{N-2}-s,\frac{N+2}{N-2}-s\right)$ appears to be the separation point on the diagonal; notice that $P_{s}\in\mathcal{H}_{s}.$

Next we prove our main existence result of existence of a G.S. valid without restrictions on $s$ . The main idea is to introduce a new energy function $\Phi$ by adding two terms in $X^{2}$ and $Y^{2}$ to the energy $\mathcal{E}_{N}$ defined at (7.3). It is constructed in order that $\Phi^{\prime}$ does not contain $Y$ and $Z.$ Then we consider the set of couples $(X,Y)$ such that $\Phi^{\prime}$ has a sign, which is bounded by a cubic curve. When ( $\delta,\mu)$ is above $\mathcal{C}_{s}$ , the cubic curve is exterior to the square

We eliminate the terms in $Z,W$ by taking $j=\alpha=\frac{1}{\mu+1},$ $k=\beta=\frac{1}{\delta+1},$ $\theta=N\alpha,$ $\sigma=N\beta.$ Then we get the function $\Phi$ defined at (1.9). Computing its derivative, we obtain after reduction

From Proposition 7.1 we can assume that $N(\alpha+\beta)-(N-2)>0$ . We determine the sign of $\mathcal{B}$ on the boundary $\partial K$ of the square $K$ defined at (7.8). We have $\mathcal{B}(0,Y)=\alpha Y^{2}(N-2-Y)\geqq 0$ and $\mathcal{B}(X,0)=\beta X^{2}(N-2-X)\geqq 0$ . In particular $\mathcal{B}(0,0)=0.$ Otherwise $\mathcal{B}(N-2,Y)=Y\Theta(Y)$ with

On the interval $\left[0,N-2\right],$ there holds $\Theta(Y)>\Theta(0)$ . By hypothesis, $(\delta,\mu)$ is above $\mathcal{C}_{s},$ or equivalently

consequently $\mathcal{B}(N-2,Y)\geqq 0$ and similarly $\mathcal{B}(X,N-2)\geqq 0.$ Then $\mathcal{B}$ is nonnegative on $\partial K$ and is zero at $(0,0),(0,N-2),(N-2,0)$ . The curve $\mathcal{B(}X,Y)=0$ is a cubic with a double point at $(0,0),$ which is isolated under the condition (7.9): $\mathcal{B(}X,Y)>0$ near $(0,0),$ except at this point. Then $\mathcal{B(}X,Y)>0$ on the interior of $K.\vskip 6.0pt plus 2.0pt minus 2.0pt$

Suppose that there exists a regular solution such that $X(T)=Y(T)=N-2$ at the same time $T.$ Indeed up to this time $(X,Y)$ stays in $K$ , thus the function $\Phi$ is decreasing. We have $\Phi(0)=0,$ and at the value $R=e^{T},$ we find

then $\Phi(R)>0$ , since $\min(\alpha,\beta)<\alpha+\beta$ . Therefore from Theorem 1.1, there exists a G.S.

We wonder if the limit curve for existence of G.S. would be $\mathcal{H}_{s}$ , or another curve $\mathcal{L}_{s}$ defined by

which ensures that $\Phi(R)>0,$ and also $\mathcal{B}(N-2,N-2)>0.$ This curve cuts the diagonal at the same point $P_{s}$ $=\left(\frac{N+2}{N-2}-s,\frac{N+2}{N-2}-s\right)$ as $\mathcal{H}_{s}.$ Notice that $\mathcal{L}_{s}$ is under $\mathcal{H}_{s}.$

The radial potential system

Here we study the nonnegative radial solutions of system $(SP):$

with $a=b,\delta=m+1,\mu=s+1,$ and we assume (1.5). System $(M)$ becomes

For this system $D,$ $\gamma$ and $\xi$ are defined by

thus $\gamma$ and $\xi$ are linked independtly of $s,m$ by the relation

The system is variational. It admits an energy function, given at (1.13), which can also can be obtained by a direct computation in terms of $X,Y,Z,W$ :

Thus we define a critical line $\mathcal{D}$ as the set of $\left(\delta,\mu\right)=(m+1,s+1)$ such that

equivalent to $pq(m+s+2+a)=ND,$ or $N+a=(m+1)\xi+(s+1)\gamma,$ or

The subcritical case is given by the set of points under $\mathcal{D}$ , equivalently $\gamma>\frac{N-p}{p}$ , $\xi>\frac{N-q}{q}$ or $\left(s+1\right)\gamma+(m+1)\xi>N+a.$ The supercritical case is the set of points above $\mathcal{D}.$

The energy $\mathcal{(\mathcal{E}_{P})}_{0}$ of the particular solution associated to $M_{0}$ is still negative $:$ $(\mathcal{E}_{P}\mathcal{)}_{0}=-\frac{D}{pq}r^{N+a-(\gamma+1)p}\left[X_{0}^{q(p-1)}Y_{0}^{p(q-1)}Z_{0}^{q(s+1)}W_{0}^{p(m+1)}\right]^{1/D}.$

When $p=q=2$ , another energy function can be associated to the transformation given at Remark 2.2: the system (2.9) relative to $u(r)=r^{-\gamma}$ U $(t),\quad v(r)=r^{-\xi}$ V $(t)$ is

It differs from $\mathcal{E}_{P},$ even in the critical case. This point is crucial for Section 9.

Proof of Theorem 1.6. 1) Existence or nonexistence results.

$\bullet$ In the supercritical or critical case there exists a G.S. From Theorem 1.1, if it were not true, then there would exist regular positive solutions of $(MP)$ such that $X(T)=\frac{N-p}{p-1}$ and $Y(T)=\frac{N-q}{q-1}$ . It would satify $\mathcal{E}_{P}\leqq 0.$ Then at time $T,$ we find $\mathcal{E}_{P}(R)>0,$ from (8.2), since $W>0,Z>0,$ which is impossible.

$\bullet$ In the subcritical case, there exists no G.S. Suppose that there exists one. Now $\mathcal{E}_{P}$ is nondecreasing, hence $\mathcal{E}_{P}\geqq 0.$ Its trajectory stays in the box $\mathcal{A}$ defined by (5.1), thus it is bounded. If $q\geqq m+1$ and $p\geqq s+1,$ we deduce that , $\mathcal{E}_{P}(r)=O(r^{N-(\gamma+1)p})$ from (8.2), then $\mathcal{E}_{P}$ tends to at $\infty,$ which is contradictory. Next consider the general case. We have

then the same result holds. Consequently, from Theorem 1.1, there exists a solution of the Dirichlet problem

2) Behaviour of the $G.S.$ in the critical case.

Let $\mathcal{T}$ be the trajectory of a G.S.; then $\mathcal{E}_{P}(0)=0,$ thus $\mathcal{T}$ lies on the variety $\mathcal{V}$ of energy , also defined by

and $Y<\frac{N-q}{q-1},$ hence $(s+1)((p-1)X-(N-p))+pZ>0$ . From (5.2), $\mathcal{T}$ starts from $N_{0}=(0,0,N+a,N+b)$ and stays in $\mathcal{A}$ . Eliminating $W$ in system $(M),$ we find a system of three equations

(i) If $\mathcal{T}$ converges to a fixed point of the system in $\mathcal{\bar{R}}$ , the possible points on $\mathcal{V}$ are $A_{0},I_{0},J_{0},$ $P_{0},$ $Q_{0},G_{0},H_{0},$ $R_{0},S_{0}.$ The eigenvalues of the linearized problem at $A_{0},$ given by (10.3) satisfy

since $q\leqq p,$ and $\lambda_{3}<\lambda^{\ast}$ for $q\neq p,$ and $\lambda_{3}=\lambda^{\ast}<0$ for $q=p,$ from (8.3). Then $A_{0}$ can be attained only when $\lambda^{\ast}\leqq 0,$ from Proposition 4.5. And $P_{0}$ can be attained only if

from Proposition 4.6, because $\gamma=\frac{N-p}{p}<\frac{N-p}{p-1}.$ We observe that the condition $\lambda^{\ast}\geqq 0$ joint to (8.3) implies $m+1<q<p$ and is equivalent to (8.7). Indeed it implies

hence $m+1<q$ and (8.7) follows. By symmetry, $Q_{0}$ cannot be attained since $q\leqq p.$ Then $A_{0}$ and $P_{0}$ are incompatible, unless $A_{0}=P_{0}$ , and $P_{0}$ is not attained when $p=q.\vskip 6.0pt plus 2.0pt minus 2.0pt$

(ii) Next we show that $\mathcal{T}$ converges to $A_{0}$ or to $P_{0}$ . If $t$ is an extremum value of $Y$ , then

In the same way, if $t$ is an extremum value of $X,$ then $p>s+1$ and $Y_{t}(t)>0.$ Near $-\infty,$ there holds $X_{t},Y_{t}\geqq 0,$ and $Z_{t},W_{t}\leqq 0,$ from the linearization near $N_{0}.$ Suppose that $X$ has a maximum at $t_{0}$ followed by a minimum at $t_{1}.$ Then $p>s+1,$ and $Y$ is increasing on $\left[t_{0},t_{1}\right]$ . At time $t_{0}$ we have $(p-1)X(t_{0})+Z(t_{0})=N-p$ and $X_{tt}(t_{0})\leqq 0,$ thus $Z_{t}(t_{0})\leqq 0;$ eliminating $Z$ we deduce $p+a+(p-1-s)X(t_{0})\leqq(m+1)Y(t_{0})$ and similarly $(m+1)Y(t_{1})\leqq p+a+(p-1-s)X(t_{1});$ hence $Y(t_{1})<Y(t_{0})$ , which is a contradiction. Thus $X$ and $Y$ can have at most one maximum, and in turn they have no maximum point. Therefore $X$ and $Y$ are increasing, and they are bounded, hence $X$ has a limit in $\left(0,\frac{N-p}{p-1}\right]$ and $Y$ has a limit in $\left(0,\frac{N-q}{q-1}\right]$ . Then $Z,W$ are decreasing; indeed at each time where $Z_{t}=0,$ we have $Z_{tt}=Z(-sX_{t}-(m+1)Y_{t})<0,$ thus it is a maximum, which is impossible.

Then $\mathcal{T}$ converges to a fixed point of the system. Moreover, since $X$ and $Y$ are increasing, it cannot be one of the points $I_{0},J_{0},G_{0},H_{0},R_{0},S_{0}.$ It is necessarily $A_{0}$ or $P_{0}.$ We distinguish two cases:

$\bullet$ Case $q\leqq m+1$ . Then $\mathcal{T}$ converges to $A_{0},$ and $\lambda_{3},\lambda^{\ast}<0,$ then (1.10) follows.

which presents a saddle point at $(\frac{N-p}{p-1},0)$ , thus the convergence of $X$ and $Z$ is exponential, in particular we deduce the behaviour of $u.$ The trajectory enters by the central variety of dimension $1,$ and by computation we deduce that $Y=\frac{N-q}{q-1}-\frac{1}{q-1-m}t^{-1}+O(t^{-2+\varepsilon}),$ then (1.12) follows.

The nonradial potential system of Laplacians

Here we study the possibly nonradial solutions of the system of the preceeding Section when $p=q=2:$

with $D=s+m.$ We solve an open problem of : the nonexistence of (radial or nonradial) G.S. under condition (1.14).

It was shown in in the case $N+a\geqq 4.$ The problem was open when $N+a<4,$ and $m+s+1>(N+a)/(N-2),$ which implies $N<6.$ Indeed in the case $m+s+1\leqq(N+a)/(N-2),$ there are no solutions of the exterior problem, see [6, Theorem 5.3]. Recall that the main result of is the obtention of apriori estimates near or $\infty,$ by using the Bernstein technique introduced in and improved in . Then the behaviour of the solutions is obtained by using the change of unknown

extending the transformation of Remark 8.2 to the nonradial case (in fact here $t$ is $-t$ in $)$ ; it leads to the system

where $\Delta_{S}$ is the Laplace-Beltrami operator on $S_{N-1}.$ A corresponding energy is introduced in :

extending (8.5) to the nonradial case; it satisfies

Here we construct another energy function, extending the Pohozaev function defined at (1.13) to the nonradial case.

Consider the function $\mathcal{E}_{L}(r)$ defined by

and $v$ satisfies symmetrical equations. Multiplying (9.2) by $u$ and (9.1) by $(s+1)e^{(N-2)t}u_{t},$ we obtain

and symmetrically for $v,$ and adding the equalities we deduce

Proof of Theorem 1.7. Suppose that there exists a G.S. Since $s+m+1<(N+2+2a)/(N-2)$ we deduce that $E_{L}$ and $\mathcal{E}_{L}$ are increasing and start from 0, then they stay positive. From [7, Corollary 6.4], since $s+m+1<(N+2)/(N-2),$ three eventualities can hold. The first one is that $(u,v)$ behaves like the particular solution $(u_{0},v_{0});$ it cannot hold because $E_{L}$ has a negative limit, see [7, Remark 6.3]. The second one is that $(u,v)$ is regular at $\infty,$ that means $\lim_{\left|x\right|\rightarrow\infty}\left|x\right|^{N-2}u=\alpha>0,$ $\lim_{\left|x\right|\rightarrow\infty}\left|x\right|^{N-2}v=\beta>0;$ it cannot hold because $\lim_{t\rightarrow\infty}E_{L}(t)=0.$ It remains a third eventuality: when for example $m>(N+a)/(N-2)$ , and $(u,v)$ has the following behaviour at $\infty:$

The condition on $m$ implies that $N<4-a$ from assumption (1.14). In that case $\lim_{t\rightarrow\infty}E_{L}(t)=\infty,$ which gives no contradiction. Here we show that a contradiction holds by using the new energy function $\mathcal{E}_{L}.\vskip 6.0pt plus 2.0pt minus 2.0pt$

First recall the proof of (9.3). Making the substitution

Then $u\mathbf{,V}$ are bounded near $\infty,$ and from [7, Proposition 4.1] $u$ converges exponentially to the constant $\alpha,$ more precisely

because $k\neq(N-2)/2$ and all the derivatives of $\mathbf{V}$ up to the order $2$ are bounded. The equation in $\mathbf{V}$ takes the form

where $\varphi$ and its derivatives up to the order $2$ are $O(e^{-(N-2)t}).$ From [7, Theorem 4.1], the function $\mathbf{V}$ converges to $\beta$ or to in $C^{2}(S^{N-1}).\vskip 6.0pt plus 2.0pt minus 2.0pt$

Moreover $v=e^{-kt}\mathbf{V},$ and $\mathbf{V}$ and its derivatives up to the order $2$ are bounded, thus

and $N+a-k(m+1)<\frac{2-N}{m-1}<0.$ Then $\mathcal{E}_{L}$ has a finite limit $\theta<0$ at $\infty,$ which is contradictory.

Analysis of the fixed points

Here we make the local analysis around the fixed points.

Proof of Proposition 4.4. (i) Consider a regular solution $(u,v)$ with initial data $(u_{0},v_{0}).$ When when $r$ $\rightarrow$ $0,$ we have

thus from (2.1), when $t\rightarrow-\infty$

and $\lim_{t\rightarrow-\infty}Z=N+a,$ $\lim_{t\rightarrow-\infty}W=(N+b).$ In particular the trajectory tends to $N_{0}=(0,0,N+a,N+b).\vskip 6.0pt plus 2.0pt minus 2.0pt$

with $x(0)=y(0)=0.$ Then we get local existence and uniqueness. Hence for any $u_{0},v_{0}>0$ there exists a regular solution $(u,v)$ with initial data ( $u_{0},v_{0}).$ Moreover $u,v\in$ $C^{1}(\left[0,R\right))$ when $a,b>-1.\vskip 6.0pt plus 2.0pt minus 2.0pt$

$\bullet$ Convergence when $r\rightarrow$ $\infty:$ If $\lambda_{3}>0,$ or $\lambda_{4}>0,$ then the stable variety $\mathcal{V}_{s}$ has at most dimension $1,$ it satisfies $W=0$ or $Z=0,$ hence there is no admissible trajectory converging to $A_{0}$ at $\infty.$ If $\lambda_{3}<0,$ and $\lambda_{4}<0,$ then $\mathcal{V}_{s}$ has dimension $2.$ Moreover $\mathcal{V}_{s}\cap\left\{Z=0\right\}$ has dimension $1:$ the corresponding system in $X,Y,W$ has the eigenvalues $\lambda_{1},\lambda_{2},\lambda_{4};$ similarly $\mathcal{V}_{s}\cap\left\{W=0\right\}$ has dimension 1. Then there exist trajectories in $\mathcal{V}_{s}$ such that $Z>0$ and $W>0,$ included in $\mathcal{R}$ and thus admissible. They satisfy $\lim e^{-\lambda_{3}t}Z=C_{3}>0,\lim e^{-\lambda_{4}t}W=C_{4}>0,$ then (4.11) follows from (4.2).

$\bullet$ Convergence when $r\rightarrow 0:$ If $\lambda_{3}<0,$ or $\lambda_{4}<0,$ the unstable variety $\mathcal{V}_{u}$ has at most dimension $3$ , and it satisfies $W=0$ or $Z=0.$ Therefore there is no admissible trajectory converging at $-\infty.$ If $\lambda_{3},\lambda_{4}>0,$ then $\mathcal{V}_{u}$ has dimension $4;$ in that case there exist admissible trajectories, and (4.11) follows as above.

Proof of Proposition 4.6. We set $P_{0}=\left(\frac{N-p}{p-1},Y_{\ast},0,W_{\ast}\right),$ with

and the roots $\lambda_{2},\lambda_{4}$ of equation

Then if $\lambda_{3}<0$ (resp. $\lambda_{3}>0)$ there is no admissible trajectory converging when $r\rightarrow 0$ (resp. $r\rightarrow\infty).$ Indeed $\mathcal{V}_{u}=\mathcal{V}_{u}\cap\left\{Z=0\right\}$ (resp. $\mathcal{V}_{s}=\mathcal{V}_{s}\cap\left\{Z=0\right\})$ .

1) Suppose that $q>m+1.$ Since $q+b<\frac{N-p}{p-1}\mu<N+b-m\frac{N-q}{q-1},$ we have $P_{0}\in\mathcal{R},$ and $\lambda_{2}\lambda_{4}<0$ . First assume $\lambda_{3}<0,$ that means $\gamma<\frac{N-p}{p-1}$ . Then $\mathcal{V}_{s}$ has dimension 2, and $\mathcal{V}_{s}\cap\left\{Z=0\right\}$ has dimension 1, there exists trajectories with $Z>0,$ which are admissible, converging when $r\rightarrow\infty.$ Next assume $\lambda_{3}>0$ . Then $\mathcal{V}_{u}$ has dimension 3, and $\mathcal{V}_{u}\cap\left\{Z=0\right\}$ has dimension 2. Thus there exist admissible trajectories converging when $t\rightarrow-\infty.$

2) Suppose that $q<m+1$ . Since $q+b>\frac{N-p}{p-1}\mu>N+b-m\frac{N-q}{q-1},$ we have $P_{0}\in\mathcal{R},$ and $\lambda_{2}\lambda_{4}>0$ . We assume $q\frac{N-p}{p-1}\mu+m(N-q)\neq N(q-1)+(b+1)q,$ that means $Y_{\ast}\neq W_{\ast}.$ First suppose $\lambda_{3}>0,$ that means $\gamma<\frac{N-p}{p-1}.$ If Re $\lambda_{2}>0,$ then $\mathcal{V}_{u}$ has dimension 4, or Re $\lambda_{2}<0$ then $\mathcal{V}_{u}$ has dimension $2$ and $\mathcal{V}_{u}\cap\left\{Z=0\right\}$ has dimension $1.$ In any case, there exist admissible trajectories converging when $r$ $\rightarrow$ . Next assume $\lambda_{3}<0$ . If Re $\lambda_{2}>0,$ then $\mathcal{V}_{s}$ has dimension 1, and $\mathcal{V}_{s}\cap\left\{Z=0\right\}=\emptyset.$ If Re $\lambda_{2}<0,$ then $\mathcal{V}_{s}$ has dimension $3$ . In any case $\mathcal{V}_{s}$ contains trajectories with $Z>0,$ which are admissible, converging when $r\rightarrow\infty.$

Those trajectories satisfy $\lim e^{-\lambda_{3}t}Z=C_{3}>0,$ $\lim X=\frac{N-p}{p-1},$ $\lim Y=Y_{\ast}$ and $\lim W=W_{\ast},$ thus (4.12) follows from (4.2) and (2.5).

$\bullet$ Convergence when $r\rightarrow$ $\infty:$ If $\lambda_{3}>0$ or $\lambda_{4}>0,$ then $\mathcal{V}_{s}=\mathcal{V}_{s}\cap\left\{Z=0\right\}$ or $\mathcal{V}_{s}=\mathcal{V}_{s}\cap\left\{W=0\right\}$ . There is no admissible trajectory converging at $\infty$ . Next suppose that $\lambda_{3},\lambda_{4}<0.$ Then $\mathcal{V}_{s}$ has dimension $3;$ it contains trajectories with $Y,Z,W>0,$ which are admissible. They satisfy $\lim X=\frac{N-p}{p-1},$ $\lim e^{-\lambda_{2}t}Y=C_{2}>0,$ $\lim e^{-\lambda_{3}t}Z=C_{3}>0,$ $\lim e^{-\lambda_{4}t}W=C_{4}>0,$ then (4.13) follows from (4.2) and (2.4).

$\bullet$ Convergence when $r\rightarrow$ $0:$ Since $\lambda_{2}<0$ we have $\mathcal{V}_{u}=\mathcal{V}_{u}\cap\left\{Y=0\right\},$ hence there is no admissible trajectory converging when $r\rightarrow 0.$

$\bullet$ Convergence when $r\rightarrow$ $\infty:$ If $\lambda_{2}>0,$ or $\lambda_{3}>0,$ then $\mathcal{V}_{s}=$ $\mathcal{V}_{s}\cap\left\{Y=0\right\}$ or $\mathcal{V}_{s}=\mathcal{V}_{s}\cap\left\{Z=0\right\}$ , there is no admissible trajectory converging at $\infty$ . Assume $\lambda_{2},\lambda_{3}<0,$ then $\mathcal{V}_{s}$ has dimension 3, it contains trajectories with $Y,Z>0,$ which are admissible.

$\bullet$ Convergence when $r\rightarrow$ $0:$ If $\lambda_{3}<0,$ or $\lambda_{2}<0$ there is no admissible trajectory. If $\lambda_{2},\lambda_{3}>0$ then $\mathcal{V}_{s}$ has dimension $3,$ it contains admissible trajectories.

In any case $\lim X=\frac{N-p}{p-1},$ $\lim e^{-\lambda_{2}t}Y=C_{2}>0,$ $\lim e^{-\lambda_{3}t}Z=C_{3}>0,$ $\lim W=N+b-\frac{N-p}{p-1}\mu,$ hence (4.13) still follows from (4.2) and (2.4).

Proof of Proposition 4.10. We set $C_{0}=\left(0,\bar{Y},0,\bar{W}\right),$ with

and the roots $\lambda_{2},\lambda_{4}$ of equation

then $\lambda_{2}\lambda_{4}>0.$ We assume $m\neq\frac{N(q-1)+(b+1)q}{N-q},$ that means $\bar{Y}\neq\bar{W}.$

$\bullet$ Convergence when $r\rightarrow$ $\infty:$ if $\lambda_{3}>0$ we have $\mathcal{V}_{s}=\mathcal{V}_{s}\cap\left\{Z=0\right\},$ hence there is no admissible trajectory. Next assume that $\lambda_{3}<0$ , that means $\delta>(N+a)\frac{m+1-q}{q+b}.$ If $\operatorname{Re}\lambda_{2}<0$ (resp. $>0)$ then $\mathcal{V}_{s}$ has dimension $4$ (resp. $2$ ) and $\mathcal{V}_{s}\cap\left\{X=0\right\}$ and $\mathcal{V}_{s}\cap\left\{Z=0\right\}$ have dimension $3$ (resp. $1$ ) then there exist trajectories with $X,Z>0,$ which are admissible.

In any case $\lim e^{-\lambda_{1}t}X=C_{1}>0,$ $\lim Y=\bar{Y},$ $\lim e^{-\lambda_{3}t}Z=C_{3}>0,$ $\lim W=\bar{W},$ then (4.15) follows.

$\bullet$ Convergence when $r\rightarrow$ $0:$ Since $\lambda_{1}<0$ we have $\mathcal{V}_{u}=\mathcal{V}_{u}\cap\left\{X=0\right\},$ hence there is no admissible trajectory.

and the roots $\lambda_{2},\lambda_{4}$ of equation of equation (10.5).

$\bullet$ Convergence when $r\rightarrow$ $\infty:$ If $\lambda_{1}>0$ , that means $(p+a)\frac{m+1-q}{q+b}<\delta,$ then $\mathcal{V}_{s}=\mathcal{V}_{s}\cap\left\{X=0\right\},$ hence there is no admissible trajectory. Next assume $\lambda_{1}<0;$ if $\operatorname{Re}\lambda_{2}<0$ (resp. $>0)$ then $\mathcal{V}_{s}$ has dimension 4(resp. $2$ ) and $\mathcal{V}_{s}\cap\left\{X=0\right\}$ has dimension $3$ (resp. $1$ ) then there exist admissible trajectories.

$\bullet$ Convergence when $r\rightarrow$ $0:$ If $\lambda_{1}<0$ , then $\mathcal{V}_{u}=\mathcal{V}_{u}\cap\left\{X=0\right\},$ hence there is no admissible trajectory. Next assume $\lambda_{1}>0.$ If $\operatorname{Re}\lambda_{2}=\operatorname{Re}\lambda_{4}<0$ (resp. $>0)$ then $\mathcal{V}_{s}$ has dimension 4 (resp. $2$ ) and $\mathcal{V}_{s}\cap\left\{X=0\right\}$ has dimension $3$ (resp. $1$ ) then there exist admissible trajectories.

In any case $\lim e^{-\lambda_{1}t}X=C_{1}>0,$ $\lim Y=\bar{Y},$ $\lim Z=\bar{Z},$ $\lim W=\bar{W},$ then (4.15) holds again.

Finally there is no admissible trajectory converging to $0=(0,0,0,0),$ or $K_{0}=(0,0,N+a,0),$ or $L_{0}=(0,0,0,N+b)$ . Indeed the linearization at gives

The eigenvalues are $\frac{p+a}{p-1},-\frac{N-q}{q-1},-(N+a),$ $N+b.$ Then $\mathcal{V}_{s}$ and $\mathcal{V}_{u}$ have dimension $2,$ hence $\mathcal{V}_{s}$ is contained in $\left\{Z=W=0\right\}$ , and $\mathcal{V}_{u}$ in $\left\{Y=0\right\}.$ The case of $L_{0}$ follows by symmetry.