For N≥3, α>1, β>1 and 2 < q < p = α + β < 2 ∗ = 2 N / ( N − 2 ) , we consider the semilinear elliptic systems

where ε,λ,μ>0.

Let f, g and h satisfy the following conditions:

(A1) f is a positive continuous function in RN and lim | x | → ∞ f ( x ) = f ∞ > 0 .

(A2) there exist k points a 1 , a 2 , … , a k in RN such that

f ( a i ) = max x ∈ R N f ( x ) = 1 for  1 ≤ i ≤ k ,

and f ∞ < 1 .

(A3) g , h ∈ L m ( R N ) ∩ L ∞ ( R N ) where m = ( α + β ) / ( α + β − q ) , and g , h ≩ 0 .

In 1, if Ω is a smooth and bounded domain in R N (N ≤ 3 ), they considered the following system:

{ ε 2 Δ u ¯ − λ 1 u ¯ = μ 1 u ¯ 3 + β u ¯ v ¯ 2 in  Ω ; ε 2 Δ v ¯ − λ 2 v ¯ = μ 2 v ¯ 3 + β u ¯ 2 v ¯ in  Ω ; u ¯ > 0 , v ¯ > 0 ,

and proved the existence of a least energy solution in Ω for sufficiently small ε > 0 and β ∈ ( − ∞ , β 0 ) . Lin and Wei also showed that this system has a least energy solution in RN for ε = 1 and β ∈ ( 0 , β 0 ) . In this paper, we study the effect of f ( z ) of ( E ¯ ε , λ , μ ). Recently, many authors 2345 considered the elliptic systems with subcritical or critical exponents, and they proved the existence of a least energy positive solution or the existence of at least two positive solutions for these problems. In this paper, we construct the k compact Palais-Smale sequences which are suitably localized in correspondence of k maximum points of f. Then we could show that under some assumptions (A1)-(A3), for sufficiently small ε,λ,μ>0, there are at least k (≥1) positive solutions of the elliptic system ( E ε , λ , μ ). By the change of variables

x = ε z , u ( z ) = u ¯ ( ε z ) and v ( z ) = v ¯ ( ε z ) ,

System (E¯ε,λ,μ) is transformed to

Let H = H 1 ( R N ) × H 1 ( R N ) be the space with the standard norm

∥ ( u , v ) ∥ H = [ ∫ R N ( | ∇ u | 2 + u 2 ) d z + ∫ R N ( | ∇ v | 2 + v 2 ) d z ] 1 / 2 .

Associated with the problem (Eε,λ,μ), we consider the C 1 -functional J ε , λ , μ , for ( u , v ) ∈ H ,

J ε , λ , μ ( u , v ) = 1 2 ∥ ( u , v ) ∥ H 2 − 1 α + β ∫ R N f ( ε z ) | u | α | v | β d z − 1 q ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z .

Actually, the weak solution ( u , v ) ∈ H of (Eε,λ,μ) is the critical point of the functional J ε , λ , μ , that is, (u,v)∈H satisfies

for any ( φ 1 , φ 2 ) ∈ H .

We consider the Nehari manifold

M ε , λ , μ = { ( u , v ) ∈ H ∖ { ( 0 , 0 ) } | 〈 J ε , λ , μ ′ ( u , v ) , ( u , v ) 〉 = 0 } ,

where

〈 J ε , λ , μ ′ ( u , v ) , ( u , v ) 〉 = ∥ ( u , v ) ∥ H 2 − ∫ R N f ( ε z ) | u | α | v | β d z − ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z .

The Nehari manifold M ε , λ , μ contains all nontrivial weak solutions of (Eε,λ,μ).

Let

S α , β = inf u , v ∈ H 1 ( R N ) ∖ { ( 0 ) } ∥ ( u , v ) ∥ H 2 ( ∫ R N | u | α | v | β d z ) 2 / ( α + β ) ,

then by [2, Theorem 5], we have

S α , β = [ ( α β ) β α + β + ( β α ) α α + β ] S p ,

where p = α + β and S p is the best Sobolev constant defined by

S p = inf u ∈ H 1 ( R N ) ∖ { 0 } ∫ R N ( | ∇ u | 2 + u 2 ) d z ( ∫ R N | u | p d z ) 2 / p .

For the semilinear elliptic systems (λ = μ = 0 )

we define the energy functional I ε ( u , v ) = 1 2 ∥ ( u , v ) ∥ H 2 − 1 α + β ∫ R N f ( ε z ) | u | α | v | β d z , and

N ε = { ( u , v ) ∈ H ∖ { ( 0 , 0 ) } | 〈 I ε ′ ( u , v ) , ( u , v ) 〉 = 0 } .

If f ≡ max z ∈ R N f ( z ) (=1), then we define I max ( u , v ) = 1 2 ∥ ( u , v ) ∥ H 2 − 1 α + β ∫ R N | u | α | v | β d z and

θ max = inf ( u , v ) ∈ N max I max ( u , v ) ,

where N max = { ( u , v ) ∈ H ∖ { ( 0 , 0 ) } | 〈 I max ′ ( u , v ) , ( u , v ) 〉 = 0 } .

It is well known that this problem

has the unique, radially symmetric and positive ground state solution w ∈ H 1 ( R N ) . Define I ¯ max ( u ) = 1 2 ∫ R N ( | ∇ u | 2 + u 2 ) d z − 1 p ∫ R N | u | p d z and θ ¯ max = inf u ∈ N ¯ max I ¯ max ( u ) , where

N ¯ max = { u ∈ H 1 ( R N ) ∖ { 0 } | 〈 I ¯ max ′ ( u ) , u 〉 = 0 } .

Moreover, we have that

θ ¯ max = p − 2 2 p S p p p − 2 > 0 . (See Wang [6, Theorems 4.12 and 4.13].)

This paper is organized as follows. First of all, we study the argument of the Nehari manifold Mε,λ,μ. Next, we prove that the existence of a positive solution ( u 0 , v 0 ) ∈ M ε , λ , μ of (Eε,λ,μ). Finally, in Section 4, we show that the condition (A2) affects the number of positive solutions of (Eε,λ,μ); that is, there are at least k critical points ( u i , v i ) ∈ M ε , λ , μ of Jε,λ,μ such that J ε , λ , μ ( u i , v i ) = β ε , λ , μ i ((PS)-value) for 1 ≤ i ≤ k .

Theorem 1.1 (Eε,λ,μ) has at least one positive solution ( u 0 , v 0 ) , that is, (E¯ε,λ,μ) admits at least one positive solution.

Theorem 1.2 There exist two positive numbers ε 0 and Λ ∗ such that (Eε,λ,μ) has at least k positive solutions for any 0 < ε < ε 0 and 0 < λ + μ < Λ ∗ , that is, (E¯ε,λ,μ) admits at least k positive solutions.

By studying the argument of Han [7, Lemma 2.1], we obtain the following lemma.

Lemma 2.1 Let Ω ⊂ R N (possibly unbounded) be a smooth domain. If u n ⇀ u , v n ⇀ v weakly in H 0 1 ( Ω ) , and u n → u , v n → v almost everywhere in Ω, then

lim n → ∞ ∫ Ω | u n − u | α | v n − v | β d z = lim n → ∞ ∫ Ω | u n | α | v n | β d z − ∫ Ω | u | α | v | β d z .

Note that Jε,λ,μ is not bounded from below in H. From the following lemma, we have that Jε,λ,μ is bounded from below on Mε,λ,μ.

Lemma 2.2 The energy functional Jε,λ,μ is bounded from below on Mε,λ,μ.

Proof For ( u , v ) ∈ M ε , λ , μ , by (1.1), we obtain that

J ε , λ , μ ( u , v ) = ( 1 2 − 1 q ) ∥ ( u , v ) ∥ H 2 + ( 1 q − 1 p ) ∫ R N f ( ε z ) | u | α | v | β d z > 0 ,

where p=α+β. Hence, we have that J ε , λ , μ is bounded from below on Mε,λ,μ. □

We define

θ ε , λ , μ = inf ( u , v ) ∈ M ε , λ , μ J ε , λ , μ ( u , v ) .

Lemma 2.3 (i) There exist positive numbers σ and d 0 such that J ε , λ , μ ( u , v ) ≥ d 0 for ∥ ( u , v ) ∥ H = σ ;

(ii) There exists ( u ¯ , v ¯ ) ∈ H ∖ { ( 0 , 0 ) } such that ∥ ( u ¯ , v ¯ ) ∥ H > σ and J ε , λ , μ ( u ¯ , v ¯ ) < 0 .

Proof (i) By (1.2), the Hölder inequality ( p 1 = p p − q , p 2 = p q ) and the Sobolev embedding theorem, we have that

J ε , λ , μ ( u , v ) = 1 2 ∥ ( u , v ) ∥ H 2 − 1 p ∫ R N f ( ε z ) | u | α | v | β d z − 1 q ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z ≥ 1 2 ∥ ( u , v ) ∥ H 2 − 1 p S α , β − p / 2 ∥ ( u , v ) ∥ H p − 1 q Max S p − q 2 ( λ + μ ) ∥ ( u , v ) ∥ H q ,

where p=α+β and Max = max { ∥ g ∥ m , ∥ h ∥ m } . Hence, there exist positive σ and d0 such that J ε , λ , μ ( u , v ) ≥ d 0 for ∥ ( u , v ) ∥ H = σ .

(ii) For any ( u , v ) ∈ H ∖ { ( 0 , 0 ) } , since

J ε , λ , μ ( t u , t v ) = t 2 2 ∥ ( u , v ) ∥ H 2 − t p p ∫ R N f ( ε z ) | u | α | v | β d z − t q q ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z ,

then lim t → ∞ J ε , λ , μ ( t u , t v ) = − ∞ . Fix some (u,v)∈H∖{(0,0)}, there exists t ¯ > 0 such that ∥ ( t ¯ u , t ¯ v ) ∥ H > σ and J ε , λ , μ ( t ¯ u , t ¯ v ) < 0 . Let ( u ¯ , v ¯ ) = ( t ¯ u , t ¯ v ) . □

Define

ψ ( u , v ) = 〈 J ε , λ , μ ′ ( u , v ) , ( u , v ) 〉 .

Then for (u,v)∈Mε,λ,μ, we obtain that

〈 ψ ′ ( u , v ) , ( u , v ) 〉 = 2 ∥ ( u , v ) ∥ H 2 − p ∫ R N f ( ε z ) | u | α | v | β d z − q ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z = ( p − q ) ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z − ( p − 2 ) ∥ ( u , v ) ∥ H 2

= ( 2 − q ) ∥ ( u , v ) ∥ H 2 + ( q − p ) ∫ R N f ( ε z ) | u | α | v | β d z < 0 .

Lemma 2.4 For each ( u , v ) ∈ H ∖ { ( 0 , 0 ) } , there exists a unique positive number t u , v such that ( t u , v u , t u , v v ) ∈ M ε , λ , μ and J ε , λ , μ ( t u , v u , t u , v v ) = sup t ≥ 0 J ε , λ , μ ( t u , t v ) .

Proof Fixed (u,v)∈H∖{(0,0)}, we consider

R ( t ) = J ε , λ , μ ( t u , t v ) = t 2 2 ∥ ( u , v ) ∥ H 2 − t p p ∫ R N f ( ε z ) | u | α | v | β d z − t q q ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z .

Since R ( 0 ) = 0 , lim t → ∞ R ( t ) = − ∞ , by Lemma 2.3(i), then sup t ≥ 0 R ( t ) is achieved at some t u , v > 0 . Moreover, we have that R ′ ( t u , v ) = 0 , that is, (tu,vu,tu,vv)∈Mε,λ,μ. Next, we claim that tu,v is a unique positive number such that R′(tu,v)=0. Consider

r ( t ) = ∥ ( u , v ) ∥ H 2 − t p − 2 ∫ R N f ( ε z ) | u | α | v | β d z − t q − 2 ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z ,

then R ′ ( t ) = t r ( t ) . Since r ( 0 ) = ∥ ( u , v ) ∥ H 2 > 0 ,

r ′ ( t ) = − ( p − 2 ) t p − 3 ∫ R N f ( ε z ) | u | α | v | β d z − ( q − 2 ) t q − 3 ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z < 0 ,

there exists a unique positive number t ¯ u , v such that r ( t ¯ u , v ) = 0 . It follows that R ′ ( t ¯ u , v ) = 0 . Hence, t ¯ u , v = t u , v . □

Remark 2.5 By Lemma 2.3(i) and Lemma 2.4, then θ ε , λ , μ ≥ d 0 > 0 for some constant d0.

Lemma 2.6 Let ( u 0 , v 0 ) ∈ M ε , λ , μ satisfy

J ε , λ , μ ( u 0 , v 0 ) = min ( u , v ) ∈ M ε , λ , μ J ε , λ , μ ( u , v ) = θ ε , λ , μ ,

then (u0,v0) is a solution of (Eε,λ,μ).

Proof By (2.2), 〈 ψ ′ ( u , v ) , ( u , v ) 〉 < 0 for ( u , v ) ∈ M ε , λ , μ . Since J ε , λ , μ ( u 0 , v 0 ) = min ( u , v ) ∈ M ε , λ , μ J ε , λ , μ ( u , v ) , by the Lagrange multiplier theorem, there is τ ∈ R such that J ε , λ , μ ′ ( u 0 , v 0 ) = τ ψ ′ ( u 0 , v 0 ) in H − 1 . Then we have

0 = 〈 J ε , λ , μ ′ ( u 0 , v 0 ) , ( u 0 , v 0 ) 〉 = τ 〈 ψ ′ ( u 0 , v 0 ) , ( u 0 , v 0 ) 〉 .

It follows that τ = 0 and J ε , λ , μ ′ ( u 0 , v 0 ) = 0 in H−1. Therefore, (u0,v0) is a nontrivial solution of (Eε,λ,μ) and J ε , λ , μ ( u 0 , v 0 ) = θ ε , λ , μ . □

First of all, we define the Palais-Smale (denoted by (PS)) sequence and (PS)-condition in H for some functional J.

Definition 3.1 (i) For γ ∈ R , a sequence { ( u n , v n ) } is a (PS)_γ-sequence in H for J if J ( u n , v n ) = γ + o n ( 1 ) and J ′ ( u n , v n ) = o n ( 1 ) strongly in H−1 as n → ∞ , where H−1 is the dual space of H;

(ii) J satisfies the (PS)_γ-condition in H if every (PS)_γ-sequence in H for J contains a convergent subsequence.

Applying Ekeland’s variational principle and using the same argument as in Cao-Zhou 8 or Tarantello 9, we have the following lemma.

Lemma 3.2 (i) There exists a (PS)-sequence { ( u n , v n ) } in Mε,λ,μ for J ε , λ , μ .

In order to prove the existence of positive solutions, we want to prove that Jε,λ,μ satisfies the (PS)_γ-condition in H for γ ∈ ( 0 , p − 2 2 p ( S α , β ) p / ( p − 2 ) ( f ∞ ) 2 / ( p − 2 ) ) .

Lemma 3.3 Jε,λ,μ satisfies the (PS)_γ-condition in H for γ ∈ ( 0 , p − 2 2 p ( S α , β ) p / ( p − 2 ) ( f ∞ ) 2 / ( p − 2 ) ) .

Proof Let {(un,vn)} be a (PS)_γ-sequence in H for Jε,λ,μ such that J ε , λ , μ ( u n , v n ) = γ + o n ( 1 ) and J ε , λ , μ ′ ( u n , v n ) = o n ( 1 ) in H−1. Then

γ + c n + d n ∥ ( u n , v n ) ∥ H q ≥ J ε , λ , μ ( u n , v n ) − 1 q 〈 J ε , λ , μ ′ ( u n , v n ) , ( u n , v n ) 〉 = ( 1 2 − 1 q ) ∥ ( u n , v n ) ∥ H 2 + ( 1 q − 1 p ) ∫ R N f ( ε z ) | u n | α | v n | β d z ≥ q − 2 2 q ∥ ( u n , v n ) ∥ H 2 ,

where c n = o n ( 1 ) , d n = o n ( 1 ) as n→∞. It follows that {(un,vn)} is bounded in H. Hence, there exist a subsequence { ( u n , v n ) } and (u,v)∈H such that

Moreover, we have that J ε , λ , μ ′ ( u , v ) = 0 in H−1. We use the Brézis-Lieb lemma to obtain (3.1) and (3.2) as follows:

Next, we claim that

∫ R N g ( ε z ) | u n − u | q d z → 0 as  n → ∞

and

∫ R N h ( ε z ) | v n − v | q d z → 0 as  n → ∞ .

Since g ∈ L m ( R N ) , where m = p / ( p − q ) , then for any σ > 0 , there exists r > 0 such that ∫ [ B r N ( 0 ) ] c g ( ε z ) p p − q d z < σ . By the Hölder inequality and the Sobolev embedding theorem, we get

Similarly, ∫ R N h ( ε z ) | v n − v | q d z → 0 as n→∞. By (A1) and un→u, vn→v strongly in L loc p ( R N ) , we have that

∫ R N f ( ε z ) | u n − u | α | v n − v | β d z = ∫ R N f ∞ | u n − u | α | v n − v | β d z = o n ( 1 ) .

Let p n = ( u n − u , v n − v ) . By (3.1)-(3.4) and Lemma 2.1, we deduce that

∥ p n ∥ H 2 = ( ∥ u n ∥ H 2 + ∥ v n ∥ H 2 ) − ( ∥ u ∥ H 2 + ∥ v ∥ H 2 ) + o n ( 1 ) = ∫ R N f ( ε z ) | u n | α | v n | β d z + ∫ R N ( λ g ( ε z ) | u n | q + μ h ( ε z ) | v n | q ) d z − ∫ R N f ( ε z ) | u | α | v | β d z − ∫ R N ( λ g ( ε z ) | u | q + μ h ( ε z ) | v | q ) d z + o n ( 1 ) = ∫ R N f ( ε z ) | u n − u | α | v n − v | β d z + o n ( 1 ) ,

and

1 2 ∥ p n ∥ H 2 − 1 α + β ∫ R N f ( ε z ) | u n − u | α | v n − v | β d z = γ − J ε , λ , μ ( u , v ) + o n ( 1 ) .

We may assume that

∥ p n ∥ H 2 → l and ∫ R N f ( ε z ) | u n − u | α | v n − v | β d z → l as  n → ∞ .

Recall that

S α , β = inf u , v ∈ H 1 ( R N ) ∖ { ( 0 ) } ∥ ( u , v ) ∥ H 2 ( ∫ R N | u | α | v | β d z ) 2 / p , where  p = α + β .

If l > 0 , by (3.5), then

S α , β l 2 p = S α , β ( ∫ R N f ( ε z ) | u n − u | α | v n − v | β d z ) 2 / p + o n ( 1 ) = S α , β ( ∫ R N f ∞ | u n − u | α | v n − v | β d z ) 2 / p + o n ( 1 ) ≤ ( f ∞ ) 2 / p ∥ p n ∥ H 2 + o n ( 1 ) = ( f ∞ ) 2 / p l .

This implies l ≥ ( S α , β ) p / ( p − 2 ) / ( f ∞ ) 2 / ( p − 2 ) . By (3.6) and (3.7), we obtain that

γ = ( 1 2 − 1 p ) l + J ε , λ , μ ( u , v ) ≥ p − 2 2 p ( S α , β ) p / ( p − 2 ) ( f ∞ ) 2 / ( p − 2 ) ,

which is a contradiction. Hence, l = 0 , that is, ( u n , v n ) → ( u , v ) strongly in H. □

Let w∈H1(RN) be the unique, radially symmetric and positive ground state solution of equation (E0) in RN. Recall the facts (or see Bahri-Li 10, Bahri-Lions 11, Gidas-Ni-Nirenberg 12 and Kwong 13):

(i) w ∈ L ∞ ( R N ) ∩ C loc 2 , θ ( R N ) for some 0 < θ < 1 and lim | z | → ∞ w ( z ) = 0 ;

(ii) for any ε>0, there exist positive numbers C 1 , C 2 ε and C 3 ε such that for all z ∈ R N

C 2 ε exp ( − ( 1 + ε ) | z | ) ≤ w ( z ) ≤ C 1 exp ( − | z | )

and

| ∇ w ( z ) | ≤ C 3 ε exp ( − ( 1 − ε ) | z | ) .

By Lien-Tzeng-Wang 14, then

S p = ∫ R N ( | ∇ w | 2 + w 2 ) d z ( ∫ R N w p d z ) 2 / p .

For 1≤i≤k, we define

w ε i ( z ) = w ( z − a i ε ) , where  f ( a i ) = max z ∈ R N f ( z ) = 1 .

Clearly, w ε i ( z ) ∈ H 1 ( R N ) .

First of all, we want to prove that

lim ε → 0 + sup t ≥ 0 J ε , λ , μ ( t α w ε i , t β w ε i ) ≤ p − 2 2 p ( S α , β ) p / ( p − 2 ) uniformly in  i .

Lemma 4.1 For λ > 0 and μ > 0 , then

lim ε → 0 + sup t ≥ 0 J ε , λ , μ ( t α w ε i , t β w ε i ) ≤ p − 2 2 p ( S α , β ) p / ( p − 2 ) uniformly in   i .

Moreover, we have that

0 < θ ε , λ , μ ≤ p − 2 2 p ( S α , β ) p / ( p − 2 ) .

Proof Part I: Since Jε,λ,μ is continuous in H, J ε , λ , μ ( 0 , 0 ) = 0 , and { ( α w ε i , β w ε i ) } is uniformly bounded in H for any ε>0 and 1≤i≤k, then there exists t 0 > 0 such that for 0 ≤ t < t 0 and any ε>0,

J ε , λ , μ ( t α w ε i , t β w ε i ) < p − 2 2 p ( S α , β ) p / ( p − 2 ) uniformly in  i .

From (A1), we have that inf z ∈ R N f ( z ) > 0 . Then

J ε , λ , μ ( t α w ε i , t β w ε i ) ≤ t 2 2 ∥ ( α w , β w ) ∥ H 2 − t α + β α + β ( inf z ∈ R N f ( z ) ) ∫ R N | α w | α | β w | β d z → − ∞ as  t → ∞ .

It follows that there exists t 1 > 0 such that for t > t 1 and any ε>0,

J ε , λ , μ ( t α w ε i , t β w ε i ) < p − 2 2 p ( S α , β ) p / ( p − 2 ) uniformly in  i .

From now on, we only need to show that

lim ε → 0 + sup t 0 ≤ t ≤ t 1 J ε , λ , μ ( t w ε i ) ≤ p − 2 2 p ( S α , β ) p / ( p − 2 ) uniformly in  i .

Since

sup t ≥ 0 ( t 2 2 a − t α + β α + β b ) = α + β − 2 2 ( α + β ) ( a b 2 α + β ) α + β α + β − 2 , where  a , b > 0 ,

and by (4.1), then

For t 0 ≤ t ≤ t 1 , by (4.2), we have that

J ε , λ , μ ( t α w ε i , t β w ε i ) = t 2 2 ∥ ( α w ε i , β w ε i ) ∥ H 2 − t α + β α + β ∫ R N f ( ε z ) | α w ε i | α | β w ε i | β d z − t q q ∫ R N ( λ g ( ε z ) | α w ε i | q + μ h ( ε z ) | β w ε i | q ) d z ≤ p − 2 2 p ( S α , β ) p / ( p − 2 ) + t 1 p p ∫ R N ( 1 − f ( ε z ) ) | α w ε i | α | α w ε i | β d z .

Since

then

lim ε → 0 + sup t 0 ≤ t ≤ t 1 J ε , λ , μ ( t α w ε i , t β w ε i ) ≤ p − 2 2 p ( S α , β ) p / ( p − 2 ) ,

that is, for λ>0 and μ>0,

lim ε → 0 + sup t ≥ 0 J ε , λ , μ ( t α w ε i , t β w ε i ) ≤ p − 2 2 p ( S α , β ) p / ( p − 2 ) uniformly in  i .

Part II: By Lemma 2.4, there is a number t ε i > 0 such that ( t ε i α w ε i , t ε i β w ε i ) ∈ M ε , λ , μ , where 1≤i≤k. Hence, from the result of Part I, we have that for λ>0 and μ>0,

0 < θ ε , λ , μ ≤ lim ε → 0 + sup t ≥ 0 J ε , λ , μ ( t α w ε i , t β w ε i ) ≤ p − 2 2 p ( S α , β ) p / ( p − 2 ) .

 □

Proof of Theorem 1.1 By Lemma 3.2, there exists a (PS)-sequence { ( u n , v n ) } in Mε,λ,μ for Jε,λ,μ. Since 0 < θ ε , λ , μ ≤ p − 2 2 p ( S α , β ) p / ( p − 2 ) < p − 2 2 p ( S α , β ) p / ( p − 2 ) ( f ∞ ) 2 / ( p − 2 ) for λ>0 and μ>0, by Lemma 3.3, there exist a subsequence {(un,vn)} and ( u 0 , v 0 ) ∈ H such that ( u n , v n ) → ( u 0 , v 0 ) strongly in H. It is easy to check that (u0,v0) is a nontrivial solution of (Eε,λ,μ) and Jε,λ,μ(u0,v0)=θε,λ,μ. Since J ε , λ , μ ( u 0 , v 0 ) = J λ , μ ( | u 0 | , | v 0 | ) and ( | u 0 | , | v 0 | ) ∈ M ε , λ , μ , by Lemma 2.6, we may assume that u 0 ≥ 0 , v 0 ≥ 0 . Applying the maximum principle, u 0 > 0 and v 0 > 0 in Ω. □