In this article, we consider the following problem

( P ) a ∫ Ω 1 p ( x ) ( | ∇ u | p ( x ) + | u | p ( x ) ) d x ( - d i v ( | ∇ u | p ( x ) - 2 ∇ u ) + | u | p ( x ) - 2 u ) = b ∫ Ω F ( x , u ) d x f ( x , u ) in Ω a ∫ Ω 1 p ( x ) ( | ∇ u | p ( x ) + | u | p ( x ) ) d x | ∇ u | p ( x ) - 2 ∂ u ∂ ν = g ( x , u ) on ∂ Ω ,

where Ω is a smooth bounded domain in R^N, p ∈ C Ω ̄ with 1 < p^- := inf_Ω p(x) ≤ p(x) ≤ p⁺ := sup_Ω p(x) < N, a(t) is a continuous real-valued function, f : Ω × R → R, g : ∂Ω × R → R satisfy the Caratheodory condition, and F ( x , u ) = ∫ 0 u f ( x , t ) d t . Since the equation contains an integral related to the unknown u over Ω, it is no longer an identity pointwise, and therefore is often called nonlocal problem.

Kirchhoff 1 has investigated an equation

ρ ∂ 2 u ∂ t 2 - P 0 h + E 2 L ∫ 0 L ∂ u ∂ x 2 d x ∂ 2 u ∂ x 2 = 0 ,

which is called the Kirchhoff equation. Various equations of Kirchhoff type have been studied by many authors, especially after the work of Lions 2, where a functional analysis framework for the problem was proposed; see e.g. 3456 for some interesting results and further references. In the following, a key work on nonlocal elliptic problems is the article by Chipot and Rodrigues 7. They studied nonlocal boundary value problems and unilateral problems with several applications. And now the study of nonlocal elliptic problem has already been extended to the case involving the p-Laplacian; see e.g. 89. Recently, Autuori, Pucci and Salvatori 10 have investigated the Kirchhoff type equation involving the p(x)-Laplacian of the form

u t t - M ∫ Ω 1 p ( x ) | ∇ u | p ( x ) d x Δ p ( x ) u + Q ( t , x , u , u t ) + f ( x , u ) = 0 .

The study of the stationary version of Kirchhoff type problems has received considerable attention in recent years; see e.g. 5111213141516.

The operator Δ_p(x)u = div(|∇u|^p(x)-2∇u) is called p(x)-Laplacian, which becomes p-Laplacian when p(x) ≡ p (a constant). The p(x)-Laplacian possesses more complicated nonlinearities than p-Laplacian. The study of various mathematical problems with variable exponent are interesting in applications and raise many difficult mathematical problems. We refer the readers to 17181920212223 for the study of p(x)-Laplacian equations and the corresponding variational problems.

Corrêa and Figueiredo 13 presented several sufficient conditions for the existence of positive solutions to a class of nonlocal boundary value problems of the p-Kirchhoff type equation. Fan and Zhang 20 studied p(x)-Laplacian equation with the nonlinearity f satisfying Ambrosetti-Rabinowitz condition. The p(x)-Kirchhoff type equations with Dirichlet boundary value problems have been studied by Dai and Hao 24, and much weaker conditions have been given by Fan 25. The elliptic problems with nonlinear boundary conditions have attracted expensive interest in recent years, for example, for the Laplacian with nonlinear boundary conditions see 2627282930, for elliptic systems with nonlinear boundary conditions see 3132, for the p-Laplacian with nonlinear boundary conditions of different type see 3334353637, and for the p(x)-Laplacian with nonlinear boundary conditions see 383940. Motivated by above, we focus the case of nonlocal p(x)-Laplacian problems with nonlinear Neumann boundary conditions. This is a new topics even when p(x) ≡ p is a constant.

This rest of the article is organized as follows. In Section 2, we present some necessary preliminary knowledge on variable exponent Sobolev spaces. In Section 3, we consider the case where the energy functional associated with problem (P) is coercive. And in Section 4, we consider the case where the energy functional possesses the Mountain Pass geometry.

In order to discuss problem (P), we need some theories on variable exponent Sobolev space W^1,p(x)(Ω). For ease of exposition we state some basic properties of space W^1,p(x)(Ω) (for details, see 224142).

Let Ω be a bounded domain of R^N, denote

C + ( Ω ̄ ) = { p | p ∈ C ( Ω ̄ ) , p ( x ) > 1 , ∀ x ∈ Ω ̄ } , p + = max x ∈ Ω ̄ p ( x ) , p - = min x ∈ Ω ̄ p ( x ) , ∀ p ∈ C ( Ω ̄ ) , L p ( x ) ( Ω ) = u | u is a measurable real - valued function on Ω , ∫ Ω | u | p ( x ) d x < ∞ ,

we can introduce the norm on L^p(x) (Ω) by

u p ( x ) = inf λ > 0 : ∫ Ω u ( x ) λ p ( x ) d x ≤ 1

and (L^p(x) (Ω), | · |_p(x)) becomes a Banach space, we call it the variable exponent Lebesgue space.

The space W^{1, p(x)}(Ω) is defined by

W 1 , p ( x ) ( Ω ) = { u ∈ L p ( x ) ( Ω ) | | ∇ u | ∈ L p ( x ) ( Ω ) } ,

and it can be equipped with the norm

| | u | | = | u | p ( x ) + | ∇ u | p ( x ) ,

where |∇u|_p(x) = ||∇u||_p(x); and we denote by W 0 1 , p ( x ) ( Ω ) the closure of C 0 ∞ ( Ω ) in W^{1, p(x)}(Ω), p * = N p ( x ) N - p ( x ) , p * = ( N - 1 ) p ( x ) N - p ( x ) , when p(x) < N, and p* = p_* = ∞, when p(x) > N.

Proposition 2.1 2241. (1) If p ∈ C + ( Ω ¯ ) , the space (L^p(x) (Ω), | · |_p(x)) is a separable, uniform convex Banach space, and its dual space is L^q(x) (Ω), where 1/q(x) + 1/p(x) = 1. For any u ∈ L^p(x) (Ω) and v ∈ L^q(x) (Ω), we have

∫ Ω u v d x ≤ ( 1 p - + 1 q - ) | u | p ( x ) | v | q ( x ) ;

(2) If p 1 , p 2 ∈ C + ( Ω ¯ ) , p₁ (x) ≤ p₂ (x), for any x ∈ Ω, then L p 2 ( x ) ( Ω ) ↪ L p 1 ( x ) ( Ω ) , and the imbedding is continuous.

Proposition 2.2 22. If f : Ω × R → R is a Caratheodory function and satisfies

| f ( x , s ) | ≤ d ( x ) + e | s | p 1 ( x ) p 2 ( x ) , f o r a n y x ∈ Ω , s ∈ R ,

where p 1 , p 2 ∈ C + ( Ω ¯ ) , d ∈ L p 2 ( x ) ( Ω ) , d(x) ≥ 0 and e ≥ 0 is a constant, then the superposition operator from L p 1 ( x ) ( Ω ) to L p 2 ( x ) ( Ω ) defined by (N_f (u)) (x) = f (x, u (x)) is a continuous and bounded operator.

Proposition 2.3 22. If we denote

ρ ( u ) = ∫ Ω | u | p ( x ) d x , ∀ u ∈ L p ( x ) ( Ω ) ,

then for u, u_n ∈ L^p(x) (Ω)

(1) |u (x)|_p(x) < 1(= 1; > 1) ⇔ρ (u) < 1(= 1; > 1);

(2) | u ( x ) | p ( x ) > 1 ⇒ | u | p ( x ) p - ≤ ρ ( u ) ≤ | u | p ( x ) p + ; | u ( x ) | p ( x ) < 1 ⇒ | u | p ( x ) p - ≥ ρ ( u ) ≥ | u | p ( x ) p + ;

(3) | u n ( x ) | p ( x ) → 0 ⇔ ρ ( u n ) → 0 as n → ∞ ; | u n ( x ) | p ( x ) → ∞ ⇔ ρ ( u n ) → ∞ as n → ∞ .

Proposition 2.4 22. If u, u_n ∈ L^p(x) (Ω), n = 1, 2, ..., then the following statements are equivalent to each other

(1) lim_{k → ∞} |u_k - u|_p(x) = 0;

(2) lim_{k → ∞} ρ |u_k - u| = 0;

(3) u_k → u in measure in Ω and lim_{k → ∞} ρ (u_k) = ρ (u).

Proposition 2.5 22. (1) If p ∈ C + ( Ω ¯ ) , then W 0 1 , p ( x ) ( Ω ) and W^1,p(x)(Ω) are separable reflexive Banach spaces;

(2) if q ∈ C + ( Ω ¯ ) and q (x) < p* (x) for any x ∈ Ω ¯ , then the imbedding from W^{1, p(x)}(Ω) to L^q(x) (Ω) is compact and continuous;

(3) if q ∈ C+(Ω¯) and q (x) < p_* (x) for any x ∈Ω¯ , then the trace imbedding from W^{1, p(x)}(Ω) to L^q(x) (∂Ω)is compact and continuous;

(4) (Poincare inequality) There is a constant C > 0, such that

| u | p ( x ) ≤ C | ∇ u | p ( x ) ∀ u ∈ W 0 1 , p ( x ) ( Ω ) .

So, |∇u|_p(x) is a norm equivalent to the norm || u || in the space W 0 1 , p ( x ) ( Ω ) .

In this and the next sections we consider the nonlocal p(x)-Laplacian-Neumann problem (P), where a and b are two real functions satisfying the following conditions

(a₁) a : (0, + ∞) → (0, + ∞) is continuous and a ∈ L¹ (0, t) for any t > 0.

(b₁) b : R → R is continuous.

Notice that the function a satisfies (a₁) may be singular at t = 0. And f, g satisfying

(f_l) f : Ω × R → R satisfies the Caratheodory condition and there exist two constants C₁ ≥ 0, C₂ ≥ 0 such that

| f ( x , t ) | ≤ C 1 + C 2 | t | q 1 ( x ) - 1 , ∀ ( x , t ) ∈ Ω × R ,

where q 1 ∈ C + ( Ω ¯ ) and q₁ (x) < p* (x), ∀ x ∈ Ω ¯ .

(g₁) g : ∂Ω × R → R satisfies the Caratheodory condition and there exist two constants C 1 ′ ≥ 0 , C 2 ′ ≥ 0 such that

| g ( x , t ) | ≤ C 1 ′ + C 2 ′ | t | q 2 ( x ) - 1 , ∀ ( x , t ) ∈ ∂ Ω × R ,

where q₂ ∈ C₊ (∂Ω) and q₂ (x) < p_* (x), ∀x ∈ ∂Ω. For simplicity we write X = W^{1, p(x)}(Ω), denote by C the general positive constant (the exact value may change from line to line).

Define

a ^ ( t ) = ∫ 0 t a ( s ) d s , b ^ ( t ) = ∫ 0 t b ( s ) d s , ∀ t ∈ R , I 1 ( u ) = ∫ Ω 1 p ( x ) | ∇ u | p ( x ) + | u | p ( x ) d x , I 2 ( u ) = ∫ Ω F ( x , u ) d x , ∀ u ∈ X , J ( u ) = a ^ ( I 1 ( u ) ) = a ^ ∫ Ω 1 p ( x ) | ∇ u | p ( x ) + | u | p ( x ) d x , Φ ( u ) = b ^ ( I 2 ( u ) ) = b ^ ∫ Ω F ( x , u ) d x and Ψ ( u ) = ∫ ∂ Ω G ( x , u ) d σ , ∀ u ∈ X , E ( u ) = J ( u ) - Φ ( u ) - Ψ ( u ) , ∀ u ∈ X , ,

where F ( x , u ) = ∫ 0 u f ( x , t ) d t , G ( x , u ) = ∫ 0 u g ( x , t ) d t .

Lemma 3.1. Let (f₁), (g₁) (a₁) and (b₁) hold. Then the following statements hold true:

(1) a ^ ∈ C 0 ( [ 0 , ∞ ) ) ∩ C 1 ( ( 0 , ∞ ) ) , a ^ ( 0 ) = 0 , a ^ ′ ( t ) = a ( t ) > 0 ; b ^ ∈ C 1 ( R ) , b ^ ( 0 ) = 0 .

(2) J, Φ, Ψ and E ∈ C⁰ (X), J (0) = Φ (0) = Ψ (0) = E (0) = 0. Furthermore J ∈ C¹ (X\{0}), Φ, Ψ ∈ C¹ (X), E ∈ C¹ (X\{0}). And for every u ∈ X\{0}, v ∈ X, we have

E ′ ( u ) v = a ∫ Ω 1 p ( x ) | ∇ u | p ( x ) + | u | p ( x ) d x ∫ Ω | ∇ u | p ( x ) - 2 ∇ u ∇ v + | u | p ( x ) - 2 u v d x - b ∫ Ω F ( x , u ) d x ∫ Ω f ( x , u ) v d x - ∫ ∂ Ω g ( x , u ) v d σ .

Thus u ∈ X\{0} is a (weak) solution of (P) if and only if u is a critical point of E.

(3) The functional J : X → R is sequentially weakly lower semi-continuous, Φ, Ψ: X → R are sequentially weakly continuous, and thus E is sequentially weakly lower semi-continuous.

(4) The mappings Φ' and Ψ' are sequentially weakly-strongly continuous, namely, u_n ⇀ u in X implies Φ' (u_n) → Φ' (u) in X*. For any open set D ⊂ X\{0} with D ¯ ⊂ X \ { 0 } , The mappings J' and E ′ : D ¯ → X * are bounded, and are of type (S₊), namely,

u n ⇀ u   and   lim ¯ n → ∞ J ′ ( u n )   ( u n − u ) ≤ 0 ,   implies   u n → u .

Definition 3.1. Let c ∈ R, a C¹-functional E : X → R satisfies (P.S)_c condition if and only if every sequence {u_j} in X such that lim_j E (u_j) = c, and lim_j E' (u_j) = 0 in X* has a convergent subsequence.

Lemma 3.2. Let (f₁), (g₁), (a₁), (b₁) hold. Then for any c ≠ 0, every bounded (P. S)_c sequence for E, i.e., a bounded sequence {u_n} ⊂ X\{0} such that E (u_n) → c and E' (u_n) → 0, has a strongly convergent subsequence.

The proof of these two lemmas can be obtained easily from 2540, we omitted them here.

Theorem 3.1. Let (f₁), (g₁), (a₁), (b₁) and the following conditions hold true:

(a₂) There are positive constants α₁, M, and C such that a ^ ( t ) ≥ C t α 1 for t ≥ M.

(b₂) There are positive constants β₁ and C such that | b ^ ( t ) | ≤ C + C | t | β 1 for t ∈ R.

(H₁) β₁ q₁₊ < α₁ p_-, q₂₊ < α₁p_-.

Then the functional E is coercive and attains its infimum in X at some u₀ ∈ X. Therefore, u₀ is a solution of (P) if E is differentiable at u₀.

Proof. For || u || large enough, by (f₁), (g₁), (a₂), (b₂) and (H₁), we have that

J ( u ) = a ^ ( I 1 ( u ) ) = a ^ ∫ Ω 1 p ( x ) | ∇ u | p ( x ) + | u | p ( x ) d x ≥ a ^ ( C 1 | | u | | p - ) ≥ C 2 | | u | | α 1 p - , ∫ Ω F ( x , u ) d x ≤ C 3 | | u | | q 1 + , Φ ( u ) = b ^ ( I 2 ( u ) ) = b ^ ∫ Ω F ( x , u ) d x ≤ C 4 | | u | | β 1 q 1 + + C 4 ̃ , Ψ ( u ) = ∫ ∂ Ω G ( x , u ) d σ ≤ C 5 | | u | | q 2 + + C 5 ̃ , E ( u ) = J ( u ) - Φ ( u ) - Ψ ( u ) ≥ C 2 | | u | | α 1 p - - C 4 | | u | | β 1 q 1 + - C 5 | | u | | q 2 + - + C 6 ̃ ,

and hence E is coercive. Since E is sequentially weakly lower semi-continuous and X is reflexive, E attains its infimum in X at some u₀ ∈ X. In this case E is differentiable at u₀, then u₀ is a solution of (P).

Theorem 3.2. Let (f₁), (g₁), (a₁), (b₁), (a₂), (b₂), (H₁) and the following conditions hold true:

(a₃) There is a positive constant α₂ such that lim   sup t → 0 + a ^ ( t ) t α 2 < + ∞ .

(b₃) There is a positive constant β₂ such that lim   inf t → 0 b ^ ( t ) | t | β 2 > 0 .

(f₂) There exist an open subset Ω₀ of Ω and r₁ > 0 such that lim   inf t → 0 F ( x ,   t ) | t | r 2 > 0 uniformly for x ∈ Ω₀.

(g₂) There exists r₂ > 0 such that lim   inf t → 0 G ( x ,   t ) | t | r 2 > 0 uniformly for x ∈ ∂Ω.

(H₂) β₂r₁ < α₂ p_-, r₂ < α₂ p_-.

Then (P) has at least one nontrivial solution which is a global minimizer of the energy functional E.

Proof. From Theorem 3.1 we know that E has a global minimizer u₀. It is clear that a ^ ( 0 ) = 0 , b ^ ( 0 ) = 0 , F (x, 0) and consequently E (0) = 0. Take w ∈ C 0 ∞ ( Ω 0 ) \ { 0 } . Then, by (f₂), (g₂) (a₃), (b₃) and (H₂), for sufficiently small λ > 0 we have that

E ( λ w ) = a ^ ∫ Ω λ p ( x ) p ( x ) | ∇ w | p ( x ) + | w | p ( x ) d x - b ^ ∫ Ω F ( x , λ w ) d x - ∫ ∂ Ω G ( x , λ w ) d σ ≤ C 1 ∫ Ω λ p ( x ) p ( x ) | ∇ w | p ( x ) + | w | p ( x ) d x α 2 - C 2 ∫ Ω 0 F ( x , λ w ) d x ) β 2 - C 3 ∫ ∂ Ω | λ w | r 2 d σ ≤ C 4 λ α 2 p - - C 5 λ β 2 r 1 - C 6 λ r 2 < 0 .

Hence E (u₀) < 0 and u₀ ≠ 0.

By the genus theorem, similarly in the proof of Theorem 4.3 in 18, we have the following:

Theorem 3.3. Let the hypotheses of Theorem 3.2 hold, and let, in addition, f and g satisfy the following conditions:

(f₃) f (x, - t) = - f (x, t) for x ∈ Ω and t ∈ R.

(g₃) g (x, - t) = - g (x, t) for x ∈ ∂Ω and t ∈ R.

Then (P) has a sequence of solutions {u_n} such that E(u_n) < 0.

Theorem 3.4. Let (f₁), (g₁), (a₁), (b₁), (a₂), (b₂), (a₃), (b₃), (H₁), (H₂) and the following conditions hold true:

(b₊) b(t) ≥ 0 for t ≥ 0.

(f₊) f(x, t) ≥ 0 for x ∈ Ω and t ≥ 0.

(g₊) g(x, t) ≥ 0 for x ∈ ∂Ω and t ≥ 0.

(f₂)₊There exist an open subset Ω₀ of Ω and r₁ > 0 such that lim inf t → 0 + F ( x , t ) t r 1 > 0 uniformly for x ∈ Ω₀.

(g₂)₊ There exists r₂ > 0 such that lim inf t → 0 + G ( x , t ) t r 2 > 0 uniformly for x ∈ ∂Ω.

Then (P) has at least one nontrivial nonnegative solution with negative energy.

Proof. Define

f ̃ ( x , t ) = f ( x , t ) if  t ≥ 0 , f ( x , 0 ) if t < 0 , g ̃ ( x , t ) = g ( x , t ) if t ≥ 0 , g ( x , 0 ) if t < 0 ,

F ̃ ( x , t ) = ∫ 0 t f ̃ ( x , s ) d s , ∀ x ∈ Ω , t ∈ R , G ̃ ( x , t ) = ∫ 0 t g ̃ ( x , s ) d s , ∀ x ∈ ∂ Ω , t ∈ R ,

b ̃ ( t ) = b ( t ) if  t ≥ 0 , b ( 0 ) if  t < 0 , b ̃ ^ ( t ) = ∫ 0 t b ̃ ( s ) d s , ∀ t ∈ R ,

E ˜ ( u ) = a ^ ( ∫ Ω 1 p ( x ) ( | ∇ u | p ( x ) + | u | p ( x ) ) d x ) − b ˜ ^ ( ∫ Ω F ˜ ( x , u ) d x ) − ∫ ∂ Ω G ˜ ( x , u ) d σ , ∀ u ∈ X

Then, using truncation functions above, similarly in the proof of Theorem 3.4 in 25, we can prove that E ̃ has a nontrivial global minimizer u₀ and u₀ is a nontrivial nonnegative solution of (P).

In this section we will find the Mountain Pass type critical points of the energy functional E associated with problem (P).

Lemma 4.1. Let (f₁), (g₁), (a₁), (b₁) and the following conditions hold true:

( a 2 ) ′ ∃ α 1 > 0 , M > 0, and C > 0 such that

a ^ ( t ) ≥ C t α 1 for all t ≥ M

with α₁p_- > 1.

(a₄) ∃ λ > 0, M > 0 such that

λ a ^ ( t ) ≥ a ( t ) t for all t ≥ M

(b₄) ∃θ > 0, M > 0 such that:

0 ≤ θ b ^ ( t ) ≤ b ( t ) t , for all t ≥ M.

(f₄) ∃μ > 0, M > 0 such that:

0 ≤ μF(x, t) ≤ f(x, t)t, for |t| ≥ M and x ∈ Ω.

(g₄) ∃κ > θμ > 0, M > 0 such that:

0 ≤ κG(x, t) ≤ g(x, t)t, |t| ≥ M and x ∈ ∂Ω.

(H₃) λp₊ < θμ.

Then E satisfies condition (P.S)c for any c ≠ 0.

Proof. By (a₄), for ||u|| large enough,

λ p + J ( u ) = λ p + a ^ ( ∫ Ω 1 p ( x ) ( | ∇ u | p ( x ) + | u | p ( x ) ) d x ) ≥ p + a ( ∫ Ω 1 p ( x ) ( | ∇ u | p ( x ) + | u | p ( x ) ) d x ) ∫ Ω 1 p ( x ) ( | ∇ u | p ( x ) + | u | p ( x ) ) d x ≥ a ( ∫ Ω 1 p ( x ) ( | ∇ u | p ( x ) + | u | p ( x ) ) d x ) ∫ Ω ( | ∇ u | p ( x ) + | u | p ( x ) ) d x = J ′ ( u ) u .

From (f₄) and (g₄) we can see that there exists C₁ > 0 and C₂ > 0 such that

- C 1 ≤ μ ∫ Ω F ( x , u ) d x ≤ ∫ Ω f ( x , u ) u d x + C 1 , ∀ u ∈ X , - C 2 ≤ κ ∫ ∂ Ω G ( x , u ) d σ ≤ ∫ ∂ Ω g ( x , u ) u d σ + C 2 , ∀ u ∈ X ,

and thus, given any ε ∈ (0, μ), there exists M_ε ≥ M > 0 and M ε ′ ≥ M > 0 such that

( μ - ε ) ∫ Ω F ( x , u ) d x ≤ ∫ Ω f ( x , u ) u d x , if ∫ Ω F ( x , u ) d x ≥ M ε , θ ( μ - ε ) ∫ ∂ Ω G ( x , u ) d σ ≤ ∫ ∂ Ω g ( x , u ) u d σ , if ∫ ∂ Ω G ( x , u ) d σ ≥ M ′ ε .

We may assume M ε > c 1 μ and M ε ′ > c 2 θ μ . Note that in this case the inequalities ∫ Ω F ( x , u ) d x ≥ M ε and ∫ ∂ Ω G ( x , u ) d σ ≥ M ε ′ are equivalent to | ∫ Ω F ( x , u ) d x | ≥ M ε and | ∫ ∂ Ω G ( x , u ) d σ | ≥ M ε ′ , because ∫ Ω F ( x , u ) d x ≥ - C 1 μ and ∫ ∂ Ω G ( x , u ) d σ ≥ - c 2 θ μ for all u ∈ X. We claim that there exist C_ε > 0 and C ε ′ > 0 such that

Φ ′ ( u ) u - θ ( μ - ε ) Φ ( u ) ≥ - C ε  for  u ∈ X , Ψ ′ ( u ) u - θ ( μ - ε ) Ψ ( u ) ≥ - C ′ ε  for  u ∈ X .

Indeed, when | ∫ Ω F ( x , u ) d x | ≤ M ε and | ∫ ∂ Ω G ( x , u ) d σ | ≤ M ε ′ , the validity is obvious. When | ∫ Ω F ( x , u ) d x | ≥ M ε and | ∫ ∂ Ω G ( x , u ) d σ | ≥ M ε ′ , i.e., ∫ Ω F ( x , u ) d x ≥ M ε and ∫ ∂ Ω G ( x , u ) d σ ≥ M ε ′ , we have that

θ ( μ - ε ) Φ ( u ) = θ ( μ - ε ) b ^ ∫ Ω F ( x , u ) d x ≤ ( μ - ε ) b ∫ Ω F ( x , u ) d x ∫ Ω F ( x , u ) d x ≤ b ∫ Ω F ( x , u ) d x ∫ Ω f ( x , u ) u d x = Φ ′ ( u ) u ,

and

θ ( μ - ε ) Ψ ( u ) = θ ( μ - ε ) ∫ ∂ Ω G ( x , u ) d σ ≤ ∫ ∂ Ω g ( x , u ) u d σ = Ψ ′ ( u ) u .

Now let {u_n} ⊂ X\{0}, E(u_n) → c ≠ 0 and E'(u_n) → 0. By (H₃), there exists ε > 0 small enough such that λp₊ < θ(μ - ε). Then, since {u_n} is a (P.S)_c sequence, for sufficiently large n, we have

θ ( μ - ε ) c + 1 + ∥ u n ∥ ≥ θ ( μ - ε ) E ( u n ) - E ′ ( u n ) u n ≥ ( θ ( μ - ε ) - λ p + ) J ( u n ) + ( λ p + J ( u n ) - J ′ ( u n ) u n ) + ( Φ ′ ( u n ) u n - θ ( μ - ε ) Φ ( u n ) ) + ( Ψ ′ ( u n ) u n - θ ( μ - ε ) Ψ ( u n ) ) ≥ C 3 ∥ u n ∥ α 1 p - - C 4 - C ε - C ′ ε

Since α₁p_- > 1, we have that {||u_n||} is bounded. By Lemma 3.2, E satisfies condition (P.S)_c for c ≠ 0.

Theorem 4.1. Under the hypotheses of Lemma 4.1, and let the following conditions hold:

(a₅) There is a positive constant α₃ such that lim sup t → 0 + a ^ ( t ) t α 3 > 0 .

(b₅) There is a positive constant β₃ such that lim inf t → 0 b ^ ( t ) | t | β 3 < + ∞ .

(f₅) There exists r 1 ∈ C 0 ( Ω ¯ ) such that 1 < r₁(x) < p^*(x) for x ∈ Ω ¯ and lim inf t → 0 | F ( x , t ) | | t | r 1 ( x ) < + ∞ uniformly for x ∈ Ω.

(g₅) There exists such r 2 ∈ C 0 ( Ω ¯ ) such that 1 < r₂(x) < p_*(x) for x ∈ ∂ Ω and lim inf t → 0 | G ( x , t ) | | t | r 2 ( x ) < + ∞ uniformly for x ∈ ∂ Ω.

(H₄) α₃p₊ < β₃r_1-, α₃p₊ < r_2-, λp₊ < θμ.

Then (P) has a nontrivial solution with positive energy.

Proof. Let us prove this conclusion by the Mountain Pass lemma. E satisfies condition (P.S)_c for c ≠ 0 has been proved in Lemma 4.1.

For ||u|| small enough, from (a₅) we can obtain easily that J ( u ) ≥ C 1 ∥ u ∥ α 3 p + , from (b₅), (f₁) and (f₅) we have| Φ ( u ) | ≤ C 2 ∥ u ∥ β 3 r 1 - , and in the similar way from(g₁) and (g₅) we have | Ψ ( u ) | ≤ C 2 ∥ u ∥ r 2 - . Thus by (H₄), we conclude that there exist positive constants ρ and δ such that E(u) ≥ for ||u|| = ρ.

Let w ∈ X\{0} be given. From (a₄) for sufficiently large t > 0 we have a ^ ( t ) ≤ C 1 t λ , which follows that J ( s w ) ≤ d 1 s λ p + for s large enough, where d₁ is a positive constant depending on w. From (f₄) and (f₁) for |t| large enough we have ∫ Ω F ( x , s w ) d x ≥ d 2 s μ for s large enough, where d₂ is a positive constant depending on w. From (b₄) for t large enough we have Φ ( s w ) = b ^ ( ∫ Ω F ( x , s w ) d x ) ≥ d 3 s θ μ for s large enough, where d₃ is a positive constant depending on w. From (g₄) and (g₁) for |t| large enough we have Ψ ( s w ) = ∫ ∂ Ω G ( x , s w ) d σ ≥ d 4 s θ μ . Hence for any w ∈ X\{0} and s large enough, E ( s w ) ≤ d 1 s λ p + - d 3 s θ μ - d 4 s θ μ , thus by (H₃), We conclude that E(sw) → -∞ as s → +∞.

So by the Mountain Pass lemma this theorem is proved.

By the symmetric Mountain Pass lemma, similarly in the proof of Theorem 4.8 in 40, we have the following:

Theorem 4.2. Under the hypotheses of Theorem 4.1, if, in addition, (f₃) and (g₃) are satisfied, then (P) has a sequence of solutions {±u_n} such that E(±u_n) → +∞ as n → ∞.

The authors declare that they have no competing interests.

EG and PZ contributed to each part of this work equally. All the authors read and approved the final manuscript.