Multiplicity and boundedness of solutions for quasilinear elliptic equations on Heisenberg group

Abstract

In this paper, we study a class of quasilinear elliptic equations on Heisenberg group by using the nonsmooth critical point theory. Under some weaker assumptions, the multiplicity and boundedness of solutions for these equations are obtained.

1 Introduction

Let $\mathbb{H}^{N}$ be the space $\mathbb{R}^{N}\times \mathbb{R}^{N}\times\mathbb{R}$. Then $\mathbb{H}^{N}$ is a Lie group by the following group operation:

$$(x,y,t)\circ\bigl(x',y',t'\bigr)= \bigl(x+x',y+y',t+t'+2\bigl(x' \cdot y-x\cdot y'\bigr)\bigr), $$

where ‘⋅’ represents the usual inner-product in $\mathbb{R}^{N}$. The vector fields $X_{1},\ldots,X_{N}$, $Y_{1},\ldots ,Y_{N}$, and T given by

$$X_{j}=\frac{\partial}{\partial x_{j}}+2y_{j}\frac{\partial }{\partial t},\qquad Y_{j}=\frac{\partial}{\partial y_{j}}-2x_{j}\frac{\partial }{\partial t},\qquad T= \frac{\partial}{\partial t} $$

form a basis for the tangent space at $\eta=(x,y,t)$.

Definition 1.1

The Heisenberg Laplacian is defined by

$$\Delta_{\mathbb{H}}=\sum_{j=1}^{N} \bigl(X_{j}^{2}+Y_{j}^{2}\bigr). $$

Denote $\nabla_{\mathbb{H}}u$ as the 2N-vector $(X_{1}u,\ldots ,X_{N}u,Y_{1}u,\ldots,Y_{N}u)$, and then $\operatorname{div}_{\mathbb{H}}\vec{F}=X_{1}F_{1}+\cdots +X_{N}F_{N}+Y_{1}G_{1}+\cdots+Y_{N}G_{N}$, where $\vec {F}=(F_{1}, \ldots, F_{N},G_{1},\ldots,G_{N})$.

In this paper, we will study the multiplicity and boundedness of solutions for the equation

$$ -\operatorname{div}_{\mathbb{H}}J_{\xi}(\eta,u,\nabla_{\mathbb {H}}u)+J_{s}( \eta,u,\nabla_{\mathbb{H}}u)+\bigl(b(\eta)-\lambda \bigr)u=f(\eta,u),\quad \eta \in\mathbb{H}^{N}, $$

(1.1)

where $N>2$, $\lambda\in\mathbb{R}$, and $b(\cdot)$ is a continuous function, satisfying $b(\eta)\geq0$ for all $\eta\in\mathbb{H}^{N}$ and $\lim_{|\eta|_{\mathbb{H}^{N}}\rightarrow\infty}b(\eta )=+\infty$.

There have been a number of papers concerned with the existence and multiplicity of solutions for nonlinear equations or systems, such as [1–12].

In [1], Aouaoui established the existence of infinitely many solutions for the problem

$$-\operatorname{div}\bigl(A(x,u)\nabla u\bigr)+\frac{1}{2}A_{s}(x,u)| \nabla u|^{2}+\bigl(b(x)-\lambda\bigr)u=f(x,u), \quad x\in \mathbb{R}^{N}. $$

In [13], Pellacci and Squassina studied the quasilinear elliptic problem

$$-\operatorname{div}\bigl(j_{\xi}(x,u,\nabla u)\bigr)+j_{s}(x,u, \nabla u)=g(x,u) \quad \mbox{in } \Omega, $$

with homogeneous boundary and bounded open set $\Omega\subset\mathbb{R}^{N}$.

Set

$$E= \biggl\{ u\in L^{2}\bigl(\mathbb{H}^{N}\bigr)\Bigm|\int _{\mathbb{H}^{N}}\bigl(b(\eta )|u|^{2}+|\nabla_{\mathbb{H}}u|^{2} \bigr) < \infty \biggr\} . $$

We will use the variational methods to solve the problem of (1.1). Explicitly, we will look for critical points of the functional $I:E\rightarrow\mathbb{R}$,

$$ I(u)=\int_{\mathbb{H}^{N}}J(\eta,u,\nabla_{\mathbb{H}}u)+ \frac {1}{2}\int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda \bigr)u^{2}-\int_{\mathbb{H}^{N}}F(\eta,u), $$

(1.2)

where $F(\eta,\xi)=\int_{0}^{\xi}f(\eta,t)\, dt$. The main difficulty in this problem is that the functional is continuous but not differentiable in whole space E. Nevertheless, the derivatives of I exist along the directions of $E\cap L^{\infty}(\mathbb{H}^{N})$.

Remark 1.1

$E\hookrightarrow L^{p}(\mathbb{H}^{N})$, when $2\leq p\leq2^{*}$; $E\hookrightarrow\hookrightarrow L^{p}(\mathbb {H}^{N})$, when $2\leq p<2^{*}$, where ${{2^{*}=\frac{2Q}{Q-2}}}$, $Q=2N+2$.

Proof

When $2\leq p\leq2^{*}$, it is obvious that $E\hookrightarrow L^{p}(\mathbb{H}^{N})$ by Folland-Stein embedding theorem [11].

It is sufficient to prove the conclusion when $p=2$. Let $\{u_{n}\}$ be a weakly convergent sequence to zero in E. Since $\lim_{|\eta|_{\mathbb{H}^{N}}\rightarrow\infty }b(\eta )=+\infty$, for any $\varepsilon>0$ there exists $M_{\varepsilon}>0$, such that ${\frac{1}{b(\eta)}}<\varepsilon$ for any $|\eta |_{\mathbb{H}^{N}}>M_{\varepsilon}$. Thus, we have

$$\begin{aligned} \Vert u \Vert _{L^{2}(\mathbb{H}^{N})}^{2} =&\int_{\{|\eta|_{\mathbb{H}^{N}}>M_{\varepsilon}\}} \frac {1}{b(\eta )}b(\eta)u_{n}^{2} +\int_{\{|\eta|_{\mathbb{H}^{N}}\leq M_{\varepsilon}\}}u_{n}^{2} \\ \leq& \varepsilon \Vert u \Vert ^{2}+\int_{\{|\eta |_{\mathbb {H}^{N}}\leq M_{\varepsilon}\}}u_{n}^{2}. \end{aligned}$$

As $\{u_{n}\}$ is bounded in E, $\{u_{n}\}$ possess a subsequence strongly converging to zero in $L^{2}(\{|\eta|_{\mathbb{H}^{N}}\leq M_{\varepsilon}\})$ by the Folland-Stein embedding theorem [11]. The proof is completed. □

Firstly, we introduce the eigenvalue problem. Let $\mathfrak {L}u=-\Delta _{\mathbb{H}}u+b(\eta)u$. We consider the following eigenvalue problem:

$$\mathfrak{L}u=\lambda u. $$

By virtue of the spectral theory for compact operators, we get a sequence of eigenvalues

$$0< \lambda_{1}< \lambda_{2}\leq\cdots\leq \lambda_{n}\leq\cdots, $$

with $\lambda_{n}\rightarrow+\infty$ as $n\rightarrow\infty$ and the first eigenvalue $\lambda_{1}$ has the variational characterization

$$\lambda_{1}=\inf\bigl\{ \Vert u \Vert ^{2}; u\in E, \Vert u \Vert _{L^{2}(\mathbb{H}^{N})}=1\bigr\} . $$

Definition 1.2

A critical point u of the functional I is defined to be a function $u\in E$ such that $\langle I^{\prime}(u), h\rangle=0$, $\forall h\in E\cap L^{\infty}(\mathbb{H}^{N})$.

Next we state the assumptions and main results of this paper. We make the following hypotheses:

(J₁):

$J(\cdot,\cdot,\cdot):\mathbb{H}^{N}\times\mathbb {R}\times \mathbb{R}^{2N}\rightarrow\mathbb{R}$ satisfies:

• for each $(s,\xi)\in\mathbb{R}\times\mathbb{R}^{2N}$, $J(\eta ,s,\xi )$ is measurable with respect to η;

• for a.e. $\eta\in\mathbb{H}^{N}$, $J(\eta,s,\xi)$ is of class $C^{1}$ with respect to $(s,\xi)$;

• $J(\eta,s,\xi)$ is convex with respect to ξ.

(J₂):

There exist $0< \alpha< \beta< +\infty$ such that

$$\begin{aligned}& \alpha|\xi|^{2}\leq J(\eta,s,\xi)\leq\beta|\xi|^{2} \quad \text{a.e. } \eta\in\mathbb{H}^{N} \text{ and } \forall(s,\xi)\in \mathbb {R}\times\mathbb{R}^{2N}, \\& \bigl\vert J_{s}(\eta,s,\xi)\bigr\vert \leq\beta| \xi|^{2} \quad \text{a.e. } \eta\in \mathbb {H}^{N} \text{ and } \forall(s,\xi)\in\mathbb{R}\times\mathbb {R}^{2N}. \end{aligned}$$

(J₃):

There exist $R>0$, $\theta>2$, $1<\gamma<\frac{\theta }{2}$, and $\alpha_{1}>0$ such that

$$\begin{aligned}& J_{s}(x,s,\xi)s\geq0, \quad |s|>R, \\& \theta J(x,s,\xi)-\gamma J_{s}(x,s,\xi)s-\gamma J_{\xi}(x,s, \xi )\cdot\xi \geq\alpha_{1}|\xi|^{2}. \end{aligned}$$

(J₄):

$J(\eta,s,\xi)=J(\eta,-s,-\xi)$ a.e. $\eta\in\mathbb{H}^{N}$ and $\forall(s,\xi)\in\mathbb{R}\times\mathbb{R}^{2N}$.

(f₁):

We assume that $f(\cdot,\cdot):\mathbb{H}^{N}\times \mathbb {R}\rightarrow\mathbb{R}$ is a Carathéodory function such that

$$\begin{aligned}& \theta F(\eta,s)\leq f(\eta,s)s+a_{0}(\eta)+b_{0}(\eta)|s| \quad \text{a.e. } \eta\in\mathbb{H}^{N}, \\& F(\eta,s)\geq k|s|^{\theta}-\bar{a}(\eta)-\bar {b}(\eta)|s| \quad \text{a.e. } \eta\in\mathbb{H}^{N}, \end{aligned}$$

where θ is as in (J₃), k is a positive constant, $\bar{a}(\eta),a_{0}(\eta)\in L^{1}(\mathbb{H}^{N})$, and $b_{0}(\eta ), \bar{b}(\eta)\in L^{\frac{2Q}{Q+2}}(\mathbb{H}^{N})$.

(f₂):

$|f(\eta,s)|\leq a_{\varepsilon}(\eta)+\varepsilon |s|^{2^{*}-1}$ a.e. $\eta\in\mathbb{H}^{N}$ and $\forall s\in\mathbb{R}$.

(f₃):

$f(\eta,-s)=-f(\eta,s)$ a.e. $\eta\in\mathbb {H}^{N}$ and $\forall s\in\mathbb{R}$.

Remark 1.2

Under assumptions (J₁) and (J₂), we have

• (1) $|J_{\xi}(\eta,s,\xi)|\leq4\beta|\xi|$,

• (2) $J_{\xi}(\eta,s,\xi)\cdot\xi\geq\alpha|\xi|^{2}$.

Proof

$J(\eta,s,\xi)$ is convex with respect to ξ, which means

$$J(\eta,s,\xi+\zeta)\geq J(\eta,s,\xi)+J_{\xi}(\eta,s,\xi)\cdot \zeta. $$

If $J_{\xi}(\eta,s,\xi)=0$, then (1) holds obviously. If $J_{\xi }(\eta ,s,\xi)\neq0$, by taking $\zeta={\frac{J_{\xi}(\eta,s,\xi)|\xi |}{|J_{\xi}(\eta,s,\xi)|}}$, then (1) holds by using (J₂).

On the other hand,

$$J(\eta,s,0)\geq J(\eta,s,\xi)+J_{\xi}(\eta,s,\xi)\cdot(-\xi) $$

by virtue of assumption (J₂), one has (2). □

Remark 1.3

Under assumptions (J₁)-(J₄) and (f₁)-(f₃), for the functional I, we have the following assertions:

• (1) $I:E\rightarrow\mathbb{R}$ is continuous.

• (2) For any $u\in E$ and $h\in E\cap L^{\infty}(\mathbb{H}^{N})$, we have

  $$\begin{aligned} \bigl\langle I^{\prime}(u),h\bigr\rangle =& \int_{\mathbb{H}^{N}}J_{\xi }( \eta ,u,\nabla_{\mathbb{H}}u)\nabla_{\mathbb{H}}h+ \int_{\mathbb {H}^{N}}J_{s}( \eta,u,\nabla_{\mathbb{H}}u)h \\ &{}+\int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda\bigr)uh-\int _{\mathbb {H}^{N}}f(\eta, u)h. \end{aligned}$$

Moreover, for any $h\in E\cap L^{\infty}(\mathbb{H}^{N})$, the map $u\mapsto\langle I^{\prime}(u),h\rangle$ is continuous.

Thirdly, we recall some definitions and properties of nonsmooth critical theory (see [6, 14–19]).

Definition 1.3

Let $f:X\rightarrow\mathbb{R}$ be a continuous functional and $u\in X$. We denote by $|df|(u)$ the supremum of the $\sigma^{\prime}$ in $[0,+\infty)$ such that there exist $\delta>0$ and a continuous map $\mathcal{H}:B(u,\delta)\times[0,\delta ]\rightarrow X$ such that for all $(v,t)\in B(u,\delta)\times[0,\delta]$,

$$d\bigl(\mathcal{H}(v,t),v\bigr)\leq t \quad \text{and} \quad f\bigl(\mathcal {H}(v,t)\bigr)\leq f(v)-\sigma t. $$

The extended real number $|df|(u)$ is called the weak slope of f at u.

Remark 1.4

For any $u\in E$,

$$|dI|(u)\geq\sup\bigl\{ \bigl\langle I^{\prime}(u),h\bigr\rangle ;h\in E\cap L^{\infty }\bigl(\mathbb{H}^{N}\bigr),\Vert h \Vert \leq1 \bigr\} . $$

Proof

If $\sup\{\langle I^{\prime}(u),h\rangle;h\in E\cap L^{\infty}(\mathbb{H}^{N}),\Vert h \Vert \leq1 \}=0$, then the conclusion holds.

Otherwise, for a given σ with

$$0< \sigma< \sup\bigl\{ \bigl\langle I^{\prime}(u),h\bigr\rangle ;h\in E\cap L^{\infty }\bigl(\mathbb{H}^{N}\bigr),\Vert h \Vert \leq1 \bigr\} , $$

there exists $h\in E\cap L^{\infty}(\mathbb{H}^{N})$ such that $\Vert h \Vert \leq1$ and $\langle I^{\prime}(u),h\rangle>\sigma$. Since $\langle I^{\prime}(u),h\rangle$ is continuous with respect to u, there exists $\delta_{1}>0$ such that

$$\bigl\langle I^{\prime}(v),h\bigr\rangle >\sigma $$

for any $v\in B(u,\delta_{1})$. Define a continuous map:

$$\mathcal{H}:B(u,\delta)\times[0,\delta]\rightarrow E \quad (\delta = \delta_{1}/2) $$

by $\mathcal{H}(v,t)=v-th$. It is trivial that $\Vert \mathcal {H}(v,t)-v \Vert \leq t$. On the other hand, by Lagrange mean value theorem, it is easy to see that

$$I\bigl(\mathcal{H}(v,t)\bigr)\leq I(v)-\sigma t. $$

It follows that $|dI|(u)\geq\sigma$, and we complete the proof by the arbitrariness of σ. □

Definition 1.4

Let X be a metric space and $f:X\rightarrow \mathbb{R}$ be a continuous functional. For a $c\in\mathbb{R}$, we say that f satisfies the Palais-Smale condition at level c, denoted by $(\mathit{PS})_{c}$, if every sequence $\{u_{n}\}$ in X with $|df|(u_{n})\rightarrow0$ and $f(u_{n})\rightarrow c$ admits a strongly convergent subsequence.

The main result of this paper is the following theorem.

Theorem 1.1

Assume (J₁)-(J₄) and (f₁)-(f₃) hold. Then there exists a sequence $\{u_{n}\} \subset E\cap L^{\infty}(\mathbb{H}^{N})$ of weak solutions of problem (1.1) with $I(u_{n})\rightarrow+\infty$.

The paper is organized as follows. In Section 2, we introduce and establish some lemmas for Theorem 1.1. In Section 3, we will prove the main theorem. In the last section, we obtain boundedness of critical points (Theorem 4.1).

2 Preliminaries and fundamental lemmas

First we introduce the following fundamental theorem (see Theorem 1.4 of [15]), which is an extension of a well-known result for $C^{1}$ functionals (see Theorem 9.12 of [20]).

Lemma 2.1

Let X be an infinite-dimensional Banach space and $f:X\rightarrow\mathbb{R}$ be continuous, even and satisfy $(\mathit{PS})_{c}$ for any $c\in\mathbb{R}$. Assume, in addition, that:

• (1) there exist $\rho>0$, $\alpha>f(0)$ and a subspace $V\subset X$ of finite codimension such that

  $$\forall u\in V\mbox{:} \quad \Vert u \Vert =\rho\quad \Rightarrow\quad f(u) \geq \alpha; $$

• (2) for every finite-dimensional subspace $W\subset E$, there exists $R>0$ such that

  $$\forall u\in W\mbox{:}\quad \Vert u \Vert =R\quad \Rightarrow \quad f(u) \leq f(0). $$

Then there exists a sequence $\{c_{n}\}$ of critical values of f with $c_{n}\rightarrow+\infty$.

Now, in order to prove that the functional I satisfies the Palais-Smale condition, we will introduce an auxiliary notion.

Definition 2.1

Let c be a real number. We say that functional I satisfies the concrete Palais-Smale condition at level c (denoted by $(\mathit{CPS})_{c}$), if every sequence $\{u_{n}\}\subset E$ satisfies

$$\lim_{n\rightarrow\infty}I(u_{n})=c \quad \text{and} \quad \bigl\langle I^{\prime}(u_{n}),h\bigr\rangle =\langle \omega_{n},h\rangle, $$

where $\{\omega_{n}\}$ is a sequence converging to zero in $E^{*}$, which is possible to extract a strongly convergent subsequence in E.

Lemma 2.2

Let c be a real number. If I satisfies $(\mathit{CPS})_{c}$, then I satisfies $(\mathit{PS})_{c}$.

By Remark 1.4, the proof of this lemma is standard, and we omit it here.

Lemma 2.3

Let $\{u_{n}\}$ be a bounded sequence in E, satisfying

$$ \bigl\langle I^{\prime}(u_{n}),h\bigr\rangle =\langle \omega_{n},h\rangle,\quad \forall h\in E\cap L^{\infty}\bigl( \mathbb{H}^{N}\bigr), $$

(2.1)

where $\{\omega_{n}\}$ is a sequence converging to zero in $E^{*}$. Then there exists $u\in E$ such that $\nabla_{\mathbb {H}}u_{n}\rightarrow\nabla_{\mathbb{H}}u$ a.e. in $\mathbb{H}^{N}$ and, up to a subsequence, $\{u_{n}\}$ is weakly convergent to u in E. Moreover, we have

$$ \bigl\langle I^{\prime}(u),h\bigr\rangle =0,\quad \forall h\in E\cap L^{\infty }\bigl(\mathbb{H}^{N}\bigr), $$

(2.2)

i.e., u is a critical point of I.

Proof

By the argument as [21], we get $\nabla_{\mathbb {H}}u_{n}(\eta)\rightarrow\nabla_{\mathbb{H}}u(\eta)$ a.e. $\eta\in \mathbb{H}^{N}$. Since $\{u_{n}\}$ is bounded in E, we have

$$\begin{aligned}& u_{n}\rightharpoonup u \quad \mbox{in } E , \\& u_{n} \rightarrow u \quad \mbox{in } L^{q} , 2\leq q< 2^{*} , \\& u_{n}(\eta)\rightarrow u(\eta)\quad \mbox{a.e. } \eta\in \mathbb{H}^{N} . \end{aligned}$$

Substituting $h=\varphi e^{-M(u_{n}-R)^{-}}$ into (2.1), where $\varphi\in E\cap L^{\infty}(\mathbb{H}^{N})$, $\varphi\geq0$, and $M=\frac{\beta }{\alpha}$, we have

$$\bigl\langle I^{\prime}(u_{n}),\varphi e^{-M(u_{n}-R)^{-}}\bigr\rangle =\bigl\langle \omega _{n},\varphi e^{-M(u_{n}-R)^{-}}\bigr\rangle , $$

i.e.,

$$\begin{aligned} \bigl\langle \omega_{n},\varphi e^{-M(u_{n}-R)^{-}}\bigr\rangle =& \int _{\mathbb {H}^{N}}J_{\xi}(\eta,u_{n}, \nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb{H}}\varphi\cdot e^{-M(u_{n}-R)^{-}} \\ &{}+\int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda\bigr)u_{n} \varphi e^{-M(u_{n}-R)^{-}}+\int_{\mathbb{H}^{N}}\bigl(J_{s}( \eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \\ &{}-MJ_{\xi}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb {H}}(u_{n}-R)^{-}\bigr)\varphi e^{-M(u_{n}-R)^{-}} \\ &{}-\int_{\mathbb{H}^{N}}f(\eta,u_{n})\varphi e^{-M(u_{n}-R)^{-}}. \end{aligned}$$

When $u_{n}\geq R$, by (J₃), we have

$$\begin{aligned}& \bigl(J_{s}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n})- MJ_{\xi}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb {H}}(u_{n}-R)^{-}\bigr)\varphi e^{-M(u_{n}-R)^{-}} \\& \quad =J_{s}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \varphi e^{-M(u_{n}-R)^{-}}\geq0. \end{aligned}$$

When $u_{n}\leq R$, by (J₂) and Remark 1.2, we have

$$\begin{aligned} \begin{aligned} &\bigl(J_{s}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n})- MJ_{\xi}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb {H}}(u_{n}-R)^{-}\bigr)\varphi e^{-M(u_{n}-R)^{-}} \\ &\quad \geq-\beta|\nabla_{\mathbb{H}}u_{n}|^{2}+M\alpha| \nabla_{\mathbb{H}}u_{n}|^{2}=0. \end{aligned} \end{aligned}$$

By the convergency of $\{u_{n}\}$, Remark 1.2, $\omega_{n}\rightarrow0$ in $E^{*}$ and (f₂), we get

$$\begin{aligned}& \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u_{n}, \nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb{H}}\varphi e^{-M(u_{n}-R)^{-}}\rightarrow\int_{\mathbb {H}^{N}}J_{\xi}(\eta,u, \nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}\varphi e^{-M(u-R)^{-}}, \\& \int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda\bigr)u_{n}\varphi e^{-M(u_{n}-R)^{-}}\rightarrow\int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda \bigr)u\varphi e^{-M(u-R)^{-}}, \\& \int_{\mathbb{H}^{N}}f(\eta,u_{n})\varphi e^{-M(u_{n}-R)^{-}} \rightarrow \int_{\mathbb{H}^{N}}f(\eta,u)\varphi e^{-M(u-R)^{-}}, \\& \bigl\langle \omega_{n},\varphi e^{-M(u_{n}-R)^{-}}\bigr\rangle \rightarrow0, \end{aligned}$$

as $n\rightarrow\infty$. We apply Fatou’s lemma to get

$$\begin{aligned}& \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}\varphi e^{-M(u-R)^{-}}+\int_{\mathbb{H}^{N}} \bigl(b(\eta )-\lambda\bigr)u\varphi e^{-M(u-R)^{-}} \\& \quad {}+\int_{\mathbb{H}^{N}}\bigl(J_{s}(\eta,u, \nabla_{\mathbb{H}}u)-MJ_{\xi }(\eta ,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}(u-R)^{-}\bigr)\varphi e^{-M(u-R)^{-}} \\& \quad {}-\int_{\mathbb{H}^{N}}f(\eta,u)\varphi e^{-M(u-R)^{-}}\leq0. \end{aligned}$$

Next, let $\varphi=\psi e^{M(u-R)^{-}}H(\frac{u}{k})$, where $\psi \geq0$, $\psi\in E\cap L^{\infty}(\mathbb{H}^{N})$, $k\in\mathbb{N}$ and $H(\eta)\in C_{0}^{\infty}(\mathbb{H}^{N})$ with $H(\eta)=1$ as $|\eta |_{\mathbb{H}^{N}}\leq1$, $H(\eta)=0$ as $|\eta|_{\mathbb{H}^{N}}\geq2$.

Then we have

$$\begin{aligned}& \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}\psi H\biggl(\frac{u}{k}\biggr)+\int _{\mathbb{H}^{N}}\bigl(b(\eta )-\lambda\bigr)u\psi H\biggl( \frac{u}{k}\biggr) \\& \quad {} +\int_{\mathbb{H}^{N}} J_{s}(\eta,u, \nabla_{\mathbb{H}}u)\psi H\biggl(\frac{u}{k}\biggr)+ \int _{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u)\psi H^{\prime}\biggl(\frac{u}{k}\biggr)\frac{\nabla_{\mathbb{H}}u}{k} \\& \quad {}-\int_{\mathbb{H}^{N}}f(\eta,u)\psi H\biggl(\frac{u}{k} \biggr)\leq0. \end{aligned}$$

Putting $k\rightarrow\infty$, we get

$$\bigl\langle I^{\prime}(u),\psi\bigr\rangle \leq0. $$

By taking $h=\varphi e^{-M(u_{n}+R)^{+}}$ and a similar argument we can get the opposite inequality. So we have

$$\bigl\langle I^{\prime}(u),\psi\bigr\rangle =0\quad \mbox{for }\psi\geq0, \psi\in E\cap L^{\infty}\bigl(\mathbb{H}^{N}\bigr). $$

Hence,

$$\bigl\langle I^{\prime}(u),h\bigr\rangle =0 , \quad h\in E\cap L^{\infty}\bigl(\mathbb{H}^{N}\bigr). $$

The proof has been completed. □

In (2.2) we can only select test functions in $E\cap L^{\infty}(\mathbb {H}^{N})$. In the following lemma, we will enlarge the class of test functions.

Lemma 2.4

Suppose that $u\in E$ satisfies $\langle I^{\prime}(u),h\rangle=\langle\omega,h\rangle$, $h\in E\cap L^{\infty}(\mathbb {H}^{N})$, where $\omega\in E^{*}$. For $v\in E$, there exists $W(\eta )\in L^{1}(\mathbb{H}^{N})$ with

$$ J_{s}(\eta,u,\nabla_{\mathbb{H}^{N}}u)v\geq W(\eta) \quad \textit{a.e. } \eta\in \mathbb {H}^{N}, $$

(2.3)

then $\langle I^{\prime}(u),v\rangle=\langle\omega,v\rangle$.

Proof

Let

$$T_{k}(s)=\left \{ \textstyle\begin{array}{l@{\quad}l} s, & |s|\leq k, \\ k\frac{s}{|s|}, & |s|\geq k. \end{array}\displaystyle \right . $$

Then $T_{k}(v)\in E\cap L^{\infty}(\mathbb{H}^{N})$ for every $v\in E$. By Lemma 2.3, we have

$$\bigl\langle I^{\prime}(u),T_{k}(v) \bigr\rangle =\bigl\langle \omega,T_{k}(v)\bigr\rangle , $$

i.e.,

$$\begin{aligned}& \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}T_{k}(v)+ \int_{\mathbb{H}^{N}} J_{s}(\eta,u,\nabla_{\mathbb{H}}u)T_{k}(v) \\& \quad {}+\int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda\bigr)uT_{k}(v) -\int_{\mathbb{H}^{N}}f(\eta,u)T_{k}(v)=\bigl\langle \omega,T_{k}(v)\bigr\rangle . \end{aligned}$$

(2.4)

Since

$$\bigl\vert J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}T_{k}(v)\bigr\vert \leq\bigl\vert J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}v\bigr\vert , $$

by Remark 1.2, we have $J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}v\in L^{1}(\mathbb{H}^{N})$. From Lebesgue’s dominated convergence theorem, as $k\rightarrow\infty$, we have

$$\begin{aligned}& \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla _{\mathbb{H}}T_{k}(v)\rightarrow\int_{\mathbb{H}^{N}}J_{\xi}( \eta ,u,\nabla _{\mathbb{H}}u)\nabla_{\mathbb{H}}v, \\& \int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda\bigr)uT_{k}(v) \rightarrow\int_{\mathbb {H}^{N}}\bigl(b(\eta)-\lambda\bigr)uv, \\& \int_{\mathbb{H}^{N}}f(\eta,u)T_{k}(v)\rightarrow\int _{\mathbb {H}^{N}}f(\eta,u)v, \\& \bigl\langle \omega,T_{k}(v)\bigr\rangle \rightarrow\langle\omega,v \rangle. \end{aligned}$$

Then it is easy to see that

$$J_{s}(\eta,u,\nabla_{\mathbb{H}^{N}}u)T_{k}(v)\geq-W^{-}(\eta) $$

from (2.3). By taking the inferior limit in (2.4) and applying Fatou’s lemma, we obtain

$$\begin{aligned}& \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}v+ \int_{\mathbb{H}^{N}} J_{s}(\eta,u, \nabla_{\mathbb{H}}u)v \\& \quad {}+\int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda\bigr)uv -\int _{\mathbb{H}^{N}}f(\eta,u)v\leq\langle\omega,v\rangle. \end{aligned}$$

Then $J_{s}(\eta,u,\nabla_{\mathbb{H}^{N}}u)v\in L^{1}(\mathbb{H}^{N})$. Using Lebesgue’s dominated convergence theorem in (2.4) again, we obtain

$$\begin{aligned}& \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}v+ \int_{\mathbb{H}^{N}} J_{s}(\eta,u, \nabla_{\mathbb{H}}u)v \\& \quad {}+\int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda\bigr)uv -\int _{\mathbb{H}^{N}}f(\eta,u)v=\langle\omega,v\rangle. \end{aligned}$$

The lemma has been proved. □

Lemma 2.5

Let $c\in\mathbb{R}$ and $\{u_{n}\}$ be a sequence satisfying (2.1) and

$$ \lim_{n\rightarrow\infty}I(u_{n})=c. $$

(2.5)

Then $\{u_{n}\}$ is bounded in E.

Proof

By (J₃), we have

$$J_{s}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n})u_{n} \geq-\beta R|\nabla _{\mathbb {H}}u_{n}|^{2}\in L^{1}\bigl(\mathbb{H}^{N}\bigr). $$

Further, we get

$$ \bigl\langle I^{\prime}(u_{n}),u_{n}\bigr\rangle = \langle\omega_{n},u_{n}\rangle $$

(2.6)

by Lemma 2.4. From the assumptions we have

$$\begin{aligned}& \theta I(u_{n})-\gamma\bigl\langle I^{\prime}(u_{n}),u_{n} \bigr\rangle \\& \quad = \theta\int_{\mathbb{H}^{N}}J(\eta,u_{n}, \nabla_{\mathbb{H}}u_{n}) -\gamma\int_{\mathbb{H}^{N}}J_{s}( \eta,u_{n},\nabla_{\mathbb {H}}u_{n})u_{n} \\& \qquad {}-\gamma\int_{\mathbb{H}^{N}}J_{\xi}(\eta,u_{n}, \nabla_{\mathbb {H}}u_{n})\nabla_{\mathbb{H}}u_{n} + \biggl(\frac{\theta}{2}-\gamma\biggr)\int_{\mathbb{H}^{N}}\bigl(b(\eta)- \lambda \bigr)u_{n}^{2} \\& \qquad {}+\int_{\mathbb{H}^{N}}\bigl(\gamma f(\eta,u_{n})u_{n}- \theta F(\eta ,u_{n})\bigr) \\& \quad \leq c\bigl(1+\Vert u_{n} \Vert \bigr). \end{aligned}$$

From (J₃) and (f₁), it follows that

$$\begin{aligned}& \alpha_{1}\int_{\mathbb{H}^{N}}|\nabla_{\mathbb{H}}u_{n}|^{2} +\biggl(\frac{\theta}{2}-\gamma\biggr)\int_{\mathbb{H}^{N}}\bigl(b( \eta)-\lambda\bigr)u_{n}^{2} +\theta(\gamma-1)\int _{\mathbb{H}^{N}}F(\eta,u_{n}) \\& \quad \leq c\bigl(1+\Vert u_{n} \Vert \bigr)+\gamma \Vert a_{0} \Vert +\gamma \Vert b_{0}\Vert _{L^{\frac{2Q}{Q+2}}(\mathbb {H}^{N})} \Vert u_{n}\Vert _{L^{\frac{2Q}{Q-2}}(\mathbb{H}^{N})}. \end{aligned}$$

(2.7)

There exist $M>0$ and $c(M,\lambda)>0$ such that

$$\biggl(\frac{\theta}{2}-\gamma\biggr)\int_{\mathbb{H}^{N}}\bigl(b( \eta)-\lambda \bigr)u_{n}^{2}\geq \biggl(\frac{\theta}{2}- \gamma\biggr)\int_{\mathbb{H}^{N}}\frac{b(\eta)}{2}u_{n}^{2} -c\int_{\{|\eta|_{\mathbb{H}^{N}}< M\}}u_{n}^{2}. $$

By (2.7), we obtain

$$\begin{aligned}& \alpha_{1}\int_{\mathbb{H}^{N}}|\nabla_{\mathbb{H}}u_{n}|^{2} +\biggl(\frac{\theta}{4}-\frac{\gamma}{2}\biggr)\int_{\mathbb{H}^{N}}b( \eta)u_{n}^{2} +\theta(\gamma-1)\int_{\mathbb{H}^{N}}F( \eta,u_{n}) \\& \quad \leq c\bigl(1+\Vert u_{n} \Vert \bigr)+c\Vert u_{n} \Vert _{L^{2}(\{|\eta|_{\mathbb{H}^{N}}< M\})}^{2}+\gamma \Vert a_{0}\Vert +\gamma \Vert b_{0}\Vert _{L^{\frac{2Q}{Q+2}}(\mathbb {H}^{N})} \Vert u_{n}\Vert _{L^{\frac{2Q}{Q-2}}(\mathbb {H}^{N})}. \end{aligned}$$

(2.8)

It follows from (f₁) and (2.8) that

$$\begin{aligned}& \min\biggl\{ \alpha_{1},\frac{\theta}{4}-\frac{\gamma}{2}\biggr\} \Vert u_{n} \Vert ^{2}+\theta(\gamma-1)k\Vert u_{n} \Vert _{L^{\theta }(\{|\eta|_{\mathbb{H}^{N}}< M\})}^{\theta} \\& \quad \leq c\bigl(1+\Vert u_{n} \Vert \bigr)+c\Vert u_{n} \Vert _{L^{\theta}(\{|\eta|_{\mathbb{H}^{N}}< M\})}^{2}+\gamma \Vert a_{0}\Vert _{L^{1}(\mathbb{H}^{N})}+\gamma \Vert b_{0} \Vert _{L^{\frac{2Q}{Q+2}}(\mathbb{H}^{N})}\Vert u_{n}\Vert _{L^{\frac {2Q}{Q-2}}(\mathbb{H}^{N})} \\& \qquad {} +\theta(\gamma-1)\Vert \bar{a}\Vert _{L^{1}(\mathbb {H}^{N})}+\theta( \gamma-1)\Vert \bar{b}\Vert _{L^{\frac {2Q}{Q+2}}(\mathbb{H}^{N})}\Vert u_{n}\Vert _{L^{\frac {2Q}{Q-2}}(\mathbb{H}^{N})} \\& \quad \leq c\bigl(1+\Vert u_{n} \Vert \bigr)+\frac{\theta(\gamma -1)k}{2} \Vert u_{n} \Vert _{L^{\theta}(\{|\eta|_{\mathbb {H}^{N}}< M\})}^{\theta}+\gamma \Vert a_{0}\Vert _{L^{1}(\mathbb {H}^{N})}+c\Vert b_{0}\Vert _{L^{\frac{2Q}{Q+2}}(\mathbb {H}^{N})}\Vert u_{n}\Vert \\& \qquad {} +\theta(\gamma-1)\Vert \bar{a}\Vert _{L^{1}(\mathbb {H}^{N})}+c\Vert \bar{b}\Vert _{L^{\frac {2Q}{Q+2}}(\mathbb {H}^{N})}\Vert u_{n}\Vert . \end{aligned}$$

Therefore,

$$\min\biggl\{ \alpha_{1},\frac{\theta}{4}-\frac{\gamma}{2}\biggr\} \Vert u_{n} \Vert ^{2} \leq c\bigl(1+\Vert u_{n} \Vert \bigr). $$

This implies that $\{u_{n}\}$ is bounded in E. □

Lemma 2.6

Let $\{u_{n}\}$ be a subsequence as in Lemma  2.5. Then $\{u_{n}\}$, possessing a subsequence, converges strongly in E.

Proof

Consider the cut-off function

$$\zeta(s)=\left \{ \textstyle\begin{array}{l@{\quad}l} M|s|, & |s|\leq R, \\ MR, & |s|\geq R, \end{array}\displaystyle \right . $$

where $M=\frac{\beta}{\alpha}$. It is easy to prove $\{u_{n} e^{\zeta (u_{n})}\}$ is bounded in E, up to a subsequence, having

$$\begin{aligned}& u_{n} e^{\zeta(u_{n})}\rightharpoonup ue^{\zeta(u)} \quad \mbox{in } E , \\& u_{n} e^{\zeta(u_{n})} \rightarrow ue^{\zeta(u)} \quad \mbox{in } L^{q} , 2\leq q< 2^{*} , \\& u_{n} e^{\zeta(u_{n})}(\eta)\rightarrow ue^{\zeta(u)}(\eta)\quad \mbox{a.e. } \eta \in\mathbb{H}^{N} . \end{aligned}$$

By Lemma 2.3 we know that $ue^{\zeta(u)}$ is a critical point of the functional I. Let $h=u_{n}e^{\zeta(u_{n})}$ in (2.1). It follows from Lemma 2.4 that

$$\begin{aligned} \begin{aligned}[b] &\int_{\mathbb{H}^{N}}J_{\xi}(\eta,u_{n}, \nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb{H}}u_{n}e^{\zeta(u_{n})}+ \int_{\mathbb{H}^{N}}\bigl(b(\eta )-\lambda\bigr)u_{n}^{2}e^{\zeta(u_{n})} \\ &\quad {}+\int_{\mathbb{H}^{N}} \bigl(J_{s}( \eta,u_{n},\nabla_{\mathbb{H}}u_{n})+ J_{\xi}( \eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb{H}}u_{n} \zeta^{\prime}(u_{n})\bigr)u_{n} e^{\zeta(u_{n})} \\ &\quad {}-\int_{\mathbb{H}^{N}}f(\eta,u_{n})u_{n}e^{\zeta(u_{n})}= \bigl\langle \omega _{n},u_{n}e^{\zeta(u_{n})}\bigr\rangle . \end{aligned} \end{aligned}$$

(2.9)

We claim

$$\bigl(J_{s}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n})+ J_{\xi}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb{H}}u_{n}\zeta^{\prime}(u_{n}) \bigr)u_{n} e^{\zeta(u_{n})}\geq0. $$

In fact, when $u_{n}\geq R$, we have

$$\bigl(J_{s}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n})+ J_{\xi}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb{H}}u_{n}\zeta^{\prime}(u_{n}) \bigr)u_{n} e^{\zeta (u_{n})}=J_{s}(\eta ,u_{n}, \nabla_{\mathbb{H}}u_{n})u_{n} e^{\zeta(u_{n})}\geq0. $$

When $0\leq u_{n}\leq R$, we have

$$\bigl(J_{s}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n})+ J_{\xi}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb{H}}u_{n}\zeta^{\prime}(u_{n}) \bigr)u_{n} e^{\zeta(u_{n})}\geq (M\alpha -\beta)u_{n} e^{\zeta(u_{n})}|\nabla_{\mathbb{H}}u_{n}|^{2}=0. $$

In the case $u_{n}\leq0$, the proof is similar.

By Lemma 2.3, $\nabla_{\mathbb{H}}u_{n}\rightarrow\nabla_{\mathbb{H}}u$ a.e. in $\mathbb{H}^{N}$. By virtue of Fatou’s lemma, we have

$$\begin{aligned}& \int_{\mathbb{H}^{N}}\bigl(J_{s}(\eta,u,\nabla_{\mathbb{H}}u)+ J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}u \zeta^{\prime}(u)\bigr)u e^{\zeta(u)} \\& \quad \leq \liminf_{n\rightarrow\infty}\int_{\mathbb {H}^{N}} \bigl(J_{s}(\eta ,u_{n},\nabla_{\mathbb{H}}u_{n})+ J_{\xi}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb{H}}u_{n}\zeta^{\prime}(u_{n}) \bigr)u_{n} e^{\zeta(u_{n})}. \end{aligned}$$

(2.10)

Moreover, it is easy to prove $f(\eta,\cdot): E\rightarrow E^{*}$ is a compact operator, and

$$\begin{aligned}& \lim_{n\rightarrow\infty}\int_{\mathbb{H}^{N}}f(\eta ,u_{n})u_{n}e^{\zeta(u_{n})}=\lim_{n\rightarrow\infty} \int_{\mathbb {H}^{N}}f(\eta,u)ue^{\zeta(u)}, \end{aligned}$$

(2.11)

$$\begin{aligned}& \lim_{n\rightarrow\infty}\int_{\mathbb {H}^{N}}u_{n}^{2}e^{\zeta (u_{n})}= \int_{\mathbb{H}^{N}}u^{2}e^{\zeta(u)}. \end{aligned}$$

(2.12)

By Lemma 2.4, let $h=ue^{\zeta(u)}$ in (2.2), and then

$$\begin{aligned}& \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}ue^{\zeta(u)}+\int_{\mathbb{H}^{N}}\bigl(b(\eta )-\lambda \bigr)u^{2}e^{\zeta(u)} -\int_{\mathbb{H}^{N}}f( \eta,u)ue^{\zeta(u)} \\& \quad {}+\int_{\mathbb{H}^{N}} \bigl(J_{s}(\eta,u, \nabla_{\mathbb{H}}u)+ J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}u\zeta^{\prime}(u)\bigr)u e^{\zeta(u)}=0. \end{aligned}$$

(2.13)

Combining (2.9)-(2.13), we have

$$\begin{aligned}& \limsup_{n\rightarrow\infty} \biggl(\int_{\mathbb{H}^{N}} J_{\xi }(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb {H}}u_{n}e^{\zeta (u_{n})}+\int_{\mathbb{H}^{N}}b( \eta)u_{n}^{2}e^{\zeta(u_{n})} \biggr) \\& \quad = \limsup_{n\rightarrow\infty}\biggl(\int_{\mathbb {H}^{N}}- \bigl(J_{s}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n})+ J_{\xi}(\eta,u_{n},\nabla_{\mathbb{H}}u_{n}) \nabla_{\mathbb {H}}u_{n}\zeta ^{\prime}(u_{n}) \bigr)u_{n} e^{\zeta(u_{n})} \\& \qquad {} +\int_{\mathbb{H}^{N}}\bigl(\lambda u_{n}^{2}e^{\zeta (u_{n})}+f( \eta ,u_{n})u_{n}e^{\zeta(u_{n})}\bigr)\biggr) \\& \quad \leq \int_{\mathbb{H}^{N}}-\bigl(J_{s}(\eta,u, \nabla_{\mathbb {H}}u)+J_{\xi }(\eta,u,\nabla_{\mathbb{H}}u) \nabla_{\mathbb{H}}u\zeta^{\prime}(u)\bigr)u e^{\zeta(u)} \\& \qquad {} +\int_{\mathbb{H}^{N}}\bigl(\lambda u^{2}e^{\zeta(u)}+f( \eta ,u)u_{n}e^{\zeta(u)}\bigr) \\& \quad =\int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla _{\mathbb {H}}u)\nabla_{\mathbb{H}}ue^{\zeta(u)}+\int_{\mathbb{H}^{N}}b( \eta )u^{2}e^{\zeta(u)}. \end{aligned}$$

(2.14)

By Fatou’s lemma, we have

$$\begin{aligned}& \int_{\mathbb{H}^{N}} J_{\xi}(\eta,u,\nabla_{\mathbb {H}}u) \nabla _{\mathbb{H}}ue^{\zeta(u)}+\int_{\mathbb{H}^{N}}b(\eta )u^{2}e^{\zeta (u)} \\& \quad \leq \liminf_{n\rightarrow\infty} \biggl(\int_{\mathbb{H}^{N}}J_{\xi}( \eta,u_{n},\nabla_{\mathbb {H}}u_{n})\nabla_{\mathbb{H}}u_{n}e^{\zeta(u_{n})}+ \int_{\mathbb {H}^{N}}b(\eta)u_{n}^{2}e^{\zeta(u_{n})} \biggr) \\& \quad \leq \limsup_{n\rightarrow\infty} \biggl(\int_{\mathbb{H}^{N}}J_{\xi}( \eta,u_{n},\nabla_{\mathbb {H}}u_{n})\nabla_{\mathbb{H}}u_{n}e^{\zeta(u_{n})}+ \int_{\mathbb {H}^{N}}b(\eta)u_{n}^{2}e^{\zeta(u_{n})} \biggr) \\& \quad \leq \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u, \nabla_{\mathbb {H}}u)\nabla _{\mathbb{H}}ue^{\zeta(u)}+\int _{\mathbb{H}^{N}}b(\eta )u^{2}e^{\zeta(u)}. \end{aligned}$$

Hence, we get

$$\begin{aligned}& \lim_{n\rightarrow\infty} \biggl(\int_{\mathbb{H}^{N}}J_{\xi }( \eta ,u_{n},\nabla_{\mathbb{H}}u_{n})\nabla_{\mathbb{H}}u_{n}e^{\zeta (u_{n})}+ \int_{\mathbb{H}^{N}}b(\eta)u_{n}^{2}e^{\zeta(u_{n})} \biggr) \\& \quad = \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u, \nabla_{\mathbb {H}}u)\nabla _{\mathbb{H}}ue^{\zeta(u)}+\int _{\mathbb{H}^{N}}b(\eta )u^{2}e^{\zeta(u)}. \end{aligned}$$

From Remark 1.2,

$$|\nabla_{\mathbb{H}}u_{n}|^{2}+b(\eta)u_{n}^{2} \leq\frac{J_{\xi}(\eta ,u_{n},\nabla_{\mathbb{H}}u_{n})\nabla_{\mathbb{H}}u_{n}e^{\zeta (u_{n})}}{\alpha}+b(\eta)u_{n}^{2}. $$

It follows that

$$ \lim_{n\rightarrow\infty}\int_{\mathbb{H}^{N}}\bigl(|\nabla _{\mathbb {H}}u_{n}|^{2}+b(\eta)u_{n}^{2} \bigr)=\int_{\mathbb{H}^{N}}\bigl(|\nabla_{\mathbb {H}}u|^{2}+b( \eta)u^{2}\bigr). $$

(2.15)

By Lebesgue’s dominated convergence theorem and the weak convergence of $\{u_{n}\}$ to u in E, we get

$$\begin{aligned}& \lim_{n\rightarrow\infty} \int_{\mathbb{H}^{N}}\nabla_{\mathbb{H}}u_{n} \nabla_{\mathbb {H}}u=\int_{\mathbb{H}^{N}}|\nabla_{\mathbb{H}}u|^{2}, \end{aligned}$$

(2.16)

$$\begin{aligned}& \lim_{n\rightarrow\infty} \int_{\mathbb{H}^{N}}b( \eta)u_{n}u=\int_{\mathbb{H}^{N}}b(\eta )u^{2}. \end{aligned}$$

(2.17)

By (2.15), (2.16) and (2.17), we have

$$\lim_{n\rightarrow\infty}\biggl(\int_{\mathbb{H}^{N}} | \nabla_{\mathbb{H}}u_{n}-\nabla_{\mathbb{H}}u|^{2}+ \int _{\mathbb{H}^{N}}b(\eta)|u_{n}-u|^{2}\biggr)=0. $$

That is to say, $\{u_{n}\}$ converges strongly to u in E. □

Remark 2.1

By Lemmas 2.5 and 2.6, for any c, the functional I satisfies the $(\mathit{CPS})_{c}$ condition.

3 Proof of Theorem 1.1

Proof of Theorem 1.1

The functional I is continuous and even. Moreover, by Lemma 2.2 we know that I satisfies $(\mathit{PS})_{c}$ for any $c\in\mathbb{R}$.

First we verify the condition (2) in Lemma 2.1. Let W be a finite-dimensional subspace of E. For any $u\in W$, by (f₁), we have

$$\begin{aligned} I(u) =&\int_{\mathbb{H}^{N}}J(\eta,u,\nabla_{\mathbb{H}}u)+ \frac {1}{2}\int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda \bigr)u^{2}-\int_{\mathbb{H}^{N}}F(\eta,u) \\ \leq& \max\biggl\{ \frac{1}{2},\beta\biggr\} \Vert u \Vert ^{2}-\frac {\lambda}{2}\Vert u \Vert _{L^{2}(\mathbb{H}^{N})}^{2} -\int_{\mathbb{H}^{N}}k|u|^{\theta}+\Vert \bar{a} \Vert _{L^{1}(\mathbb{H}^{N})}+c\Vert \bar{b}\Vert _{L^{\frac {2Q}{Q+2}}(\mathbb{H}^{N})}\Vert u\Vert . \end{aligned}$$

Since $(\int_{\mathbb{H}^{N}}|u|^{\rho})^{\frac{1}{\rho }}$ is a norm on W, there exists $c_{\rho}>0$ such that

$$c_{\rho} \Vert u \Vert ^{\rho}\leq \Vert u \Vert _{L^{\rho}(\mathbb{H}^{N})}^{\rho}. $$

Considering $\theta>2$, there exists $R>1$ such that $I(u)<0$ when $\Vert u \Vert =R$.

Next we consider the condition (1) in Lemma 2.1. By (J₂) and (f₂), for any $u\in E$ we have

$$I(u)\geq\frac{\min(1,\alpha)}{2}\Vert u \Vert ^{2}-\frac {\lambda}{2} \int_{\mathbb{H}^{N}}u^{2} -\int_{\mathbb{H}^{N}}a_{\varepsilon}|u|-c \varepsilon\int_{\mathbb {H}^{N}}|u|^{2^{*}}. $$

There exist $a_{1}(\eta)\in C_{0}^{\infty}$ and $a_{2}(\eta)\in L^{\frac {2Q}{Q+2}}(\mathbb{H}^{N})$ such that

$$a_{\varepsilon}=a_{1}+a_{2},\qquad \Vert a_{2} \Vert _{L^{\frac{2Q}{Q+2}}(\mathbb {H}^{N})}< \varepsilon. $$

Let $V_{k}=\operatorname{span}\{v_{1},v_{2},\ldots,v_{k}\}^{\bot}$ and $\{v_{j}\}_{j\geq1}$ be an orthonormal basis of eigenvectors of the operator $\mathfrak{L}$. Then for any $u\in V_{k}$ we have

$$\begin{aligned} I(u) \geq& \frac{\min\{1,\alpha\}}{2}\Vert u \Vert ^{2}-\frac {\lambda}{2} \Vert u \Vert _{L^{2}(\mathbb{H}^{N})}^{2} -\Vert a_{1} \Vert _{L^{2}(\mathbb{H}^{N})}\Vert u \Vert _{L^{2}(\mathbb{H}^{N})} \\ &{}-\Vert a_{2} \Vert _{L^{\frac{2Q}{Q+2}}(\mathbb {H}^{N})} \Vert u \Vert _{L^{2^{*}}(\mathbb{H}^{N})}-c\varepsilon \Vert u \Vert _{L^{2^{*}}(\mathbb{H}^{N})}^{2^{*}} \\ \geq& \frac{\min\{1,\alpha\}}{2}\Vert u \Vert ^{2}-\frac {\lambda}{2\lambda_{k+1}} \Vert u \Vert ^{2} -\frac{\Vert a_{1} \Vert _{L^{2}(\mathbb{H}^{N})}}{\sqrt {\lambda _{k+1}}}\Vert u \Vert -c \varepsilon \Vert u \Vert -c\varepsilon \Vert u \Vert ^{2^{*}}. \end{aligned}$$

When $\Vert u \Vert =1$, we can choose k large enough and ε small enough such that

$$I(u)\geq\delta>I(0)=0. $$

Hence the condition (1) of Lemma 2.1 holds with $V=V_{k}$. □

4 Boundedness of critical points

In this section, we will prove the critical point $u\in L^{\infty}(\mathbb{H}^{N})$. We make the following hypotheses:

($\mathrm{J}^{*}_{3}$):

there exist $R>0$, $\theta>2$, $1<\gamma<\frac {\theta}{2}$, and $\alpha_{1}>0$ such that

$$J_{s}(x,s,\xi)s\geq0; $$

($\mathrm{f}^{*}_{2}$):

$|f(\eta,s)|\leq c|s|^{p}$ a.e. $\eta\in\mathbb{H}^{N}$ and $\forall s\in\mathbb{R}$, where $p<2^{*}-1$ and c is a positive constant.

Theorem 4.1

Suppose that (J₃) and (f₂) are replaced with ($\mathrm{J}^{*}_{3}$) and ($\mathrm{f}^{*}_{2}$). If $u\in E$ is a critical point of I, then $u\in L^{\infty}(\mathbb{H}^{N})$.

Proof

For $k>1$ and $M>0$, define

$$G_{k}=\left \{ \textstyle\begin{array}{l@{\quad}l} s-\frac{ks}{|s|} &\text{if } |s|>k, \\ 0 & \text{if } |s|\leq k. \end{array}\displaystyle \right . $$

Let $\Phi_{M}(s)=\min(G_{k}(s),M)$ and $\Psi_{M}(s)=\max(G_{k}(s),-M)$. Denote $s^{+}=\max(s,0)$ and $s^{-}=\min(s,0)$. Considering that u is a critical point of I, since $\Phi_{M}(u^{+})\in E\cap L^{\infty }(\mathbb{H}^{N})$, we can take $\Phi_{M}(u^{+})$ as a test function in $\langle I^{\prime}(u),h\rangle=0$. Thus

$$\begin{aligned}& \int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla _{\mathbb {H}}\Phi_{M}\bigl(u^{+}\bigr)+\int _{\mathbb{H}^{N}}J_{s}(\eta,u,\nabla _{\mathbb {H}}u) \Phi_{M}\bigl(u^{+}\bigr) \\& \quad {}+\int_{\mathbb{H}^{N}}\bigl(b(\eta)-\lambda\bigr)u \Phi_{M}\bigl(u^{+}\bigr)=\int_{\mathbb {H}^{N}}f( \eta,u)\Phi_{M}\bigl(u^{+}\bigr). \end{aligned}$$

Now, observing that $u^{+}\Phi_{M}(u^{+})\geq0$ and by (J₃), $A_{s}(\eta,u)\Phi_{M}(u^{+})\geq0$, we get

$$\int_{\mathbb{H}^{N}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla _{\mathbb {H}}\Phi_{M}\bigl(u^{+}\bigr)\leq | \lambda|\int_{\mathbb{H}^{N}}u^{+}\bigl\vert \Phi_{M}\bigl(u^{+}\bigr)\bigr\vert +\int _{\mathbb {H}^{N}}\bigl\vert f\bigl(\eta,u^{+}\bigr)\bigr\vert \bigl\vert \Phi_{M}\bigl(u^{+}\bigr)\bigr\vert . $$

From (f₂) and the fact that $|\Phi_{M}(u^{+})|\leq|u^{+}|$, we deduce

$$\int_{\{u^{+}>k\}}J_{\xi}(\eta,u,\nabla_{\mathbb{H}}u) \nabla _{\mathbb {H}}\Phi_{M}\bigl(u^{+}\bigr)\leq c\int _{\{u^{+}>k\}}\bigl(u^{+}\bigr)^{p+1}. $$

Taking M to +∞ and taking into account that, as $M\rightarrow +\infty$, $\Phi_{M}(u^{+})\rightarrow G_{k}(u^{+})$ a.e. in $\mathbb {H}^{N}$ and $\Phi_{M}(u^{+})\rightharpoonup G_{k}(u^{+})$ in E, it follows that

$$\int_{\{u^{+}>k\}}J_{\xi}\bigl(\eta,u^{+}, \nabla_{\mathbb {H}}u^{+}\bigr)\nabla _{\mathbb{H}} \bigl(u^{+}\bigr)\leq c\int_{\{u^{+}>k\}} \bigl(u^{+}\bigr)^{p+1}. $$

Denote $\Omega_{k}^{+}= \{\eta\in\mathbb{H}^{N}, u^{+}>k\}$. By Remark 1.2, we obtain

$$ \int_{\Omega_{k}^{+}}\bigl\vert \nabla_{\mathbb{H}}u^{+} \bigr\vert ^{2}\leq c\int_{\Omega _{k}^{+}} \bigl(u^{+}-k\bigr)^{p+1}+ck^{p+1}m\bigl( \Omega_{k}^{+}\bigr). $$

(4.1)

Since $u\in E$, it implies that $(\int_{\Omega _{k}^{+}}(u^{+}-k)^{p+1})^{1-\frac{2}{p+1}}\leq c$, or

$$ \int_{\Omega_{k}^{+}}\bigl(u^{+}-k\bigr)^{p+1}\leq c \biggl(\int_{\Omega _{k}^{+}}\bigl(u^{+}-k\bigr)^{p+1} \biggr)^{\frac{2}{p+1}}. $$

(4.2)

On the other hand, we have

$$\int_{\Omega_{k}^{+}}k^{2^{*}}\leq\int_{\Omega _{k}^{+}}\bigl|u^{+}\bigr|^{2^{*}} =c, $$

which implies that

$$ k^{2^{*}}\leq\frac{c}{m(\Omega_{k}^{+})}. $$

(4.3)

Using (4.1), (4.2), and (4.3), the following inequality holds:

$$\int_{\Omega_{k}^{+}}\bigl\vert \nabla_{\mathbb{H}}u^{+} \bigr\vert ^{2}\leq c\biggl(\int_{\Omega _{k}^{+}} \bigl(u^{+}-k\bigr)^{p+1}\biggr)^{\frac{2}{p+1}}+ck^{2}m \bigl(\Omega _{k}^{+}\bigr)^{1-\frac{p-1}{2^{*}}},\quad \forall k>1. $$

From Theorem 5.2, Chapter II of [22], we deduce that $u^{+}\in L^{\infty}(\mathbb{H}^{N})$. Replacing $\Phi_{M}(u^{+})$ by $\Psi _{M}(u^{-})$ and by the same steps we can easily prove that $u^{-}\in L^{\infty}(\mathbb{H}^{N})$, which yields the conclusion. □