### Abstract

This study considers the dynamics of cellular neural network-based inhomogeneous lattice dynamical systems (CNN-based ILDS). The influence of three kinds of boundary conditions, say, the periodic, Dirichlet, and Neumann boundary conditions, is elucidated. We reveal that the complete stability of CNN-based ILDS and, under some prescriptions, the topological entropies of CNN-based ILDS with/without the boundary condition are identical.

**MSC: **
37B10.

##### Keywords:

inhomogeneous lattice dynamical systems; topological entropy; boundary value problem; multiplicative shift spaces; cellular neural networks### 1 Introduction

In the past few decades, the standard cellular neural networks (CNNs) introduced by
Chua and Yang [1] have been one of the most investigated paradigms for neural information processing
[2]. In a wide range of applications, the CNNs are required to be completely stable,
*i.e.*, each trajectory should converge toward some stationary state. In the study of stationary
solution, the investigation of mosaic solutions is most essential in CNNs due to the
learning algorithm and training processing. More abundant output patterns make the
learning algorithm more efficient. Mathematically, the study of the mosaic solutions
is reasonable due to the following two facts: (1) complete stability of a wide range
of parameters, and (2) the output function of CNNs is a piecewise linear function
with constant value for

The outputs
*i.e.*, the template is space-invariant. However, there are more and more CNNs using inhomogeneous
templates to describe some of the problems that arise from the biological and ecological
contexts [3-8], skeletonization [9], image processing [10,11], artificial locomotion control [12], and delayed-type CNN [13-16]. Some new and interesting phenomena of pattern formation and spatial chaos were also
found in inhomogeneous multi-layer neural networks. In this paper, the entropy with/without
the boundary effect for stable patterns of inhomogeneous CNN is investigated. Entropy
is a quantity used for measuring the complexity of the output patterns and it plays
an important role in learning algorithm. Surprisingly, such a topic reveals the deep
connection with symbolic dynamical systems (SDS). In 1-*d* CNN, it has been proved that the space of the mosaic solutions (defined later) forms
a 1-*d* subshift of finite type (SFT, [17]). Recently, it has also been proved that the mosaic solutions of a multi-layer CNN
(MCNN) form a *sofic space*[18-20], which is a factor of SFT. The mosaic solutions of inhomogeneous CNN, indeed, produce
new shift spaces in SDS. To clarify the investigation of inhomogeneous CNNs, we concentrate
our discussion on two classes, and the methodology can be applied in a general case.
More specifically, two types of inhomogeneous CNN, *constant* and *arithmetic* CNN, are presented herein. It is proved that the space of the mosaic solutions forms
a new class in SDS (Theorem 2.10 and Theorem 3.5), called a *multiple shift space*, which was initiated from the study of the arithmetic regression property in the
number theory of mathematics [21-24]. The complexity (topological entropy) can be computed due to the equivalence of the
mosaic solutions and multiple shift spaces (Theorem 2.13 and Theorem 3.7). The positivity
of entropy unveils the spatial chaos for given systems and pattern formation for zero
entropy. Such topics, *e.g.*, pattern formation or synchrony phenomena on LDS, have been investigated by many
mathematicians and physicists [25-30].

Besides the entropy formula being established, the boundary effect for constant CNNs
and arithmetic CNNs are also considered. Three types of boundary conditions, *periodic*, *Dirichlet*, and *Neumann*, are proposed to a given constant CNN and arithmetic CNN. Sufficient conditions are
found for the preservation of entropy under the boundary constraint (Theorem 2.13
and Theorem 3.7), *i.e.*,
*cf.*[31,32]). The preservation of entropy under the boundary constraint is unavoidable [33]; since the number of nodes in a lattice is infinite, one usually uses the finite
approximation method to exploit the statistical properties of the whole lattice.

Some related topics are also addressed herein. It is known that the mosaic solution
of single/multi-layer template-invariant CNNs is constrained by the so-called *separation property*, namely, not all but some of the patterns that satisfy this property will appear
as the mosaic solution for a given CNN [34]. However, more combinations of mosaic patterns will help the learning and training
process to be more efficient. It is believed that the template-variant or the multi-layer
CNN will achieve this goal. In mathematical language, it means that
*ϵ*-dense in
*ϵ*-dense property (Theorem 2.14), and it seems that arithmetic CNNs also satisfy the
*ϵ*-dense property by numerical computation (Conjecture 3.8). We believe that further
interesting applications of the results presented (or of the generalizations) can
be obtained.

We organize the material in this paper as follows. Section 2 introduces the concepts of general inhomogeneous CNN-based LDS and constant-type multiple CNNs. Stability, partition of the parameter space and the equivalence of mosaic solutions with a multiple shift space are discussed therein. This together with the exact number of mosaic solutions under the boundary constraint (Lemma 2.12) is used to derive the entropy formula and entropy preservation property. Parallel discussions for arithmetic-type multiple CNNs can be found in Section 3. Some one- and two-dimensional examples are addressed in Section 4, and we leave the discussion in Section 5.

### 2 Constant cellular neural networks

In this section, we investigate a specified type of inhomogeneous LDS named *constant-type multiple cellular neural network* (*constant CNN*). To clarify the elucidation, Section 2.1 concentrates on the constant CNNs with
nearest neighborhood. The general cases of constant CNNs and deeper architecture are
investigated in the rest of this section.

#### 2.1 Constant cellular neural networks with nearest neighborhood

First we consider the LDS realized as

for

A system of ordinary differential equations is called *completely stable* if each of its solution **x** approaches an equilibrium state. Let

where

**Theorem 2.1***A constant CNN is completely stable if*, *for*
*one of the following conditions is satisfied*.

S1
*is symmetric*.

S2
*and*
*for all**i*, *j*, *where*

The complete stability of (1) demonstrates that the investigation of the equilibrium
solutions is essential. To make the discussion more clear, we focus on the *mosaic solutions*, *i.e.*,
*i*, and study the complexity of the output space

•
*n*.

•

To achieve our target, we introduce the *ordering matrix* and *transition matrix* first. The ordering matrix is defined as

herein the pattern ‘−’ stands for the state

For

where
*n* in *X*. Yielding

**Theorem 2.2***Suppose*
*for some*
*and*
*are the transition matrices of*
*and*
*respectively*. *Then*

*where*
*for any nonnegative matrix*
*Moreover*, *the topological entropy of***Y***is*

*where*
*and*
*are the spectral radii of*
*and*
*respectively*.

In the meantime, it is natural to elucidate the influence of boundary conditions on
the exact number of patterns of length *n* and topological entropy. Three types of boundary conditions, periodic, Neumann, and
Dirichlet boundary conditions, are considered. To reflect the influence of the boundary
conditions, we introduce three boundary matrices. Let

The periodic boundary matrix

The Neumann boundary condition infers zero flux on both sides of the space. The left and right Neumann boundary matrices are then defined by

respectively. Furthermore, the Dirichlet boundary condition indicates that both sides of the space are constant states and the corresponding boundary matrices are

Herein
*i.e.*,
*i.e.*,

**Definition 2.3**

1. Suppose that

2. Suppose that

With the introduction of the boundary matrices and the Kronecker and Hadamard products, we obtain Theorem 2.4 which reveals the formulae of exact number of patterns and topological entropy under the influence of three kinds of boundary conditions. The extension of Theorem 2.4 for general constant CNNs is demonstrated by Lemma 2.12 and Theorem 2.13.

**Theorem 2.4***Suppose*
*for some*
*and*
*are the transition matrices of*
*and*
*respectively*. *Then*
*if*
*and*
*are primitive matrices*. *Furthermore*, *the exact number of patterns of length**n**with boundary condition*
*are as follows*:

• *The periodic boundary condition*:

• *The Neumann boundary condition*:

*Herein*

• *The Dirichlet boundary condition*:

*Herein*
*relate to the conditions that the patterns on the boundary are ‘*−*’ and ‘*+*’*, *respectively*.

#### 2.2 Stability of constant cellular neural networks

The rest of this section extends the results in Section 2.1. To make the paper compact, we introduce the general setting for multi-dimensional inhomogeneous LDS and then concentrate on the one-dimensional case. The elucidation of multi-dimensional systems will be investigated in another paper.

A *D*-dimensional inhomogeneous CNN-based LDS is realized as

where
*neighborhood* for neuron
*output function*;
*threshold*, and the *feedback template*

An inhomogeneous CNN-based LDS is called a constant CNN if the neighborhood , the template , and **z** are periodic up to shifts. More precisely, there exists

It is seen that the constant CNNs generalize the concept of the classical CNNs that
were introduced in [1,35]. More precisely, a classical CNN is a constant CNN with

where
*mosaic solution* if
*mosaic pattern*. A system of ordinary differential equations is said to be *completely stable* if every trajectory tends to an equilibrium point. Theorem 2.5 infers that a constant
CNN is a completely stable system. (We remark that Theorem 2.5 is an extension of
Theorem 2.1.)

**Theorem 2.5***Suppose that*
*is the template of* (7) *and the system is written as*

*Then a constant CNN is completely stable if*, *for*
*one of the following conditions is satisfied*.

(1)
*is symmetric*.

(2)
*is nonsingular and*
*where*
*is defined in* (10).

Let
*ℓ* non-overlapping subspaces

Equation (7) can then be restated as

(It is easily seen that

**Theorem 2.6** ([36])

*A classical CNN with symmetric feedback template is completely stable*.

For the case that the feedback template is not symmetrical, suppose that a CNN with
*n*-neurons is described as follows:

where
**A** is an

**Theorem 2.7** ([37])

*Let**K**be an*
*matrix satisfying*

*for*
*A classical CNN with asymmetric feedback template is completely stable if**K**is nonsingular and*
*herein a matrix*
*means that*
*for all**i*, *j*.

It comes immediately from Theorem 2.7 that if the feedback template
*r* such that

*Proof of Theorem 2.5* Suppose
*K* defined in (10) is nonsingular and
*ℓ* independent CNN subsystems, the complete stability of a constant CNN comes from the
complete stability of every subsystem. □

For a fixed template, the collection of mosaic patterns
*output space* of (7). Since the neighborhood
*i*, the output space is determined by the so-called *admissible local patterns*. Suppose that *y* is a mosaic pattern, for each

and the necessary and sufficient condition for

Set

The set of admissible local patterns ℬ of a constant CNN is then

Similar to the discussion in [17], the output space **Y** can be represented as

(Recall that in the above equation,

One of the important research issues in the circuit theory is the learning problem.
That is to say, mathematically, for what and how many phenomena the constant CNNs
are capable of exhibiting. Theorem 2.9 infers that once
**z** so that the basic sets of admissible local patterns

**Theorem 2.8** ([34])

*There is a positive integer*
*and a unique set of open subregions*
*satisfying*

(i)

(ii)
*if*

(iii)
*and*
*for some**k**if and only if*

*Here*
*is the closure of**P**in*

Let

**Theorem 2.9** (Separation property)

*There is a positive integer**K**and a unique set of open subregions*
*satisfying*

(i)

(ii)
*if*

(iii)
*and*
*for some**k**if and only if*

*Proof* Similar to the proof of Theorem 2.5, a constant CNN is reduced to a classical CNN
whenever
_{j}, the subsystem of (7) restricting to the cells

Let

which is invariant under *σ*. The set
**Y** of a constant CNN is decomposed into subspaces
**Y** is topologically conjugated to the direct product of the output spaces

**Theorem 2.10***Given a set of templates*
*where*
*and*
*Let***Y***be the solution space of the constant CNN with respect to*
*Then*

*if*
*and*
*for*
*where* Ω *is a SFT that comes from the output space of the classical CNN with respect to template*

#### 2.3 Boundary effect on constant cellular neural networks

This subsection elucidates the influence of the boundary condition on the exact number
of mosaic patterns of finite length and on the growth rate as the length increases.
The investigation starts with formulating the number of patterns. Denote by

(i) (7)_{n}-N: constant CNNs with Neumann boundary condition on

(ii) (7)_{n}-P: constant CNNs with periodic boundary condition on

(iii) (7)_{n}-D: constant CNNs with Dirichlet boundary condition on

These boundary conditions are discrete analogues of the ones in PDEs; to be specific,
a pattern

Since
^{a} (*cf.*[17] and Section 4). Lemma 2.11 elucidates the exact number of mosaic patterns of length
*n* of a constant CNN without the influence of the boundary condition. The verification
is straightforward and is omitted.

**Lemma 2.11***For*
*write*
*for some*
*and*
*Then*

*where*
*and*
*denotes the number of patterns of length**q**in**X*.

Let
*n* with boundary condition *B*, where
*D* stands for the periodic, Neumann, and Dirichlet boundary conditions, respectively.
To find the exact number

(i) Periodic boundary matrix

(ii) Dirichlet boundary matrices

(iii) Neumann boundary matrices

Here ⊗ is the Kronecker product, *E* is a
*I* is the
*M* is a
*M* by setting each of the lower-/upper-half rows as a zero vector.

Recall that a set function
*E* being a nonempty subset of ℝ. For

It is seen that
*n* of constant CNNs with boundary conditions, we introduce some notations first. Suppose

and

Herein
*M*. Lemma 2.12 demonstrates the explicit formulae of the number of patterns of length
*n* with boundary conditions.

**Lemma 2.12***Let*
*where*
*Suppose*
*then the exact number*
*with boundary condition*
*are as follows*:

(i) *The periodic boundary condition*:

(ii) *The Dirichlet boundary condition*:

*where*
*means the pattern on the boundary is ‘*
*’*.

(iii) *The Neumann boundary condition*:

*and*

*otherwise*.

*Here*
*is a*
*matrix with entries being* 1*’s*, *and* ∘ *means the Hadamard product*.

*Proof* We address the proof of

Suppose that

At the same time,
*j*. A straightforward examination demonstrates that

and

If

where

with

This derives

and completes the proof. □

Next, to study the influence of boundary conditions on the exact number of patterns
of finite length, we consider the effect on the growth rate of the number of patterns;
more specifically, the topological entropy of the output space **Y**. The topological entropy
*X* is defined by

The existence of

**Theorem 2.13**
*Moreover*,
*for*
*provided*
*is mixing for all*

*Proof* For

Lemma 2.11 infers that

Applying the squeeze theorem, we have

This completes the first part of the proof.

To evaluate the boundary effect on the topological entropy of **Y**, we demonstrate that
*τ* denote the smallest integer such that

Suppose

if

otherwise. On the other hand, it is easily checked that

The above observation derives that

and thus we have

The following theorem comes immediately from Theorem 2.13, the proof is omitted.

**Theorem 2.14***The set of topological entropies of the constant CNNs is dense in the closed interval*
*More precisely*, *given*
*and*
*there exists a constant CNN such that*

### 3 Arithmetic cellular neural networks

This section elucidates another kind of inhomogeneous CNN-based LDS named *arithmetic-type multiple cellular neural network* (*arithmetic CNN*). It is seen that the templates of a constant CNN are periodic; in other words, the
number of distinct templates is finite. This section investigates inhomogeneous CNNs
whose number of distinct templates is infinite. First we consider a one-dimensional
LDS with nearest neighborhood to interpret the idea of our methodology, then the derived
results are generalized to general cases in the rest of this section.

#### 3.1 Arithmetic cellular neural networks with nearest neighborhood

To clarify the study of an inhomogeneous LDS with nearest neighborhood, we consider the following system,

where

Similar to the discussion in the previous subsection, we demonstrate that arithmetic
CNNs are completely stable. Let
*q*. Express (22) as

where

**Theorem 3.1***An arithmetic CNN is completely stable if*
*and*
*for all**q*, *i*, *j*, *where*

Following the complete stability of an arithmetic CNN is the spatial complexity of
the output space and the influence of boundary conditions. Note that the output space

After redefining the ordering matrix, we obtain a sequence of transition matrices

**Theorem 3.2***Suppose that***Y***is the output space of an arithmetic CNN*. *Then*

*where**q**is odd and*
*is the Gauss function*. *Furthermore*, *the topological entropy of***Y***is*

For the influence of the boundary conditions, we define the boundary matrices as follows. Let

The periodic boundary matrix

and the left and right Neumann boundary matrices are then defined by

respectively. Furthermore, the Dirichlet boundary matrices are

To simplify the formulae of

**Theorem 3.3***Suppose*
*for some**k**and***Y***is the output space of an arithmetic CNN*. *Then*
*if*
*are primitive for all**q*. *Furthermore*, *the exact number of patterns of length**n**with boundary condition*
*are as follows*:

• *The periodic boundary condition*:

• *The Neumann boundary condition*:

• *The Dirichlet boundary condition*:

*where*

#### 3.2 Stability of arithmetic cellular neural networks

The rest of this section considers the inhomogeneous CNN-based LDS with the neighborhood
consisting of infinitely many elements. A *D*-dimensional inhomogeneous CNN-based LDS is called an arithmetic CNN if the neighborhood
, the template , and the threshold **z** are periodic up to a multiplication. More precisely, there exists a positive integer
^{b}

Herein

The essential description of a one-dimensional arithmetic CNN is that

where

Let

where

In this case, the feedback template

**Theorem 3.4***Suppose that an arithmetic CNN is presented as*

*Then the system is completely stable if*
*for all*
*where*
*comes from*
*defined in* (10).

Suppose that *y* is a mosaic pattern; for each

and the necessary and sufficient condition for

Set

The set of admissible local patterns ℬ of an arithmetic CNN is then

The output space **Y** is then represented as

Recall that the output space **Y** of a constant CNN can be decomposed into finitely many subspaces
*j*. In other words, the output space of a constant CNN extends the concept of SFTs.
The output space of an arithmetic CNN is decomposed into countable subspaces; more
precisely,
*multiplicative shifts*.

In [22], the authors introduced the concept of multiplicative subshifts in the context of
symbolic dynamical systems. Let Ω be a subshift of

which is invariant under the action of multiplicative integers:

Then

A straightforward examination indicates that the output space **Y** of an arithmetic CNN is a multiplicative subshift if the neighborhood and the templates
of (27) are invariant; restated,

**Theorem 3.5***Given a set of templates*
*Let***Y***be the solution space of the arithmetic CNN with respect to*
*Then***Y***is a multiplicative subshift if*
*and*
*for all*
*More precisely*,

*where* Ω *is the SFT that comes from the output space of the classical CNN with respect to the
template*

#### 3.3 Boundary effects on arithmetic cellular neural networks

Recall that a set function
*E* is a nonempty subset of ℝ. For

It is seen that both
*n* of an arithmetic CNN with boundary condition, we introduce some notations first.
Set

Recall that

The exact number of patterns of the arithmetic CNNs with boundary condition is obtained
via a small modification of the discussion in the proof of Lemma 2.12. Before presenting
the formulae, we assume that
*j* and redefine the boundary matrices as follows. Suppose that *E* is a

and

respectively. The left and right Dirichlet boundary matrices are defined as

where *I* is the

**Lemma 3.6***Suppose*
*then*

*Suppose*
*where*
*and*
*Then*:

(i) *The periodic boundary condition*:

(ii) *The Dirichlet boundary condition*:

(iii) *The Neumann boundary condition*:

Theorem 3.7 formulates the topological entropy of the output space of an arithmetic CNN with/without boundary conditions.

**Theorem 3.7***Suppose that there exists*
*such that*
*for*
*Then*

*where*
*is defined in* (33). *Furthermore*,
*for*
*provided*
*is mixing for*

*Proof* The calculation of

Observe that

Hence we have

This completes the proof. □

The numerical experiment asserts that, similar to Theorem 2.14, the set of topological
entropies of the arithmetic CNNs is dense in the closed interval

**Conjecture 3.8***Given*
*and*
*there exists an arithmetic CNN such that*

### 4 Examples

#### 4.1 One-dimensional cellular neural networks

**Example 4.1** Consider a constant CNN with templates
**z** being given by

(Notably,

respectively. Theorem 2.13 infers that

where

To estimate the exact number of the mosaic patterns of length *n* with boundary conditions, we consider the case where

The periodic and Neumann boundary matrices are then

and