THE CNN PARADIGM: SHAPES AND COMPLEXITY
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
August 2, 2005 9:44 01330
Tutorials and Reviews
International Journal of Bifurcation and Chaos, Vol. 15, No. 7 (2005) 2063–2090
c World Scientific Publishing Company
THE CNN PARADIGM: SHAPES AND COMPLEXITY
PAOLO ARENA, MAIDE BUCOLO, STEFANO FAZZINO,
LUIGI FORTUNA∗ and MATTIA FRASCA
Dipartimento di Ingegneria Elettrica Elettronica e dei Sistemi,
Universitá degli Studi di Catania, viale A. Doria 6, 95123 Catania, Italy
∗lfortuna@dees.unict.it
Received October 8, 2004; Revised November 12, 2004
The paper stresses the universal role that Cellular Nonlinear Networks (CNNs) are assuming
today. It is shown that the dynamical behavior of 3D CNN-based models allows us to approach
new emerging problems, to open new research frontiers as the generation of new geometrical
forms and to establish some links between art, neuroscience and dynamical systems.
Keywords: CNN; shape; complexity; art; neuroscience.
1. Introduction shape archetypes. Moreover, in the Still Life with
Old Shoe the idea of broken forms typical of the
Shapes are the fundamental attribute of visible
objects; they give us the perception of structure. period of analytical Cubism appears. The broken
They also represent the alphabet of art [Arnheim, forms, like bursts, are evident in the Salvator Dalı́
1974]. Shapes represent the object of visible beauty. art and in particular in Tête Raphaelesque Éclatée.
It is well known that the mathematics of shape The dynamical trends of Giorgio De Chirico’s artis-
and space is geometry. Which is the relationship tic life are well known: he founded with Carlo Carrà
between dynamical systems and shapes? Could the Methaphysical Painting. Surrealists honored his
the mathematics of nonlinear dynamics help us to early paintings, but in the Twenties he switched to
establish an innovative way to go in depth into the a Renaissance-based Classicism. He completed his
world of shape generation? artistic experience in NeoMethaphisical Painting.
The human mind process of creating shapes is Both the artist’s evolution trends and the pro-
of course a dynamical process and often its complex- cess of shapes creation are emergent phenomena.
ity involves more steps in combining emergent con- Viewing the art in each period, accurate analysis
ditions. The shape generation in the human mind allows us to identify the complexity of beauty and
leads often to the emergence status of mental states the perception of the harmony of created shapes,
and the brain-mind complexity is related also to its that are the signatures of the dynamical process of
shape organization process. the artist in his creative efforts. The evolutionary
Some artistic experiences and painters of the trends of figurative arts further remark the finger-
last century are taken as examples of the previ- prints of complexity in the shape history. Moreover,
ous concepts. The painter Joan Miró reached the if we try to establish when art arose, no definitive
third dimension in his “Still Life with Old Shoe” answer does exist. Art itself possibly arose before
[Lynton, 1994] that represents a bifurcation point the birth of history. The examples of primitive
in the production of the artist whose previous art like the aboriginal cave paintings in Australia,
paintings were characterized by subjects spatially representing hunt scenes, animals and even more
represented in two dimensions. This fact indicates abstract figures such as the half-human, half-ani-
in the artist’s mind a possible rearrangement of his mals figures called “therianthropes”, witness what
2063
August 2, 2005 9:44 01330
2064 P. Arena et al.
primitive men tried to reproduce from nature. The study. In fact, from the coupling of the single 3D-
role of this representation is vague possibly related CNN units impressive patterns and shapes emerge.
to religion, magic and so on. When did rich abstract It is shown that the evolution of the 3D-CNN
shapes appear in visual arts? Many, many doubts dynamics generates harmonious yet unpredictable
are related to this topic. shapes. It is well known that a partial differential
Aesthetics have recently presented more ideas equation can be matched in a CNN-based algorithm
in order to evaluate the beauty of art, and pro- [Chua, 1998] and that RD-CNNs are appropri-
vide a criteria in order to understand the role of ate to reproduce complex phenomena in biology,
art in history. New shapes genesis is essentially due chemistry, neurodynamics: the generalized 3D-CNN
to an emergent mechanism in the human mind. scheme proposed in this work adds further results in
One of the tasks of this paper is to prove experi- order to highlight the power of the CNN paradigm.
mentally that shapes are also emergent phenomena The paper is organized as follows. The defi-
in chaotic spatially extended systems. Moreover, nition of the 3D-CNN architecture is discussed in
emergent phenomena that occur in visual cortex Sec. 2. The mathematical details of the 3D-CNN
could also be related to the previous items. In configurations are reported in Sec. 3 where a gallery
fact, many researches on this subject focused on of obtained shapes and patterns is also reported.
this topic [Ermentrout et al., 1979; Dalhem et al., In Sec. 4 a discussion of the results shown in the
2000]. The hallucination phenomena arising in ill- previous section is included. In the conclusion, the
ness like migraine lead to a generation of new forms technological revolution of the CNN paradigm is
known as phosphenes or scintillating scotoma, for- emphasized. The appendix includes the technical
tifications and vortices. New forms and structures details of the Eˆ3 software tool, developed to per-
emerge in the visual cortex without external inputs. form the reported experiments.
Possibly these unpredictable events influenced pos-
itively some paints of famous artists like Giorgio De 2. 3D-CNN Model
Chirico or Vincent Van Gogh [Podoll et al., 2001;
Bogousslavsky, 2003]. Let us consider the universal definition of a CNN
Furthermore, a very appealing approach to face [Chua, 1998]: A CNN is any spatial arrangement of
the visual cortex behavior has been introduced locally coupled cells where each cell is a dynamical
in [Zeki, 1999], where the neuroscientist discusses system which has inputs, and outputs and a state
the understanding of both the single visual cor- evolving in accordance with the described dynami-
tex areas and its global organization, by analyz- cal laws.
ing various artists’ paints. In this last remark, the The following general concepts will be
twofold link between art and neuroscience appears. assumed:
In the actual literature these aspects are reinforced • the coupling laws are not generally spatially
by recent studies on surprising emergent creative invariant (even if often for practical realizations
tendencies shown in patients with strong mental these are assumed invariant);
diseases [Giles, 2004]. Shapes are the links in this • the concept of dominant local coupling is assu-
important route. med; therefore most of the connections are be-
In this paper, the universality of the CNN tween cells in a neighborhood of unitary radius,
paradigm [Chua, 1998] is proved to be the appro- but some nonlocal connections could also be
priate tool to generate emergent shape patterns, included;
to relate shape evolution with spatially extended • each cell is a dynamical system with assigned
dynamical systems and therefore to open a real state variables.
bridge among circuits, art and neuroscience. The
essential prerogative of the CNN paradigm is to Let us consider an isolated cell (from the
take advantage of the cooperative behavior of sim- microscale CNN point of view). The following
ple dynamical nonlinear circuits in order to obtain variables characterize the cell:
complex global tasks. The reported study is related • the exogenous, controllable input vector ui,j,k (t)
to three-dimensional, in the space, CNN (3D-CNN). ∈ Rmu ;
Merging cells to achieve E-merging patterns and • the exogenous, uncontrollable input vector
dynamics and to emulate complex system behav- Si,j,k (t) ∈ Rms ;
iors fully expresses the 3D-CNN role in the proposed • the state vector xi,j,k (t) ∈ Rn ;
August 2, 2005 9:44 01330
The CNN Paradigm 2065
• the output vector yi,j,k (t) ∈ Rmy ; ...
• the bias vector zi,j,k (t) ∈ Rn that is generally ẋn = −xn + zn + A(n; l)yl
assumed controllable; C(l)∈Nr (n)
where i, j, k represent the space coordinates. + B(n; l)ul + C(n; l)xl
Moreover, let us introduce some useful nota- C(l)∈Nr (n) C(l)∈Nr (n)
tions. Let us indicate with X, U, Y the whole state,
input and output sets (referring to all the cells of y1 = 0.5(|x1 + 1| − |x1 − 1|
the CNN). Let us define the neighborhood of the ...
cell C(i, j, k) as yn = 0.5(|xn + 1| − |xn − 1|
Nr (i, j, k) (1)
= {C(α, β, γ)|max(|α − i|, |β − j|, |γ − k|) ≤ r} For instance, a CNN made of three first-order cells
(n = 3) may implement the dynamics of the Chua’s
and let us indicate with circuit [Arena et al., 1995] or a Colpitts-like oscilla-
Xi,j,k = xα,β,γ tor [Arena et al., 1996]. A schematic representation
C(α,β,γ)∈Nr (i,j,k) of a microscale CNN is shown in Fig. 1, where each
first-order cell is represented by a small cube and
the state variables of cells in the neighborhood
the possibility of connecting such a CNN with other
Nr (i, j, k) of the cell C(i, j, k). Analogous definitions
CNNs is sketched.
can be given for Ui,j,k and Yi,j,k .
This model can be further generalized as
The general CNN model that we introduce is
follows:
built-up by adding complexity to the simplest CNN
and takes into account the following key points: ẋ1,ijk = f1;i,j,k (xijk , yijk , uijk )
ẋ2,ijk = f2;i,j,k (xijk , yijk , uijk )
• the cells are not required to be equal to each
other; ...
• in a 3D grid the coupling laws are locally ẋn,ijk = fn;i,j,k (xijk , yijk , uijk ) (2)
described along with the neighbor cells Sα,β,γ ; y1,ijk = g1;i,j,k (xijk , yijk , uijk )
• each node, as described below, can be realized ...
using n generalized first-order cells constituting yn,ijk = gn;i,j,k (xijk , yijk , uijk )
a small multilayer CNN architecture.
where we make explicit the connections with other
More in detail, referring to the considerations CNNs through the space coordinates i, j, k.
above, we could give definitions of CNN structures
at several levels, starting from the microscale level Definition: Mesoscale CNN. By connecting sev-
to the mesoscale and macroscale levels. eral microscale CNNs (2) in a 3D grid, a mesoscale
Definition: Microscale CNN. A CNN obtained
by connecting first-order cells may implement any
nonlinear dynamics [Fortuna et al., 2003]. The
microscale CNN is introduced to account for this
concept. It can be described by the following equa-
tions:
ẋ1 = −x1 + z1 + A(1; l)yl
C(l)∈Nr (1)
+ B(1; l)ul + C(1; l)xl
C(l)∈Nr (1) C(l)∈Nr (1)
ẋ2 = −x2 + z2 + A(2; l)yl
C(l)∈Nr (2)
+ B(2; l)ul + C(2; l)xl
C(l)∈Nr (2) C(l)∈Nr (2) Fig. 1. Schematic representation of a microscale CNN.
August 2, 2005 9:44 01330
2066 P. Arena et al.
CNN can be obtained. A microscale CNN is now a Definition: Macroscale CNN. We now add to this
cell of a mesoscale CNN. The cell equations for a structure the possibility of having long-range con-
mesoscale CNN are defined as follows: nections. By including in Eq. (3) these terms, the
ẋijk = fi,j,k (xijk , yijk , uijk ) + ai,j,k (Yijk ) macroscale model of a CNN architecture assumes
the following form:
+ bi,j,k (Uijk ) + ci,j,k (Xijk ) (3)
yijk = gi,j,k (xijk , yijk , uijk )
ẋijk = fi,j,k (xijk , yijk , uijk ) + ai,j,k (Yijk )
where ai,j,k (Yijk ), bi,j,k (Uijk ), ci,j,k (Xijk ) represent + bi,j,k (Uijk ) + ci,j,k (Xijk )
the coupling terms. It can be noticed that connec- (4)
+ alr lr lr
i,j,k (Y) + bi,j,k (U) + ci,j,k (X)
tions among cells are only local. A schematic view
of a mesoscale CNN is shown in Fig. 2. yijk = gi,j,k (xijk , yijk , uijk )
Fig. 2. Schematic representation of the generalized mesoscale model of 3D-CNN.
Fig. 3. Schematic representation of the generalized macroscale model of 3D-CNN.
August 2, 2005 9:44 01330
The CNN Paradigm 2067
where ai,j,k (Yijk ), bi,j,k (Uijk ), ci,j,k (Xijk ) represent Example: 3D-CNN with first-order cells with
local coupling terms, while alr lr
i,j,k (Yijk ), bi,j,k (Uijk ),
standard nonlinearity. A macroscale 3D-CNN
clr
i,j,k (Xijk ) take into account long-range coupling
with first-order cells with standard nonlinearity is
terms. described by the following equations:
The overall model described by Eqs. (4) is
schematized in Fig. 3, which represents a schematic
ẋijk = −xijk + zijk
view of such a CNN, in which local and long-range
+ A(i, j, k; α, β, γ)yα,β,γ
connections are allowed. Moreover, the space depen-
C(α,β,γ)∈Nr (i,j,k)
dency of each cell also includes the uncertainties in
the model of each cell. These uncertainties can be
+ B(i, j, k; α, β, γ)uα,β,γ
either parametric or structural.
This model can then be further generalized and
C(α,β,γ)∈Nr (i,j,k)
represented by the following state equation: + C(i, j, k; α, β, γ)xα,β,γ
ẋijk = fi,j,k (X, Y, U)
C(α,β,γ)∈Nr (i,j,k)
(5)
yijk = gi,j,k (X, Y, U).
+ Alr lr
i,j,k;α,β,γ (yα,β,γ ) + Bi,j,k;α,β,γ (uα,β,γ )
The overall model (macroscale CNN) schema-
lr
tized in Fig. 3 is built-up by adding complexity to
+ Ci,j,k;α,β,γ (xα,β,γ )
the simplest CNN (the microscale CNN) through
y = 0.5(|x + 1| − |x − 1|)
the mesoscale CNN in which connections are only ijk ijk ijk
local. (6)
Fig. 4. Eladio Dieste, Church of Atlantida, Atlantida, Uruguay, 1958.
August 2, 2005 9:44 01330
2068 P. Arena et al.
where A(i, j, k; α, β, γ), B(i, j, k; α, β, γ) and the painting. The unifying principle of the different
C(i, j, k; α, β, γ) are the feedback template, the con- elements is the shape that expresses the whole: the
trol template and the state template; Alr i,j,k;α,β,γ , shape is an emergent property of the whole.
lr
Bi,j,k;α,β,γ lr
and Ci,j,k;α,β,γ describe the map of the In the 3D-CNN paradigm the single contribu-
long-range connections (they are N × N × N tions from the various cells lead us to the global
matrices). Equations (6) match equations (4) if characteristics of the architecture as regards both
the following assumptions on the coupling terms the structure and the emergent dynamical behavior.
hold: There is a dichotomy in this view between CNNs
and shapes. The idea that the dynamics of a 3D-
CNN is strongly evolving is assumed. Moreover,
alr
i,j,k (Yijk ) = Alr (i, j, k; α, β, γ)yα,β,γ
we cannot directly perceive the internal dynamical
C(α,β,γ)∈Nr (i,j,k)
evolution of the single cell or of a group of cells
ai,j,k (Yijk ) = A(i, j, k; α, β, γ)yα,β,γ that leads us to understand the whole emergency of
C(α,β,γ)∈Nr (i,j,k) the CNN behavior, that could only be synthetized
into a shape. The internal global dynamics could
blr
i,j,k (Yijk ) = B lr (i, j, k; α, β, γ)yα,β,γ be only perceived by coupling to the 3D-CNN the
C(α,β,γ)∈Nr (i,j,k) vision of the shapes. A dichotomy does exist: the
bi,j,k (Yijk ) = B(i, j, k; α, β, γ)yα,β,γ shape generation capabilities of CNNs are empha-
C(α,β,γ)∈Nr (i,j,k) sized, moreover, the shape as partial abstraction
of the emergent property in a distributed nonlin-
clr
i,j,k (Yijk ) = C lr (i, j, k; α, β, γ)yα,β,γ ear dynamical system does appear. These concepts
C(α,β,γ)∈Nr (i,j,k) are the leading points of the next part of this sec-
tion where in detail both the configurations and the
ci,j,k (Yijk ) = C(i, j, k; α, β, γ)yα,β,γ
visual representations of the generalized 3D-CNN
C(α,β,γ)∈Nr (i,j,k)
paradigm are included.
The model used in the following experiments
Remark. The conceived structure, showing emerg- is related to a very simplified coupling law, where
ing pattern behaviors by using cells, indicating a a simple space-constant diffusion law in Eqs. (2) is
path from simplicity to achieve complexity, is an assumed as follows:
appealing research subject in several fields. Let
us consider the well-known architect Eladio Dieste ci,j,k (Xijk ) = D∇2ijk x (7)
[Morales, 1991]; he used the brick as the funda-
where the discretized Laplacian in a 3D space is
mental element to build his main structures in civil
defined by the following relationship:
engineering as shown, for example, in Fig. 4. The
Chua’s cell [Chua & Yang, 1988a, 1988b; Fortuna ∇2ijk x = xi−1,jk + xi+1,jk + xi,j−1,k
et al., 2003] represents the basic electronic element
+ xi,j+1,k + xij,k−1 + xij,k+1 − 6xijk . (8)
to design complex circuits exactly like the Eladio
Dieste’s brick. Moreover as the architect Frei Otto Moreover, all the cells are equal, i.e. fi,j,k (xijk ,
[Otto, 1982] considered emerging forms in nature as uijk ) = f (xijk , uijk ).
building paradigms to be realized in civil structures, With these assumptions Eqs. (2) can therefore
in a dual manner, the search for complex patterns be rewritten as follows:
and their circuit realization by means of electronic
structures find the link in the CNN architecture. ẋijk = f (xijk ) + D(xi−1,j,k + xi+1,j,k + xi,j−1,k
+ xi,j+1,k + xi,j,k−1 + xi,j,k+1 − 6xijk )
3. Emergence of Forms in 3D-CNNs which matches the well-known paradigm of
reaction–diffusion equations:
Looking at a picture and in general at an artistic
representation, a great number of impressions are ẋ = f (x) + D∇2 x.
received, all at the same time. Let us consider a
painting: a lot of elements appear, the impressions The experiments discussed below have been
that are received combining the various elements performed considering different dynamical laws for
give the perception of the specific characteristic of each cell of the CNN. In most of the cases, the
August 2, 2005 9:44 01330
The CNN Paradigm 2069
behavior of each cell is chaotic. The coupling law The definition of the function γ makes use of
is characterized by low coupling coefficient (weak complex curves used for the study of topological
diffusion). In all the experiments, zero-flux bound- forms. For instance, a (m : n) torus knot can be
ary conditions have been chosen. represented by the following equations
The experiments will be detailed in the fol-
lowing. In all of them the evolution of the sys- Z = γ(i, j, k)
tem leads to the emergence of self-organized forms. 2 m n
The rich, unpredictable, beautiful dynamics of the d −1 2k 2i 2j
= + î − + î
global forms arising in the various experiments is 1 + d2 1 + d2 1 + d2 1 + d2
related to the model used for the cell of the 3D sys-
tem. The chaotic behavior of each cell, represented with d = i2 + j 2 + k2 . Then, initial conditions for
by the beauty of the strange attractor, is reflected the state variables x, y, z of each nonlinear unit of
in the beauty of the 3D forms like those shown in the 3D-CNN are created from Eqs. (9) through the
the gallery of reported models. relations:
The evolutionary forms reported in the various
graphs are isosurfaces that have been obtained in
time in the 3D space defined by the spatial coordi- x0 (i, j, k) = Ax Re(Z) + Bx
nates i, j, k. y0 (i, j, k) = Ay Im(Z) + By
As concerns the cell dynamics f (xijk ), differ- z0 (i, j, k) = Az Re(Z) + Bz
ent chaotic laws have been simulated. In particular,
the Lorenz system, the Rossler system [Strogatz,
where Ax , Ay , Az , Bx , By and Bz are real con-
2000] and the Chua’s circuit [Chua, 1998] have been
stants used to scale the initial conditions to match
investigated. Moreover, the chaotic dynamics of sev-
the dynamic range of the nonlinear units constitut-
eral neuron models have been taken into account in
ing the 3D-CNN.
order to emulate the global behavior of neural net-
works in a 3D space.
3.2. 3D waves in homogeneous and
3.1. Initial conditions unhomogeneous media
Different initial conditions for the various experi- First of all, let us consider a 3D-CNN, where
ments have been chosen and will be discussed in each cell is a second-order nonlinear system, imple-
detail in the following. However, the idea underly- menting a reaction–diffusion. Two examples are
ing this choice is common to all the examples and discussed. The first deals with an homogeneous
will be briefly introduced in this section. medium, the second is an example of unhomoge-
Initial conditions x0 (i, j, k) have been created neous medium.
starting in some topological form. In particular, The equations of the generic second-order cell
they can be viewed as the composition of two func- Cijk are the following:
tions σ and γ, describing the topological form taken
into account.
ẋi,j,k;1 = k(−xi,j;1 + (1 + µ + ε)yi,j;1 − syi,j;2
Let us focus on a 3D-CNN made of third-order
+ i1 + D1 (yi−1,j,k;1 + yi+1,j,k;1
nonlinear units (as in most cases investigated in this
paper) and let us indicate by i, j,√k the three coor-
+ yi,j−1,k;1 + yi,j+1,k;1 + yi,j,k−1;1
dinates of the 3D space and î = −1.
+ yi,j,k−1;1 − 6yi,j,k;2 ))
Thus, the two functions σ and γ can be defined (9)
as follows:
ẋi,j,k;2 = k(−xi,j;2 + (1 + µ − ε)yi,j;2 + syi,j;1
γ : R3 ⊇ Ω → C
+ i2 + D2 (yi−1,j,k;2 + yi+1,j,k;2
σ : C → R3
+ yi,j−1,k;2 + yi,j+1,k;2 + yi,j,k−1;2
+ yi,j,k−1;2 − 6yi,j,k;2 )).
and initial conditions are given by the following
relation:
The parameters of the CNN cell have been cho-
x0 (i, j, k) = σ ◦ γ. sen according to µ = 0.7, s = 1, i1 = −0.3, i2 = 0.3,
August 2, 2005 9:44 01330
2070 P. Arena et al.
D1 = 0.1, D2 = 0.1. For these values, the 3D-CNN emergence of spiral waves. This experiment repre-
behaves as a nonlinear medium in which autowaves sents a fascinating emergent behavior shown by a
propagate. The parameter k accounts for the possi- complex system.
ble unhomogeneity of the medium.
In the first example, all the cells in the 40 × 3.3. Chua’s circuit
40 × 40 3D-CNN have the same values of k
(k = 1). In the center of the 3D-CNN, there is a This example deals with the Chua’s circuit [Chua,
“pacemaker” cell whose outputs are fixed to the 1998]. In the case of a 3D-CNN made of Chua’s cir-
values y20,20,20;1 = 1 and y20,20,20;2 = −1. This cell cuits, Eqs. (9) can be rewritten as follows:
elicits the generation of autowaves in the neighbor- ẋijk = α(yijk − h(xijk )) + D∇2ijk x
ing cells. Figure 5 shows the behavior of a 3D-CNN
ẏijk = xijk − yijk + zijk (10)
made of cells represented by Eqs. (9). As it can be
noticed, when two wavefronts collide they annihi- żijk = βyijk
late each other.
where
In the second example, the cells of the 3D-
CNN have different values of the parameter k. The h(x) = 0.5((s1 + s2 )x + (s0 − s1 )(|x − B1 | − |B1 |)
value k = 0.6 characterizes “slow” cells, while the
+ (s2 − s0 )(|x − B2 | − |B2 |)) + ε
value k = 1 characterizes “fast” cells as schemat-
ically shown in Fig. 6(a). The whole CNN con- and the diffusion term only acts on the first state
sists of 41 × 41 × 41 cells. Simulation results are variable xijk (t). An array of 80 × 80 × 80 chaotic
shown in Fig. 6. Figure 6(a) shows the initial con- units has been considered, i.e. 1 ≤ i, j, k ≤ 80 in
figuration: we simulated an initial point of excita- Eqs. (10). The parameters of each single unit have
tion (indicated by an arrow) and a “wall” in an been chosen according to α = 9, β = 30, s0 = −1.7,
unhomogeneous 3D medium. Figures 6(b)–6(d) rep- s1 = s2 = −1/7, ε = −1/14, B1 = −1 and B2 = 1,
resent the evolution of the RD-CNN. The presence in order to set the Chua’s circuit in the bistability
of unhomogeneity in the medium clearly leads to the region. The diffusion coefficient has been fixed to
Fig. 5. Behavior of a 3D-CNN generating autowaves in a homogeneous medium.
August 2, 2005 9:44 01330
The CNN Paradigm 2071
(a) (b)
(c) (d)
Fig. 6. Behavior of a 3D-CNN generating spiral waves in an unhomogeneous medium.
the value D = 0.1. In order to visualize the behavior way. Figure 7 shows some frames of the evolution
of the whole 3D-CNN, we considered an isosurface of a 3D-CNN made of Chua’s circuits, where the
defined by xijk = 0.1. The emergent behavior leads formation of shapes and structures evolving in time
to the formation evolving in time in a nonrepetitive is evident.
Fig. 7. Forms obtained by a 3D-CNN made of Chua’s circuits.
August 2, 2005 9:44 01330
2072 P. Arena et al.
Figure 7 refers to the following initial attractor. The diffusion coefficient has been fixed to
conditions: the value D = 0.1. In order to visualize the behavior
of the whole 3D-CNN, we considered a level surface
z = γ(i, j, k) defined by xijk = 0. Figure 10 shows some frames
2 7 of the evolution of a 3D-CNN made of Rossler sys-
d −1 2k 2i 2j 5
= + î − + î tems, where the formation of forms and structures
1 + d2 1 + d2 1 + d2 1 + d2 evolving in time is evident.
σ(z) = (4 Re z − 1.25, Im z, 4 Re z + 1.25). Initial conditions have been chosen as follows:
Moreover, the stretching and folding dynamics z = γ(i, j, k)
appears in the frames shown in Fig. 8. 2
d −1 2k 2i 2j
= + î + î
3.4. Lorenz system 1 + d2 1 + d2 1 + d2 1 + d2
3
The example reported in Fig. 9 deals with a CNN 2i 2j 5 d2 − 1 2k
+ + î − + î
made of 60 × 60 × 60 chaotic Lorenz systems des- 1 + d2 1 + d2 1 + d2 1 + d2
cribed by the following equations [Strogatz, 2000]:
σ(z) = (4 Re z − 0.25, Im z, 4 Re z + 0.25).
ẋijk = σ(yijk − xijk ) + D∇2ijk x
ẏijk = rxijk − yijk + xijk zijk + D∇2ijk y (11) 3.6. FitzHugh–Nagumo neuron model
żijk = xijk yijk − bzijk The first neuron model investigated is the
where the parameters have been chosen according FitzHugh–Nagumo (FHN) model [FitzHugh, 1961;
to σ = 10, r = 28, b = 8/3 in order to set the well- Nagumo et al., 1960] of spiking neurons described
known butterfly attractor. The diffusion coefficient by the following equations:
has been fixed to the value D = 0.5.
uijk + b
Initial conditions have been chosen as follows: v̇ijk = εvijk (1 − vijk ) vijk − + D∇2ijk v
a
z = γ(i, j, k) u̇ijk = vijk − uijk
2 6 (13)
d −1 2k 2i 2j 9
= + î − + î and the diffusion term only acts on the first variable.
1 + d2 1 + d2 1 + d2 1 + d2
The parameters have been chosen according to:
σ(z) = (4 Re z − 1.25, Im z, 4 Re z + 1.25) a = 0.75, b = 0.01, ε = 50. An array of 50 × 50 × 50
Figure 9 shows the isosurface defined by neurons (13) coupled with a diffusion coefficient
xijk = 2. D = 1 has been taken into account. Some frames of
Even in this case the evolution of the system the evolution of the isosurface defined by xijk = 0.5
leads to ever changing regular forms. are shown in Fig. 11.
3.5. Rossler system 3.7. Hindmarsh–Rose neuron model
The emergence of organized forms and structures A 3D-CNN where the basic cell is the Hindmarsh–
has been also observed in a 3D-CNN of Rossler units Rose model [Rose & Hindmarsh, 1989] of burst-
[Strogatz, 2000] as follows: ing neurons is discussed here. The dynamics of this
model is described by the following equations:
ẋijk = −yijk − zijk + D∇2ijk x
ẋijk = yijk + ax2ijk − x3ijk − zijk + I + D∇2ijk x
ẏijk = xijk + ayijk (12)
żijk = b + xijk zijk − czijk ẏijk = 1 − bx2ijk − yijk (14)
żijk = r(S(xijk − xc ) − zijk )
where the diffusion term only acts on the first state
variable xijk (t). An array of 60 × 60 × 60 chaotic where a diffusion term acting on the first variable
units has been considered. The parameters of each has been included.
single unit have been chosen according to a = 0.2, The parameters have been chosen according to:
b = 0.2, c = 5, in order to set the Rossler chaotic a = 3, b = 5, r = 0.0021, S = 4, xc = −1.6, andAugust 2, 2005 9:44 01330
The CNN Paradigm 2073
Fig. 8. Stretching and folding dynamics in 3D-CNN made of Chua’s circuits.
I = 0 and 30 × 30 × 30 neurons (14) have been 3.8. Inferior-Olive neuron model
coupled with a diffusion coefficient D = 0.75 lead- This neuron model was proposed in [Giaquinta
ing to very interesting results. Some frames of the et al., 2000] to mimic the behavior of Inferior-
evolution of the isosurface defined by xijk = 0.5 are Olive (IO) neurons. They are characterized by sub-
shown in Figs. 12 and 13. threshold oscillations. The dimensionless equationsAugust 2, 2005 9:44 01330
2074 P. Arena et al.
Fig. 9. Evolution of the isosurface xi,j,k = 2 generated by a 3D-CNN made of Lorenz systems.August 2, 2005 9:44 01330
The CNN Paradigm 2075
Fig. 10. Forms generated by a 3D-CNN made of Rossler systems.
describing this model are the following: d2
σ(z) = (2Re z − 0.4 , Im z + 0.4,
D2
xijk (xijk − γ)(1 − xijk ) − yijk sin((Re z)2 − (Im z)2 )
ẋijk = + D∇2ijk x
ε √
2 − r 2 ) + D∇2 y
with d = i2 + j 2 + k2 and D = M 2 + N 2 + P 2 .
ẏijk = −Ωzijk + rijk (A − zijk ijk ijk
2 − r 2 ) + D∇2 z
żijk = Ωrijk + zijk (A − zijk ijk ijk 3.9. Izhikevich neuron model
(15) A recent neuron model was proposed by Izhikevich
in order to conjugate accuracy of the model
with rijk = (yijk /M ) − x. and computational resources needed to simulate
The parameters have been chosen according to: large arrays of neurons [Izhikevich, 2003]. The
ε = 0.01, γ = 0.2, M = 0.5, A = 0.0006, Ω = −1.6. model accounts both for different spiking behav-
The CNN consists of 40 × 40 × 40 neurons (15) cou- iors (tonic, phasic and chaotic spiking) and for
pled with a diffusion coefficient D = 0.001. Some bursting behavior,dependingon the parameters cho-
frames of the evolution of the isosurface defined by sen. It can be described by the following equations
xijk = −0.15 are shown in Fig. 14. [Izhikevich, 2003]:
2 + 5v 2
Initial conditions have been chosen as follows: v̇ijk = 0.04vijk ijk + 140 − uijk + I + D∇ijk v
u̇ijk = a(bvijk − uijk )
z = γ(i, j, k)
(16)
2 7−2 d22
d −1 2k D with the spike-resetting
= + î v←c
1 + d2 1 + d2 if v ≥ 30 mV, then
5+3 d22 u←d
2i 2j D
v and u are dimensionless variables, and a = 0.2,
− 2
+ î 2
1+d 1+d b = 2, c = −56, d = −16, and I = −99 are theAugust 2, 2005 9:44 01330
2076 P. Arena et al.
Fig. 11. Shapes generated by a 3D-CNN made of FHN neurons.August 2, 2005 9:44 01330
The CNN Paradigm 2077
Fig. 12. Frames of the evolution of a 3D-CNN made of HR neurons.
parameters (chosen to set a chaotic spiking activity (vijk (0), uijk (0))
[Izhikevich, 2004]). The simulation of the 3D-CNN
(−56, −112) if rr < 5 and i > 0
made of Izhikevich neurons shown in Fig. 15 has
been carried out by considering 30 × 30 × 30 units, = (20, 40) if rr < 9 and i < 0
D = 0.01 and an isosurface defined by vijk = −65.4. (0, 0) otherwise
Initial conditions have been chosen as
follows: with r = i2 + j 2 + k2 .August 2, 2005 9:44 01330
2078 P. Arena et al.
Fig. 13. Frames of the evolution of a 3D-CNN made of HR neurons.
3.10. Neuron model exhibiting model shows Shilnikov chaos and can be also taken
homoclinic chaos as representative of a class of neuron dynamics
with chaotic inter-spike intervals. The following
Another experiment was carried out by using a dimensionless equations describe the behavior of
cell model based on a CO2 laser model. This this model:August 2, 2005 9:44 01330
The CNN Paradigm 2079
Fig. 14. Frames of the evolution of a 3D-CNN made of IO neurons.August 2, 2005 9:44 01330
2080 P. Arena et al.
Fig. 15. Shapes generated by a 3D-CNN made of Izhikevich neurons.August 2, 2005 9:44 01330
The CNN Paradigm 2081
ẋ1,ijk = k0 x1,ijk (x2,ijk − 1 − k1 sin2 (x6,ijk )) β = 0.4286, α = 32.8767, P0 = 0.016, B0 =
0.133. For this set of parameters, homoclinic chaos
+ D∇2ijk x1
appears.
ẋ2,ijk = −Γ1 x2,ijk − 2k0 x1,ijk x2,ijk + γx3,ijk We considered an array of 30×30×30 units dif-
+ x4,ijk + P0 + D∇2ijk x2 fusively connected with D = 0.01. Figures 16 and 17
show some frames of the evolution of a 3D-CNN
ẋ3,ijk = −Γ1 x3,ijk + x5,ijk + γx2,ijk + P0 (17)
made of units with Eq. (17) starting from initial
ẋ4,ijk = −Γ2 x4,ijk + γx5,ijk + zx2,ijk + zP0 conditions chosen as follows:
ẋ5,ijk = −Γ2 x5,ijk + zx3,ijk + γx4,ijk + zP0
k1 if rr < 5 and i > 0
Rx1,ijk
ẋ6,ijk = −βx6,ijk + βB0 − β xijk (0) = k2 if rr < 9 and i < 0
(18)
1 + αx1,ijk
k3 otherwise
Parameters have been chosen according to
[Pisarchik et al., 2001; Ciofini et al., 1999] as fol- where r = i2 + j 2 + k2 and k1 , k2 and k3 are
lows: R = 220, k0 = 28.5714, k1 = 4.5556, vectors of six constants. The isosurface shown in
Γ1 = 10.0643, Γ2 = 1.0643, γ = 0.05, z = 10, Figs. 16 and 17 is defined by x1,ijk = 403 ∗ 10−3 .
Fig. 16. Frames of the evolution of a 3D-CNN made of neurons [Eq. (17)] with homoclinic chaos.August 2, 2005 9:44 01330
2082 P. Arena et al.
Fig. 17. Frames of the evolution of a 3D-CNN made of neurons [Eq. (17)] with homoclinic chaos.
The movies of the experiments discussed above Let us consider a system in which each cell is a
can be downloaded from the webpage www.scg. random generator with a given probability distribu-
dees.unict.it/activities/complexity/CNNindex.html. tion and let us consider the same coupling diffusive
laws and grid dimension of the other experiments.
Figure 18 shows the results obtained by simulat-
4. General Discussion: Remarks and ing such a system. In this case, the level surface
Considerations is irregular and there is no clear form arising. For
First of all, a general remark regarding the previous regularity, self-organization is not possible.
experiments must be made: shapes in 3D-CNNs are Let us consider the various shape trends shown
the fingerprint of emergent phenomena. This occurs in the various sequences of the previous section,
for all the adopted 3D-CNN configurations. More- the following strong observation is possible: each
over, in the considered cases, the dynamical chaotic shape is not recurrent in time. Moreover, in many
behavior of each cell leads to harmonic shapes in of them, spatial symmetries in each frame are evi-
the 3D-CNN configuration. In order to reinforce the dent. The variety of shapes are related to cell
previous remark, let us consider a counterexample. dynamics: however, some of them reflect globalAugust 2, 2005 9:44 01330
The CNN Paradigm 2083
role of self-organization in the previous phenomena,
the problem of discovering recurrent patterns both
in the 3D-CNNs shapes and in modern art is
encountered. It is not the aim of this paper to
investigate on computer based arts or to deal with
the well-known evolutionary art [Bentley & Corne,
2001]. This is to remark on the value of the stagger-
ing complexity we are dealing with. Only few exam-
ples are reported here. Let us consider the Mirò
paint Still Life with Old Shoe; recurrent patterns
are found in the 3D-CNN generated shape when a
Lorenz system is adopted as cell unit. This is shown
in Fig. 20.
The form of the sculpture of Duchamp is recur-
rent in many patterns obtained during the 3D-CNN
evolution. In particular, they appear in the consid-
ered cases when either a grid of Rossler systems
or a grid of Inferior-Olive systems are taken into
account. In Fig. 21 the discussed example is shown.
Fig. 18. When random generators are coupled together into Let us consider now the Robert Delaunay’s
a 3D-CNN, there is no self-organization and regular shapes painting study. He started as an impressionist under
are not formed.
the suggestion of Cezanne and taking into account
the Cubism, started an analytical research on the
form in relationship to the multiplication of light
10000
planes. An example of this study is in Joie de
9000
vivre where he expressed the emergence of light
8000 and nature by using the contrast of colors whose
7000 expressions are sequences of closed curves. Let us
6000
compare this painting with the 3D-CNN-generated
surfaces as shown in Fig. 22. In this case the 3D-
area
5000
CNN cell is the HR dynamical system. In Fig. 23
4000 the Salvator Dalı̀’s Tête Raphaelesque Éclatée is
3000 shown. It reflects the concepts of broken forms,
2000
just introduced in the analytic cubism. In this
artistic expression the knowledge we have of the
1000
subject is a complex sum of all its perceptions.
0
0 50 100 150 200 250 300 350 400 450 500 The recurrent fingerprint patterns could be discov-
t ered in many 3D-CNN shapes like that derived by
Fig. 19. Trend of the shape area generated by a 3D-CNN of
using Izhikevich cells or like that obtained by using
HR neurons. Lorenz cells as shown in Fig. 24.
Each observed dynamics of frames that
included highly organized shapes. Even if each cell
time-related features like the stretching and folding is chaotic, even if each cell is characterized by
phenomenon. the geometrical form of the corresponding attrac-
In order to reinforce the previous remark, the tor with a defined shape, the 3D-CNNs complex
surface area for each shape family is computed patterns generate both an unusual complexity and
at each time. Chaotic time series is obtained. In an astonishing unpredictability. The discovery of
Fig. 19, the time series referring to the shape trend recurrent patterns between 3D-CNN dynamics and
of a 3D-CNN of HR neurons is reported. artist paintings indicates the emergent character-
Following the introduction of the emerg- istics of both underlying organized complexities.
ing shape generation phenomena, the complex Through examples like those, it is emphasized
evolution trend by using 3D-CNN dynamics and the that the dynamical evolution of coupled cells canAugust 2, 2005 9:44 01330
2084 P. Arena et al.
Fig. 20. A form generated by the Lorenz 3D-CNN and Still Life with Old Shoe by Mirò (black and white reproduction of the
original paint [Mirò, 1937]).
Fig. 21. Some shapes recur in different 3D-CNNs like these shapes generated by a 3D-CNN either of IO neurons or Chua’s
circuits. These forms resemble the artistic shape represented in Prière de toucher, by Duchamp [1947].August 2, 2005 9:44 01330
The CNN Paradigm 2085
Fig. 22. Shapes generated by a 3D-CNN of HR neurons and Joie de vivre, oil on canvas, by Delaunay [1930].
irregular shining flashes, like phosphenes appear in
some of the frames. Moreover, in both Figs. 15
and 24 circular waves appear. These forms are
particularly unstable showing very fast changes in
shape, size and time-scale evolution. The various
circles run giving us an impressive image of vortices
and turbulence.
The dynamical combination of shapes give us a
global view perception whose effect is much more
than the sum of single shape contributions. We
are dealing with a complex visual pattern gener-
ator. The information contained in the whole is
many times greater than the sum of the infor-
mation contained in single parts. There exists
a parallelism between the previously considered
frames and those referred in the migraine aura
or in general, in the complex hallucination phe-
nomena. A detailed description of these phenom-
ena is widely reported in literature [Sacks, 1993;
Kluver, 1967]. Moreover the visual effect of hal-
Fig. 23. Tête Raphaelesque Éclatée by Salvador Dalı́ [Delau- lucinations is considered as a complex emergent
nay, 1930]. dynamical phenomenon [Dalhem & Müller, 2003].
In particular, many scientists view hallucination
as the propagation of Reaction–Diffusion waves in
produce very beautiful and rich patterns drawn by neural tissue. Many models have been proposed in
artists. this direction. Cortical organization [Dalhem et al.,
Let us observe now the sequence of frames 2000] in migraine aura is supposed, and mathe-
respectively reported in Figs. 12 and 13. Reg- matical models introduced to explain analytically
ular line segments are evident, unpredictable the phenomena are widely accepted. Ermentrout
ordered fragmentation explodes, a certain type and Cowan [Ermentrout et al., 1979] modeled this
of intermittency frequently appears. Moreover phenomenon by using the relationship betweenAugust 2, 2005 9:44 01330
2086 P. Arena et al.
Fig. 24. Shapes generated by a 3D-CNN of HR neurons and Joie de vivre, oil on canvas, by Delaunay [1930].
inhibitory and excitatory neurons. However, they computer simulation algorithms devoted to explore
introduced a linearized model that, even if the the nervous system, the reported examples reinforce
parameter relationship for deriving the instabil- the suitability to adopt 3D-CNN circuits for emu-
ity condition is established, did not express the lating brain emergent phenomena.
emergent mechanism of the hallucinations due to
the nonlinearity effects.
The myriad pattern formation due to the
migraine aura is a fascinating phenomenon. The
diversity of migraine auras in various forms under-
lines its complexity. Moreover the phenomena
emulated in the reported experiments are gen-
erated by using 3D-CNNs in reaction–diffusion
configuration using as cells integrate-and-fire neu-
ron models. It is not the aim of this experi-
ment to model using a grid of cortical neurons;
moreover, the introduced 3D-CNN strategy allows
us to emulate real self-organizing phenomena in
the visual cortex. The CNN model for hallucina-
tions has been also approached in [Chua, 1998].
The experiments reported in our paper regard a
wider set of hallucination phenomena. They have
been obtained thanks to the 3D-CNN architec-
ture and to the introduction of more complex cell
dynamics with respect to those used in previous
papers.
In view of the appealing field of computa-
tional neuroscience and in particular, in the area of Fig. 25. Il rimorso di Oreste by Giorgio De Chirico.August 2, 2005 9:44 01330
The CNN Paradigm 2087
CNN exactly translates the meaning of com- earthquake events. The cause could be small or
plexity in terms of electronic circuits. The results big, but what is remarkable is the emergent behav-
of coupling many simple nonlinear circuits give ior that arises after the paradigm is changed. In
us a global circuit whose capabilities are much our opinion, what Kuhn remarks for the scientific
more than the predictable performances obtained revolution occurred with the invention of CNN in
by summing the single cell circuit contributions. the field of information technology.
This allows us to create artificially emergent pat- The emergent behavior of a scientific revolution
terns. Moreover, the same emergent behavior has is related to the criticism of some aspects of classical
been discovered in the visual cortex phenomena of or previously accepted paradigms: the positive crit-
migraine aura and in the hallucination. The rich- icism of scientists leads to new paradigms in order
ness of forms and their combination reflect a fur- to overcome problems not solved by previous theo-
ther impressive example of complexity. A further ries. This is the starting point: ideas slowly evolve
remark: the metaphysical art represents another until the emergence occurs and the new paradigm
example of complexity from an aesthetic point of switches to the revolution! In the history of dis-
view. Thinking Complexity is a new point of view, tributed intelligence paradigms a fundamental limit
a new methodology. The vision of complexity theme of perceptron architectures has been highlighted by
is resumed, as an example, in the famous painting Minsky and Papert [1988] that established a percep-
of Giorgio De Chirico Il rimorso di Oreste shown in tron cannot tell whether two labyrinthine patterns
Fig. 25. on the cover are connected or not. The CNN locally
connected networks can solve such a problem! It has
been proved that a locally connected network like
5. Conclusions CNN has the properties to recognize local functions
In this paper, the use of 3D-CNN generalized [Chua, 1998]. Therefore, in order to overcome a
paradigm to generate sequences of emerging shapes problem a new successful paradigm has been intro-
and forms is discussed. A wide range of organized duced. The critical point of a technological revo-
results from the evolution of 3D-CNN dynamical lution has been established and now after sixteen
system has been shown. Different cells dynamics years, the change of the paradigm in the area of
have been taken into account. Locally active cells connectionism leads to a scientific revolution! The
or cells at the edge of chaos have been chosen in revolution has been related to the new CNN pro-
order to assure pattern formation. Complex 3D pat- posed approach. Moreover, the increase of inter-
terns emerging from various experiments have been est in the CNN paradigm worked like an attractor.
critically discussed. Links among circuits, art and Starting from pattern recognition, complex model
neuroscience emerged thanks to the universality of behaviors, vision, neuromorphic models, robotics,
CNN formalization. neuroscience problems have been faced in terms
In his impressive book The Structure of Scien- of CNN paradigm. The new formalization allowed
tific Revolutions, Thomas Kuhn founded his theory the conception of new advanced equipments, and
on the concept of paradigm [Kuhn, 1962]. With this a revolution in the field of information technol-
term, Kuhn indicates the “scientific conquests uni- ogy is ongoing. The dynamical richness of CNN
versally accepted which, for a period, give a model based architecture allowed to reformulate classical
for problems and solutions for people that made problems in a new formalization. The last effort
research in a particular field”. When the paradigm proposed in this paper is to investigate on the
changes, a critical breakpoint occurs in science and universality of the 3D-CNN in order to discover
therefore bifurcation conditions occur under which emergent 3D shapes. Self-organization that is the
a new theory replaces an old one. In any case, each core of the emergent characterized CNN system
scientific relationship is a set of ideas that lead us allows shorter distances between technological sci-
to a small or big settling when they are replaced ence, art and neuroscience.
into an old scientific paradigm.
Kuhn observed that the scientific revolutions
could be small or big, but both have the same Acknowledgments
structure, the same characteristics when they occur. This work was supported by the Italian “Minis-
In fact, what happens is like the same emergent tero dell’Istruzione, dell’Università e della Ricerca”
phenomenon that occurs in the sand pile or in (MIUR) under the Firb project RBNE01CW3M.August 2, 2005 9:44 01330
2088 P. Arena et al.
References Giles, J. [2004] “Neuroscience: Change of mind,” Nature
430, p. 14.
Arena, P., Baglio, S., Fortuna, L. & Manganaro, G.
Izhikevich, E. M. [2003] “Simple model of spiking neu-
[1995] “Chua’s circuit can be generated by CNN
rons,” IEEE Trans. Neural Networks 14, 1569–1572.
cells,” IEEE Trans. Circuits Syst.-I 42, 123–125.
Izhikevich, E. M. [2004] “Which model to use for cortical
Arena, P., Baglio, S., Fortuna, L. & Manganaro, G.
spiking neurons?” IEEE Trans. Neural Networks 15,
[1996] “How state controlled CNNs generate the
1063–1070.
dynamics of the Colpitts-like oscillator,” IEEE Trans.
Klüver, H. [1967] Mescal and the Mecanism of Halluci-
Circuits Syst.-I 43, 602–605.
nations (University Press of Chicago, Chicago).
Arnheim, R. [1974] Art and Visual Perception: A Psy-
Kuhn, T. [1962] The Structure of Scientific Revolutions
chology of the Creative Eye, 2nd Revised Edition
(University of Chicago Press).
(University of California Press).
Lynton, R. [1994] The Story of Modern Art (Phaidon
Bentley, P. J. & Corne, D. W. [2001] Creative Evolution-
Press).
ary Systems (Morgan Kaufmann).
Minsky, M. & Papert, S. A. [1988] Perceptrons: An Intro-
Bogousslavsky, J. [2003] “The neurology of art — The
duction to Computational Geometry, Expanded edi-
example of Giorgio de Chirico,” Eur. Neurol. 50, 189–
tion (MIT Press, Cambridge, MA).
190.
Mirò, J. [1937] Still Life with Old Shoe, The Museum of
Chua, L. O. & Yang, L. [1988a] “Cellular neural net-
Modern Arts, NY, USA.
works: Theory,” IEEE Trans. Circuits Syst.-I 35,
Morales, C. [1991] “Profile: Eladio Dieste,” in MIMAR
1257–1272.
41: Architecture in Development (Concept Media
Chua, L. O. & Yang, L. [1988b] “Cellular neural net-
Ltd., London).
works: Applications,” IEEE Trans. Circuits Syst.-I
Nagumo, J., Arimoto, S. & Yoshizawa, S. [1960] “An
35, 1273–1290.
active pulse transmission line simulating 1214-nerve
Chua, L. O. & Roska, T. [1993] “The CNN paradigm,”
axons,” Proc. IRL 50, 2061–2070.
IEEE Trans. Circuits Syst.-I 40, 147–156.
Orsucci, F. [2002] Changing Mind, Transitions in Natu-
Chua, L. O. [1998] CNN: A Paradigm for Complexity,
ral and Artificial Environments (World Scientific).
World Scientific Series on Nonlinear Science, Series A,
Otto, F. [1982] “Frei Otto’s new work,” in MIMAR 4:
Vol. 31.
Architecture in Development (Concept Media Ltd.,
Ciofini, M., Labate, A. & Meucci, R. [1999] “Stabiliza-
Singapore).
tion of unstable fixed points in the dynamics of a laser
Pisarchik, A. N., Meucci, R. & Arecchi, F. T. [2001]
with feedback,” Phys. Rev. E60, 398–402.
“Theoretical and experimental study of discrete
Dahlem, M. A., Engelmann, R., Löwel, S. & Muller, S. C.
behavior of Shilnikov chaos in a CO2 laser,” Europ.
[2000] “Does the migraine aura reflect cortical orga-
Phys. J D13, 385–391.
nization?” Europ. J. Neurosci. 12, 767–770.
Podoll, K., Robinson, D. & Nicola, U. [2001] “The
Dalhem, M. A. & Müller, S. C. [2003] “Migraine aura
migraine of Giorgio De Chirico. 1. History of illness,”
dynamics after reverse retinotopic mapping of weak
Neurol. Psych. Brain. Res. 9, 139–156.
excitation waves in the primary visual cortex,” Biol.
Rose, R. M. & Hindmarsh, J. L. [1989] “The assembly
Cybern. 88, 419–424.
of ionic currents in a thalamic neuron. I. The three-
Dalı́, S. [1951] Tête Raphaelesque Eclatée, Scottish
dimensional model,” Proc. R. Soc. Lond. B237, 267–
National Gallery of Modern Art, Edinburgh.
288.
Delaunay, R. [1930] Joie de Vivre, Pompidou Center,
Sacks, O. W. [1992] Migraine (Vintage Books).
Paris, France.
Strogatz, S. H. [2000] Nonlinear Dynamics and Chaos
Duchamp, M. [1947] Prière de Toucher, Modern Art
(Perseus Publishing).
Museum of the City of Paris, Paris, France.
Wilkinson, M. & Robinson, D. [1985] “Migraine art,”
Ermentrout, G. B. & Cowan, J. D. [1979] “A mathe-
Cephalalgia 5, 151–157.
matical theory of visual hallucination patterns,” Biol.
Yang, T. & Chua, L. O. [2001] “Testing for local activity
Cybern. 34, 137–150.
and edge of chaos,” Int. J. Bifurcation and Chaos 11,
FitzHugh, R. [1961] “Impulses and physiological states
1495–1591.
in theoretical models of nerve membranes,” 1182-
Zeki, S. [1999] Inner Vision: An Exploration of Art and
Biophys. J. 1, 445–466.
the Brain (Oxford University Press).
Fortuna, L., Rizzo, A. & Xibilia, M. G. [2003] “Modeling
complex dynamics via extended PWL-based CNNs,”
Int. J. Bifurcation and Chaos 13, 3273–3286. Appendix A
Giaquinta, A., Argentina, M. & Velarde, M. G. [2000] Characteristics of Eˆ3
“A simple generalized excitability model mimicking
salient features of neuron dynamics,” J. Stat. Phys. In this section, the main features of Eˆ3 are illus-
101, 665–678. trated. This software has been designed to simulateAugust 2, 2005 9:44 01330
The CNN Paradigm 2089
complex systems made of interacting dynamic non- visualization, definition of cell dynamics) and
linear units as described in Sec. 2. Therefore Eˆ3 Java (communication between processors, man-
has been designed to include three properties: the agement of data distribution) and is based on
dynamics of each simple unit is arbitrary, the cells open-source components.
of the complex system are either identical or differ- Parallel¯ computing may be realized by using
ent, connections are arbitrary. coarse grained or fine grained architectures. A
Eˆ3 provides the user with the possibility coarse grained architecture refers to the case in
of implementing in a very simple way arbitrary which the computation is distributed to a small
dynamics for each cell of complex systems. Follow- number of high-capability processors, while the
ing this approach, each cell of the complex CNN architecture is fine grained when a high number of
consists of an arbitrary n-order system, and may simple processors is used. In this sense a CNN is
be defined by the user by writing its equations. the most significant example of fine grained archi-
This differs from standard CNN in which each cell tecture. The architecture of Eˆ3 is based on a small
dynamics is defined by templates, which therefore number of high-capability processors.
contain both cell dynamics and connections among The strategy adopted to implement parallel
cells. computing is the so-called domain decomposition,
Moreover, the cells of the 3D-CNN may be dif- in which data are distributed among the processors
ferent from each other. The more general case is executing the same operations on different portions
that the equations defining each of the cells consti- of the data. In fact, the problem of emulating a sys-
tuting the complex systems are different from cell tem made of many units can be simply decomposed
to cell. This is, for example, the case in which one into several domains made of subparts of the whole
would model fire propagation in two different adja- set of the cells. A processor plays the role of master
cent substratums. Moreover, cells may be nearly and collects all the results coming from the elabo-
identical. In this case, cells of the complex systems ration by the other processors. Moreover, through
may differ only for the value assumed by their char- the so-called message passing each processor may
acterizing parameters. This case is different from obtain data processed by other units of the parallel
the previous one, in fact, in this case one does not architecture.
need to write new equations, but the possibility to A very efficient way to implement message pass-
have space-variant parameters should be included. ing is to create a cluster of workstations in a LAN
Finally, the connections among the units of the network. The whole software therefore consists of
CNN and, in general, of a complex system may be of two main modules, called server and client (Fig. 26).
several types. Several examples of complex systems The server runs in each processor of the network,
made of locally interacting units have been studied while the client runs on the master PC, coordinating
in literature; all-to-all coupling is also very com- the data coming from the different processors. The
mon in modeling complex systems. While random complex system to be emulated has to be defined
networks efficiently model phenomena like stock in the client. The first operation executed by the
markets, and small world connections account for client is to create the structure and to assign to
models as spread of diseases, modelling the struc- each client a portion of the cells to be simulated.
ture of the world wide web requires a dynamically Then the integration routine is performed.
changing network (scale-free network). A general To allow the definition of the cell dynamics by
simulator for complex systems should provide the the user it has been chosen to implement a rou-
possibility of implementing all these structures in tine — called compile — able to create a .dll file
an easy way and at the same time should allow the
user the possibility of reconnecting arbitrary cells
of the system.
Another important characteristic of Eˆ3 is the
use of parallel computing. A general structure has
been designed for Eˆ3 . The simulator can be run
either on a single machine or on a network of
personal computers. This second case implements
parallel computing. The software has been written
in C (routines for numerical integration, output Fig. 26. Main modules of the software architecture of Eˆ3 .You can also read
