In an attempt to build more sophisticated neural networks and other Information Technology (I.T.) products, the industry constantly turns to the world of Biology for inspiration. The most advanced computers in the World today, are of course humans. This paper looks at Self Organisation in the Human Nervous System and aims to highlight the means by which the understanding gained, from the study of this issue, can influence and inspire the design of Neural Networks and I.T. products and services.

1. Self Organisation
Inspiring Neural Network & IT Design
www.oliviamoran.me

2. About The Authors
Olivia Moran is a training specialist who Submitted For Cognitive Computing,
specialises in E-Learning instructional Msc in Computing, University of
design and is a certified Moodle expert. Ulster 2006
She has been working as a trainer and
course developer for 3 years developing Authors included:
and delivery training courses for traditional
classroom, blended learning and E-learning.  Olivia Moran
 Eric Nichols
Self Organisation: Inspiring Neural Network  Barry Feehily
& IT Design was written as part of a group  Lisa Murphy
collaboration.
www.oliviamoran.me

3. Self Organisation: Inspiring Neural Networks & IT Design
1. ABSTRACT 2. INTRODUCTION
In an attempt to build more sophisticated neural The term self-organisation is used to describe the
networks and other Information Technology (I.T.) process by which “Internal structures can evolve
products, the industry constantly turns to the world of without the intervention of an external designer or the
Biology for inspiration. The most advanced presence of some centralised form of internal control.
computers in the World today, are of course humans. If the capacities of the system satisfy a number of
It is therefore no wonder that engineers and constraints, it can develop a distributed form of
computer scientists invest such a large amount of internal structure through a process of self-
their time examining theses biological machines and organisation” Cilliers (1998). In basic terms self-
the way in which they operate. I.T. products are often organisation is a process that involves the
constructed based on the same principles or concepts organisation of group behaviour to achieve global
on which the human body is built. These concepts order. This process occurs through interactions
relate to areas such as sensory perception and among the group and not through external influences.
processing, the motor system and social cognition etc, According to Cilliers (1998) “This process is such that
the list is endless. For the purpose of this report, structure is neither a passive reflection of the outside,
however, only one issue will be explored in depth, nor a result of active, pre-programmed internal
self-organisation. factors, but the result of a complex interaction
between the environment, the present state of the
This report aims to highlight the means by which the system and the history of the system”.
understanding gained, from the study of this issue,
can influence and inspire the design of I.T. products The concept of self-organisation is easily illustrated
and services. It will examine the role of sleep and its using an example from nature. Examples include the
effects on self-organisation. Subsequently, the reaction of hair dye to our hair and the type of activity
development of the Nervous System (N.S.) and the that occurs as well as the growth of plants and
importance of self-organisation to the development animals and the creation of a sculpture by an artist.
process will be explored in depth. This document will One of the more widely used examples is the hare and
briefly consider how connections are made between the lynx. A study was carried out and recorded by the
neurons and furthermore how these can be rewired. Hudson Bay Trading Company in Canada between
Self-organisation occurs at a number of levels, which 1849 and 1930. This recorded and examined specific
will be highlighted. A comparison will be made statistics relating to the populations of hares and
between the N.S. of invertebrates and vertebrates in lynxes. In this example, the lynx is the predator of the
an attempt to determine the effect, if any, that the hare. It was concluded from this study that a
sizes of these systems exert on the self-organisation decrease in the number of prey, would cause a
process. It will be illustrated how self-organisation corresponding decrease in the number of predators.
can be computationally modeled. Finally, this This was due to the fact that a reduction in prey
document will give some thought to future work in resulted in limited food sources and so there was not
this field of research. enough food to sustain current predator numbers.
After a period of time, the numbers of prey grew
KEYWORDS: Cognitive Computing, Self-Organisation, because the amount of predators was low. However,
Neural Networks, The Nervous System, Self- this replenished food stocks for the predators and so
Organising Maps. there numbers began to grow yet again. This process
begins over again and continues in a cyclical manner.
This behaviour is seen emerging from the interaction
between the lynxes and hares.

4. Self Organisation: Inspiring Neural Networks & IT Design
This self-organisation process also takes place in our deprivation is viewed as a lack of the necessary
bodies. One can easily forget that our bodies are amount of sleep that your body requires for healthy
some of the most complex systems around. functioning.
Consequently it is only obvious that this is a good
place in which to study the concept and gain a better The occurrence of sleep deprivation throughout
understanding of it. This document aims to explore modern day society is incredibly higher than that
the concept of self-organisation in depth. Firstly the found four or five decades ago. This is partly due to
role of sleep is considered and the part that it plays in our hectic lifestyles, jobs and of course electrical
the self-organisation process. Self-organisation is lighting. People are staying awake for longer lengths
extremely important in the development of the of time. Consequently, there has been a substantial
nervous system. It is also crucial to understanding decrease in the average amount of sleep each person
how connections between neurons are made and gets. A person can be deprived of sleep by their own
rewired. This development process is explored at mind and body. Sleep is extremely important and is
length. The occurrence of self-organisation at needed for regeneration of certain parts of the body
different levels is considered briefly focusing on the in particular the brain so that it can function properly.
single and networked cell levels. A comparison is
made of the N.S.’s of invertebrates and vertebrates When the body is asleep the brain goes through a
and thought is given to how the size of these systems process that consists of four different stages called
may impact on self-organisation. It is illustrated how the R.E.M. (Rapid Eye Movement) sleep cycles. R.E.M.
self-organisation can be computationally modelled. sleep is the desirable sleep state characterised by
Future work in this area is also addressed. rapid movements of the eyes. At certain points of the
sleep process, the brain is active in different ways.
3. SLEEP AND ITS These can be identified using
Electroencephalogram (E.E.G.) reader. In the first
an
ROLE IN SELF- stages of sleep, the body starts to relax and the heart
rate begins to slow. At this point people often feel as
ORGANISATION IN
if they are falling or feel weightless.
THE BRAIN
During the second stage of sleep it becomes evident
that the brain is not acting in the same way i.e.
emitting the same brain waves, as when the body was
In the past people taught of sleep as a dynamic and awake. This stage is where deep restful sleep occurs
dormant activity that was part of our every day lives. and the body reenergises itself. The body must go
Nowadays people are more aware that sleep can through a sufficient amount of R.E.M. cycles or else
actually affect our daily functioning along with our the body will be unable to reenergise itself.
physical and mental health. A number of activities Consequently self-deprivation would result. The
occur within the brain in order to prepare us for sleep. effects of this self-deprivation may include difficulty
Firstly, within the brain, neurotransmitters, nerve- concentrating, being in a bad mood, reduced energy
signalling chemicals, act on different neurons and and a greater risk of being in or causing an accident,
control whether we are asleep or awake. These including fall-asleep crashes. Stage three and four are
neurons are located in the brainstem, the part of the much deeper sleep states however, four is more
N.S. that connects the brain with the spinal cord. intense than three. These stages are often referred to
Here they produce neurotransmitters that keep as slow-wave sleep or delta sleep. The reason why is
different parts of the brain active when a person is evident particularly in stage four where the E.E.G.
awake. Other neurons located at the base of the reader records slow waves of high amplitude,
brain, begin signalling the relevant neurons with the demonstrating a pattern of deep sleep and rhythmic
body gradually falling into a sleep state. If the later continuity.
does not occur sleep deprivation results. Self-

5. Self Organisation: Inspiring Neural Networks & IT Design
Research shows that sleep-deprivation has noticeable Scientists are now realising that sleep deprivation can
negative effects on things such as alertness and affect the whole body not only the brain. A study that
cognitive performance. According to Thomas et al was carried out by Dr. Eve Van Cauter from the
(2000) this suggests a decrease in brain activity and University of Chicago showed that failing to get the
function primarily in the thalamus, a subcortical right amount of sleep could affect the chemical
structure involved in alertness and attention, as well balances in the body as well. The study looked at a
as the prefrontal cortex, a region subserving alertness, male after four hours sleep for a total of six nights.
attention, and higher-order cognitive processes. It is Results from blood tests showed strikingly similar
seen that after extended periods of wakefulness or results to those expected from a person with
reduced sleep, neurons can begin to malfunction. diabetes. The male’s ability to process blood sugar
This change in the neurons can have a visible affect on was reduced by a total of thirty percent, this in turn
a person’s behaviour. caused a drop in insulin levels. It was also reported
that the male had specific levels of memory
Organs such as muscles are able to regenerate impairment.
themselves when a person is not asleep so long as
they are resting. In this circumstance the cerebral Scientists strive to come up with an answer to the
cortex within the brain is not able to rest but rather question ‘how much sleep does an average person
remains alert in a state of ‘quiet readiness’. This need in order for their brain to reenergise itself and
suggests that while some stages of sleep are a ideally self-organise’. This is a difficult one to answer.
necessity for the regeneration of neurons others are It is different for everyone and is influenced by factors
suited to creating new memories and the formation of such as age and health. It is generally accepted that
synaptic connections. babies need an average of nineteen hours a day while
teenagers need a total of nine. Adults function
A study was carried out in an attempt to highlight the normally with approximately seven to eight hours
negative effects of sleep deprivation. Seventeen however, certain individuals can limit themselves to
males over an eighty-five hour period of sleep- five hours while others may require ten hours of
deprivation were examined. The subjects were sleep.
observed four times every twenty-four hours. During
this period they were asked to complete a series of It is clear that sleep is absolutely necessary and that
addition and subtraction tasks. Polysomnographic without it our bodies would be unable to survive for
examinations confirmed that the subjects were long. One study involving rats demonstrates this
awake. After twenty-four hours it was reported that point effectively. Rats are seen to live for two to
there was a significant decrease in global C.M.R.glu, three years but because of sleep deprivation and not
and dramatic falls in absolute regional C.M.R.glu in going through the R.E.M. cycle, the rats in the
several cortical and subcortical structures. The main experiment only lived for a total of five weeks. The
changes occurred primarily in the thalamus and rats developed low blood pressure, sores on their tails
prefrontal and posterior parietal cortices located near and paws as well as an impaired immune system.
the front of the brain (See Appendix 1). Researchers in recent years have become a lot more
interested in this area of study. They know that
From these experiments it was concluded that short- gaining a better understanding of sleep and all that it
term sleep deprivation produces global decreases in encompasses will in turn result in a greater insight
brain activity with larger reductions in activity in the and knowledge of the role that sleep plays in the self-
distributed cortico-thalamic network mediating organisation process.
attention and higher-order cognitive processes. It
was also complementary to studies demonstrating
deactivation of these cortical regions during N.R.E.M.
and R.E.M. sleep.

6. Self Organisation: Inspiring Neural Networks & IT Design
4. THE NERVOUS After a short period a crease or fold appears in this
plate, it begins to grow and a neural groove appears.
SYSTEM Folding action continues until the creases meet and
fuse together. This fusion results in the neural tube
that eventually develops into the nervous system. If
The human body is made up of trillions of cells that development goes according to plan the neural tube
interact and work together to achieve certain closes completely. Failure to close could result in
outcomes. Numerous amounts of these cells join abnormalities such as Spina Bifida. Vesicles grow
forces in order to create complex systems such as the from the front end of the tube. These will eventually
N.S. This system is found in both animals and people become a part of the C.N.S. During the entire process
and is crucial to their survival as it facilitates all the cells in the N.S. comply with a strict set of rules.
movement, therefore enabling them to respond to These rules determine exactly where each cell will
changes in their environment and adapt. The N.S. eventually end up and what purpose it will serve.
consists of two main parts, the first includes the spinal
cord and the brain and is known collectively as the Next, cell differentiation and division occurs. Mitosis,
Central Nervous System (C.N.S.). The second part is the process by which cells divide and thus multiple
the Peripheral Nervous System (P.N.S.) and it takes in takes place at the inner part of the wall of the neural
all the bodies’ different nerves. tube. Firstly, the cells move away from the wall to
develop further and then return to undergo mitosis.
4.1 THE DEVELOPMENT This process of division results in a huge amount of
new cells being formed and consequently the
OF THE NERVOUS SYSTEM thickening of the neural tube wall. The vesicles also
increase in size. The new cells will eventually develop
into neurons or glial cells.
Self-organisation is undertaken at different stages of
the N.S. development process. According to Willshaw
(2007) it is self-organisation that is responsible for 4.1.2 Cell Migration
“Generating nerve cells of the right type, in the right
numbers, in the right places and with the right The next major step in the development of the N.S. is
connections”. Such a procedure is highly complex and cell migration. Cell migration refers to the movement
involves “cell division, cell migration, cell death and of cells away from where they first developed, to
the formation and withdrawal of synapses”. where they are needed. This process is an extremely
complex one, requiring a high level of organisation.
All cells must end up in the exact desired position. “In
4.1.1 Cell Division the developing brain, for example, primitive neuronal
cells migrate out of the neural tube and take up
In humans after the primitive cell layers are formed, residence in distinct layers, where they send
the inner cells break into a layer of ectoderm and projections (axons and dendrites) through the layers
endoderm. A new layer called the mesoderm grows of developing cells to their final targets with which
between these two layers which all then begin to they form specific connections, called synapses that
work together to produce the notochord. The allow complex functions such as learning and
notochord is a cylindrical shaped structure that is memory” Cell Migration Consortium (2007).
responsible for organising the ectoderm layer. A
number of steps are followed in the achievement of
this task. Chemicals are released from the notochord;
these stimulate the ectoderm so that it begins to
divide. This division leads to the creation of the
neural plate (See Appendix 2).

7. Self Organisation: Inspiring Neural Networks & IT Design
4.1.3 Cell Death (See Appendix 3). The main task of any synapse is the
transformation of electrical impulses into chemical
Cell death is a normal occurrence as well as a signals so that they can be transported. The
“Fundamental and essential process in development” beginning of this conversion process is sparked by
Bähr (2006). It is necessary that some cells be what’s known as an action potential, which is in
sacrificed for the success of the entire process. There essence a nerve impulse. “The end part of an axon
are many theories attempting to shed light on the splits into a fine arborisation. Each branch of it
reason behind cell destruction. Such theories are terminates in a small end bulb almost touching the
highlighted by Willshaw (2007) and include “Failure of dendrites of neighbouring neurons” Zurada (1992).
neurons to find their targets, failure to make the The nerve impulse travels down to the bottom of the
correct connections, the elimination of entire axon where it stimulates the synaptic vesicles
structures that may act as transient scaffolds, removal resulting in the release of neurotransmitter. This
of transient branches of the tree of lineage and lack of flows into the synaptic cleft filling it up. The chemical
adequate innervation”. Such explanations fail to then makes its way towards the dendrites of the cell it
address all the issues relating to cell death. is trying to communicate with. As a result of this,
parts of the membrane open up. Through these
Another hypothesis known as ‘The Neurotrophic openings ions can flow in and out.
Hypothesis’ was constructed and is currently the most
logical way in which to explain why and how cell This flow of ions results in a change in voltage that is
destruction occurs. “Its principal tenet is that the known as a postsynaptic potential. This potential can
survival of developing neurons depends on the supply be excitatory or inhibitory. If an excitatory potential is
of a neurotrophic factor that is synthesized in limiting created in the case of depolarising currents, this
amounts in their target fields” Davies (1996). If there usually leads to the production of a second action
is not enough neurotrophic factor present, the extra potential. However, inhibitory potentials as with
neurons produced will not be able to survive and will hyperpolarising currents, inhibits any further action
simply die. potential. It is important to note that sometimes
impulses will not necessarily travel to another neuron.
“The synapses thus help regulate and route the
4.2 Neurons And Their constant flow of nerve impulses throughout the N.S.”
Connections The World Book Encyclopedia (1991).
Once the neurons find their desired position, 5. SELF-
ORGANISATION AT
connections have to be established. Such connections
accommodate communication between the neurons.
Each neuron is made up of an axon and dendrites that
are crucial to the entire communication process. For
example, when cell A wishes to communicate with cell
DIFFERENT LEVELS
B, the following sequence of events occur; Cell A using Self-organisation occurs at both a networked cell and
the axon transmits a message. Cell B is able to receive a single cell level. The networked cell level is
this message via the dendrites that act like a receptor concerned with the construction of maps that detail
antennae. The meeting point of the two cells is the connections that exist between the nerve cells.
known as the synapse (See Appendix 3). On the other hand, at the single cell level, focus is on
Neurotransmission is also dependant on chemicals the elimination of superinnervation from developing
that act as a neurotransmitter and the use of electric muscle.
signals to get the message across to the other cell.
A synapse is made up of three main parts, the axon
terminal, the synaptic cleft and the dendrite spine

8. Self Organisation: Inspiring Neural Networks & IT Design
5.1 NETWORKED CELL resemble those maps found namely in the human’s
vertebrate visual system. “Topographic maps vary
LEVEL: SELF-ORGANISING considerably from one person to another”. They serve
their purpose in that “Projections from one area of the
MAPS brain to another often preserve neighbour
relationships so that an area smoothly and
Self-organising maps are a good example of how self- continuously maps the area which project to it”
organisation occurs at a networked level. Such maps Bamford et al (2006).
are highly ordered and consist of multiple amounts of Topographic maps have two main distinguishable
nerve cells. This conclusion is according to Willshaw characteristics. Take for example, the organism
(2007), a consequence of both electrophysiological Xenopus that is made up of recoding positions. These
and anatomical experiments. The electrophysiological according to Willshaw (2007) “Can be distinguished,
experiments are concerned with the identification of all arranged in topographic order. The other
a receptive field, “An area in which stimulation leads important attribute of such maps is that they always
to response of a particular sensory neuron” Levine & have a specific orientation. All retinotectal maps are
Shefner (1991). On the other hand the anatomical arranged so that temporal retina projects to rostral
expirements focus on the “Mapping between two tectum and dorsal retina to medial tectum”.
points in different structures … using axonal tracers.
Tracers placed at one point in one structure typically According to Bamford et al (2006) these maps can
label a small, circumscribed area in the target, the “Form in the absence of any electrical (spiking) activity
spatial layout of points of administration being and mechanisms proposed include varied repulsion to
reflected in the layout of points to which the tracers chemicals with graded expression across the target
go in the target” Willshaw (2007). areas. However maps can be refined in the presence
of electrical activity (the spread of connection fields
5.1.1 Neural Maps reduced). It can be demonstrated that a combination
of spatially correlated input, recurrent connections
between target neurons and Hebbian learning can
Seiffert & Lakhmi (2001) define neural maps as maps
produce ordered projections”.
which “Project data from some possibly high-
dimensional input space on to a position in some
From examination of these maps it has been
output space”. These neural maps are made up of
concluded that “Connections cannot be made by
neuronal groups all of which are connected. “Two
means of a simple set of instructions specifying which
functionally different neural maps connected by re-
cell connects to which other cell, more likely, the
entry form a classification couple. Each map
populations of cells self-organise their connections so
independently receives signals from other brain maps
as to ensure the correct overall pattern” Willshaw
or from the world. Functions and activities in one map
(2007). A better understanding of these connections
are connected and correlated with those in another
that form under a process of self-organisation will
map. For example an input could be vision and the
undoubtedly lead to the creation of more biologically
other from touch” Clancey et al (1994). In basic terms
plausible neural networks.
neural maps are simply a projection of one two
dimensional area onto another.
5.2 SINGLE CELL LEVEL –
5.1.2 Topographic Maps ELIMINATION OF
The neural network is capable of being trained SUPERINNERVATION
through unsupervised learning. In such circumstances
the neural network can produce maps that still retain FROM DEVELOPING
their topological features (See Appendix 5). These
maps find their inspiration from humans and
MUSCLE

9. Self Organisation: Inspiring Neural Networks & IT Design
Many models exist which put forward different Invertebrates are animals sharing one common
arguments that claim to offer a plausible reason for characteristic and that is they do not have a backbone
the elimination of superinnervation during the or spine. On the other hand vertebrates are all those
development of muscles. The most widely accepted other animals who possess this spinal column
of these is the ‘Dual Constraint Model’ Bennett & structure as part of their anatomy as in the case of
Robinson (1989). This model which “Combines fish, birds, reptiles and of course humans. Freeman
competition for a pre-synaptic resource with (2005) points out that the “Architectures of the C.N.S.
competition for a post-synaptic resource, has been of intelligent invertebrate animals differ markedly
shown to be superior to others with only one type of from those in vertebrate animals”. The N.S. of the
competition” Rasmussen & Willshaw (1993). When invertebrates are fairly simple in construction in
muscle fibres are being developed, they are contrast to the vertebrates. Take for example, an
“Superinnervated and this pattern is transformed into invertebrate such as the bee or the octopus that has
one of single innervation after a few weeks” Willshaw “Parallel chains of neurons resembling a ladder
(2007). The length of time needed to complete this located ventral to the digestive system, from which
process as well as the total amount of elimination that and to which the axons of motor and sensory nerves
takes place, differs depending on the section of the extend” Freeman (2005). They may have some type
N.S. where it occurs. of eyes and mouth in which case are “Serviced by
large collections of neurons forming the dorsal
This theory basically operates on the tenant that cerebrum. Axons form bi-directional connections with
motor neurons have a particular capacity “For the ventral nerve cords around the gut, so that the
maintaining the structure and activity of its terminals, esophagus runs through the brain. Perhaps this is why
which is shared out among them” Willshaw (1981). all higher invertebrates are restricted to a liquid diet,
Each terminal has a survival strength, however this is lest they rupture their brains by swallowing solid food”
dynamic and is constantly regulated and fine-tuned. Freeman (2005).
Those terminals with a high level of strength are given
precedent over the feeble terminals. Consequently, The architecture of the vertebrates is such that it does
the survival strength of the endplate of the stronger not have to deal with a limitation of this nature. “The
terminals is increased to the detriment of the weaker C.N.S. forms by invagination of the dorsal surface and
ones. creates the neural tube. The posterior part forms the
spinal cord while the most anterior part forms the
Such a theory would lead one to conclude that the brain” Freeman (2005). Freeman (2005) also argues
muscle fibres and the construction of their that despite all the differences that might exist with
connections as well as the pattern that they follow, the architecture of the C.N.S. of both animal types
are the result of a process of self-organisation under they do however, “Share the ladder-like architecture
highly competitive conditions. The patterns are of invertebrates”.
therefore not formed by instructions specified in the Numerous studies on invertebrates have been
genome. completed concerning the role of organisation.
Research evidence suggests that the Drosophila,
6. THE NERVOUS commonly known as the fruit fly displays very “Precise
and inflexible organisation” Willshaw (2007). This
SYSTEM OF
would lead one to conclude that self-organisation in
this small and simple N.S. of the fruit fly, is in a
VERTEBRATES AND
position whereby the “Genome can afford to specify
precisely all the parameters values needed which have
INVERTEBRATES
a smaller number of neurons” Willshaw (2007). Both
small and large N.S.’s seem to display the ability to
self-organise.

10. Self Organisation: Inspiring Neural Networks & IT Design
7. Computationally As every input node is connected to every output
node, adding nodes to the system causes the network
Modelling Self- to grow exponentially. The amount of computation
required to calculate large systems can quickly
Organisation
become too data-intensive for equations to be solved
within a reasonable time-scale. As an understanding
of biological systems becomes more detailed, the
Kohonen created a computational model for self- algorithms for describing such systems require more
organisation in 1981. Bruske & Sommer (1995) states computation. At its most detailed, if quantum
“Kohonen’s self-organizing feature maps, besides back confinement can be proven to exist in the interactions
propagation networks, are now the most popular and within and between neurons, then quantum
successful types of artificial neural networks”. mechanics “Represents the ultimate tool to the
Kohonen’s model, maps every input node to every modelling of bio molecular systems” Chung (2007).
output node (See Appendix 6, Figure 1). The first step
in Kohonen’s algorithm is the initialisation the However, Chung writes that a quantum approach is
synaptic weights, which can be set to random values. “Formidable and is an extremely time consuming
The next step involves finding the winning neuron by process, even with some simplifying assumptions, its
calculating the Euclidean distance between input and applications are limited to very small systems at
output neurons. Kohonen (1997) discovered that one present”. As the speed of computational machines
could find the ‘winning’ neuron by using the following increases and we are “Equipped with powerful
formulae: computing techniques and high-performance sensors
Best matching node = mini {||x-mi||} = and actuators, we want to solve much more complex
(highly non-linear and high-dimensional) problems”
 x  mij 
n
2 Kecman (2001). This relates significantly to self-
j
j 1 organising systems, as the growth of these systems is
In this formula, x represents the input and m exponential.
represents the output map. The neuron pair with the
smallest Euclidean distance, the closest output node Kohonen’s (2001) argues that self-organising maps are
represents the winning neuron on the output map useful for classification. On the other hand, in
(See Appendix 6, Figure 2). isolation they are not as good as other methods.
Lisboa (1992) compared Kohonen’s network to other
The value of the closest output node is adjusted so classifiers (See Appendix 7). Kohonen’s self-organising
that a smaller Euclidean distance results. The nodes in map had the worst performance of the six that were
the closest output node’s neighbourhood are also compared using handwriting digit recognition as a test
updated (See Appendix 6, Figure 3). The weight of the case.
winning neuron, as well as the weights of all the
neighbouring neurons, is adjusted with the formulae Kohonen’s network was not the first computational
(Kohonen 1997). self-organising map. According to Grossberg (1994), in
1976 Grossberg wrote a mathematical model of a self-
mi(t+1) = mi(t) + hci(t)[x(t) – mi(t)] organising feature map, where “Neurobiological
modelling rules were articulated and restated in the
In this formula, t represents an integer representing a familiar SOFM formalism as an algebraic winner-take-
time interval and hci(t) represents a ‘neighbourhood all dot product rule, and a self-normalizing synaptic
function’. The process then returns to the start (See weight change rule whose weights change only if they
Appendix 6, Figure 1) with the updated weights. The are in the neighbourhood of the winner.” Five years
loop continues until the network sufficiently matches later, Kohonen (2001) sought to “Generalise and at
the target system. the same time ultimately simplify his (Grossberg’s)
system description”. With this simplification, further

11. Self Organisation: Inspiring Neural Networks & IT Design
work can be achieved on self-organisation using above to greatly enhance the computational
Kohonen’s network as a baseline. modelling of self-organising maps in Matlab. As new
self-organising algorithms are created, further Matlab
Examples of this can be found in both hardware and files can be written to continue computational
software implementations of self-organising maps. modelling of biological systems with textual and
The construction of self-organising maps can be a very graphical outputs.
time-consuming process because every input must be
mapped to every output node. Martinez et al (2002)
have found that using systolic arrays with Kohonen’s 8. FUTURE WORK
network greatly reduces computational time, as each
node can be computed in parallel on different Artificial neural networks are modelled on our
processors. perception of the way the brain processes
information. As technology develops, neurologists
Linaker and Niklasson (2000) used a neighbourhood will be able to find a more definitive understanding to
function of 0 (no neighbours) and an altered Kohonen self-organisation in the brain while biophysicists find a
network, called a Resource Allocating Vector better representation of our brain at a molecular and
Quantizer (RAVQ), for a robot to successfully learn its atomic level. These findings can then be used to
environment. The main difference between the self- develop better theories and technologies for artificial
organising map that Kohonen authored and the RAVQ neural networks.
is that the latter’s output map is dynamic. The RAVQ
more closely mimics biological systems in that output Intelligent systems’ current (third) generation models
nodes can be created and mapped. While Kohonen’s take past work on neural networks and “Raises the
original network is not great for classification, as level of biological realism by using individual spikes”
shown in Table 1, enhancements such as the RAVQ Vreeken (2002). A current active area of research in
can make self-organisation more realistic and give neural networks that can play a vital role in self-
greater performance to Kohonen’s network. organisation is in dynamic synapses. The models
above all use static synapses, whose values only
Self-organising systems can be modelled using any change after the Euclidean distance has been found.
development environment and language that has With dynamic synapses, the synaptic weight can
access to basic mathematical libraries. Matlab has change by up to a few hundred percent dependent
libraries that provide functions not only for upon the inputs to the synapses, which has been
mathematics, but also specifically for self-organising found to be the case in biological synapses.
maps. Three such functions include:
 newc – returns a new competitive layer 9. CONCLUSION
 newsom – returns a new self-organising map This document explored in depth the issue of self-
organisation. It looked at sleep and how sleep or a
 newlvq – returns a new learning vector lack of it impacts on the body’s ability to self-organise.
quantisation network for classification It examined the role of self-organisation in the
development of the N.S. as well as the connections
These functions can be used in Matlab’s scripting between different neurons. It considered briefly self-
language, as well as graphically using Matlab’s Neural organisation at different levels namely the single and
Network Toolbox graphical user interface. A great networked cell levels.
feature of Matlab is the ability to extend its
functionality with the use of Matlab *.m files. An
example of this is the SOM Toolbox, a set of 141
Matlab files written by the Helsinki University of
Technology. These files build upon the functions

12. Self Organisation: Inspiring Neural Networks & IT Design
The N.S.’s of invertebrates and vertebrates were during Polyneuronal Innervation of Muscle Cells:
analysed to determine whether or not the size of both Atrophic Hypothesis” Royal Society Publishing 235,
these systems have any noticeable effect on self- pp. 299-320.
organisation. This document illustrated how self-
organisation could be modeled computationally. Bruske, J. & Sommer, G. (1995) “Dynamic Cell
Ideas for future work were also put forward. Structure Learns Perfectly Topology Preserving Map”
Neural Computation 7(4), pp. 845-865.
The examination of such a process can aid the
construction of neural networks, especially those that CELL MIGRATION CONSORTIUM (2007) “Overview of
aim to be self-organising or self-modifying. Such a the Migration Process” *Internet+, Date Accessed: 17
network would be able to adapt to changes in the March 2007, URL:
external environment when required. Shadbolt http://www.cellmigration.org/science/.
(2004) argues strongly that the “Insights from one
subject inform the thinking in another . . . The ultimate Chung, S. H. (2007) “Large-Scale Dynamical Models
ambition is an understanding of the C.N.S.”, advances and Estimation for Permeation in Biological
in the field of science often result in complimentary Membrane Ion Channels” Proceedings of IEEE
gains in the area of computing or vice versa. Estimation and Control of Large Scale Systems, 20(20),
pp.2-23.
There is no doubt that the computing world seeks its
inspiration from the world of biology. “We see Cilliers, P. (1998) “Complexity and Postmodernism:
complexity all around us in the natural world – from Understanding Complex Systems” London: Routledge.
the cytology and fine structures of cells to the
organization of the nervous system . . . Biological Clancey, W. J, & Smoliar, S.W. & Stefik, M. J. (1994)
systems cope with and glory in complexity – they seem “Contemplating minds: A forum for artificial
to scale, to be robust and inherently adaptable at the intelligence” London: The M.I.T. Press.
system level . . . Nature might provide the most direct
inspiration” Shadbolt (2004). There is no doubt that DAVIES, A. M. (1996) “The Neurotrophic Hypothesis:
“An attempt to imitate a biological phenomenon is Where Does it Stand” Biological Sciences 351(1338),
spawning innovative system designs in an emerging pp.389-394.
alternative computational paradigm with both specific
and yet unexplored potential” Bamford et al (2006). FREEMAN, W. J. (2005) “NDN, Volume Transmission
and Self-Organization in Brain Dynamics” Journal of
BIBILOGRAPHY Integrative Neuroscience 4(4), pp. 407-421.
Grossberg, S. (1994) “Letter to the editor:
bÄhr, M. (2006) “Brain Repair – Advances in Physiological interpretation of the self-organizing map
Experimental Medicine and Biology” American Journal algorithm” *online+, Date Accessed: 22 March 2007,
of Neuroradiology 27(9), pp. 2014. URL: http://www.cns.bu.edu/Profiles/
Grossberg/Gro1994KohonenLetter.pdf.
Bamford, S. & Murray, A. & Willshaw, D. J. (2006)
“Synaptic Rewiring in Neuromorphic VLSI for Kecman, V. (2001) “Learning and Soft Computing”
Topographic Map Formation” *Internet+, Date Cambridge: MIT Press.
Accessed 15 April 2007, URL:
http://www.see.ed.ac.uk/~s0454958/interimreport.p Kohonen, T. (1997) “Self-Organizing Maps” Berlin:
df. Springer-Verlag.
BENNETT, M. R. & ROBINSON, J. (1989) “Growth and Levine, M. W. & Shefner, J. M. (1991) “Fundamentals
Elimination of Nerve Terminals at Synaptic Sites of Sensation and Perception” 2nd ed. California:
Brooks & Cole.

13. Self Organisation: Inspiring Neural Networks & IT Design
Linaker, F. & Niklasson, L. (2000) “Time Series THE WORLD BOOK ENCYCLOPEDIA (1991) “The
Segmentation Using an Adaptive Resource Allocating Nervous System” London: World Book Inc.
Vector Quantization Network Based on Change
Detection” IEEE Computer Society, Proceedings of the Thomas, M. & Sing, H. & Belenky, G. & Holcomb, H. &
International Joint Conference on Neural Networks. Mayberg, H. & Dannals, R. & Wagner, H. & Thorne, D.
& Popp, K. & Rowland, L. & Welsh, A. & Balwinski, S. &
Lisboa, P. J. G. (1992) “ Neural Networks – current Redmond, D. (2000) “Neural Basis of Alertness and
applications” London: Chapman & Hall. Cognitive Performance Impairments During Sleepiness
- Effects of 24 Hours of Sleep Deprivation on Waking
Martinez, P. & Aguilar, P. L. & Perez, R. M. & Plaza, A. Human Regional Brain Activity” Journal of Sleep
(2002) “Systolic S.O.M. Neural Network for Research 9(4), pp. 335-352.
Hyperspectral Image Classification” in Zhang, D. & Pal,
S. K. (2002) “Neural Networks and Systolic Array Vreeken, J. (2002) “Spiking Neural Networks - An
Design” London: World Scientific Publishing Co. Introduction” Technical Report UU-CS-2003-008,
Institute for Information and Computing Sciences,
RASMUSSEN, C. E. & WILLSHAW, D. J. (1993) “Pre- Utrecht University.
Synaptic and Post-Synaptic Competition in Models for WILLSHAW, D. J. (1981) “The Establishment and the
the Development of Neuromuscular Connections” Subsequent Elimination of Polyneuronal Innervation
Biological Cybernetics 68, pp. 409-419. of Developing Muscle: Theoretical Considerations”
Biological Sciences 212 (1187), pp. 233-252.
Seiffert, U. & Lakhmi, C.J. (2001) “Self-Organizing
Neural Networks: Recent Advances and Applications WILLSHAW, D. J. (2007) “Self-Organisation in the
(Studies in Fuzziness and Soft Computing)” New York: Nervous System” Foresight *Internet+, Date Accessed:
Physica-Verlag. 17 March 2007, URL: http://www.foresight.gov.uk.
SHADBOLT, N. (2004) “From the Editor in Chief: ZURADA, J. M. (1992) “Introduction to Artificial Neural
Nature-Inspired Computing” IEEE Intelligent Systems Systems” New York: West Publishing Company.
19(1), pp.2-3.
www.businesscompany.com