1. A Research Platform for Coevolving Agents
with Producer/Consumer Interactions
R. E. Smith
The Intelligent Computing Systems Centre
The University of The West of England
robert.smith@uwe.ac.uk
Introduction
This report documents progress made during the author's summer research fellowship in
the Intelligent Business Systems Group at BT Research Laboratories. The focus of the
fellowship was on the exploration of evolutionary computation (EC) in agent-based
systems, with a mind towards applications of interest to BT research. The work built on a
framework for EC in mobile agents that the author had constructed in anticipation of the
fellowship (Smith & Taylor, 1998). The result of the fellowship is a platform for
examining coevolving agents in a controllable environment. This environment also has a
direct relationship to other agent-based system efforts going on at BT, under the
supervision of Dr. Paul Kearney. The platform is an extensible system where agents
interact in a producer/consumer economic world. This report describes the motivations
for this system, its design, preliminary results, and directions for future research.
Motivations for the EC Agents Framework
Agent-based systems are of particular interest in a variety of telecommunications
applications. The key qualities that agent-based components and systems exhibit are:
autonomy, reactivity, proactivity, and social behavior. Moreover, agents have the
possibility of mobility in complex network environments, putting software functions near
the computational resources they require. Agents can also explicitly exploit the
availability of distributed, parallel computation facilities (Franklin & Graesser, 1997;
Wooldridge & Jennings, 1996).
However, these qualities ultimately depend on the potential for agent adaptation. For
instance, if an agent is to operate with true autonomy in a complex environment, it may
have to react to a spectrum of circumstances that cannot be foreseen by the agent’s
designer. Autonomous agents may need to explore alternative reactive and proactive
strategies, evaluate their performance online, and formulate new, innovative strategies
without user intervention. Moreover, for systems of agents to behave in this manner,
social interactions between agents may also need to adapt and emerge as conditions
change.
Areas where agents could benefit from adaptation are addressed by active research in
machine learning (e.g., classification of unforeseen inputs, strategy acquisition through
reinforcement learning, etc.). However, many machine learning techniques are focused
on centralized processing of databases to formulate models or strategies. In contrast,
evolutionary computation (EC) techniques are inherently based on a distributed paradigm
(natural evolution), making them particularly well suited for adaptation in agents. The

2. following sections provide further motivation for investigating EC in agent-based
systems.
Empirical Motivation
Consider the author's previous work on acquiring novel combat maneuver strategies for
fighter planes (Smith & Dike, 1995) using genetics-based machine learning. In this
project, systems of distinct rules coevolve under the action of an EC system to specify
complex maneuvers, like the one shown in Figure 1. In this maneuver, the plane on the
right is controlled by the genetics-based machine learning system, while the plane on the
left is following standard combat logic. Both planes execute a change in their controls
every 1/10th of a second. Each control action is specified by a separate rule. Note that
rules firing early in the maneuver sacrifice immediate advantage (turning nose away from
the opponent) such that later rules can dictate a tight turn, and overwhelm the opponent.
This demonstrates that although the genetically-learned rules evolve as separate entities
(agents), they exhibit complex cooperative behaviors.
Figure 1: A maneuver dictated by a set of rules, coevolved in an evolutionary
computation system (Smith & Dike, 1995).
This result shows that complex, innovative, multi-component adaptive systems can
emerge from EC processes.
Theoretical Motivation
Although the empirical evidence is compelling, one must ask whether there is a more
mathematically justifiable reason for examining EC in agent-based systems. To provide
and answer, consider the oft-cited motivations for agent-based systems themselves:
• proactivity
• reactivity
• social interaction

3. • autonomy
• distributability
Each of these motivations, to some extent, suggests adaptation or learning. However,
when one considers agent-based systems, and social interaction, one must consider the
impact of learning on the system of agents, while maintaining distributability. In other
words, one must consider how to obtain and propagate learned information in a system of
agents, without a centralized coordinator of this activity.
Holland’s seminal book, Adaptation in Natural and Artificial Systems (Holland, 1975)
was primarily addressed at this issue. Although the book is often cited as the first on
genetic algorithms (GAs), one of the earliest forms of EC, it is much more directed at
general issues of adaptation in systems of distributed entities. The book mathematically
examines how populations of entities interacting in “systems” can and should adapt.
Extending this to agents is a natural implication of Holland's work.
There are four main theoretical pillars upon which Holland's theories are based:
• The K-armed Bandit
• The Schema Theorem
• The Building Block Hypothesis
• Implicit Parallelism
To understand these motivations, consider an agent interacting with other agents and
entities in a complex environment. Assume each agent is made up of some sort of
encoded feature set. Given this assumption, one can ask the general question of how one
should determine features to try in other agents? Clearly, there is seldom a clear mapping
from these features to agent fitness. However, Holland manages to deal with this question
in a general context by casting it as a probabilistic decision problem: the k-armed bandit.
Consider two alternative “features” that an agent may have. Given a feature’s interactions
with other features within the agent, with other agents, and with a complex environment
one has, at best, a noisy evaluation of each alternative feature.
Therefore, to evaluate the feature, one can spend some time evaluating the alternative
features in a variety of agents and contexts. However, this leads to a potential loss in
agent performance while trying out features that turn out to be less advantageous. So, it is
desirable to quickly incorporate features that appear advantageous into our agents.
However, this leads to a potential performance loss if a less advantageous feature is
selected prematurely. This is the classic dilemma of exploration versus exploitation.
One can overcome this dilemma by increasing the number of agents with the apparent
best feature gradually while continuing to experiment with other candidates. However,
one has a choice with regard to the speed of this feature propagation. As a function of
time (t), should one increase the number of agents (m(H,t)) with the apparent best feature
(H):
• linearly: m( H , t + 1) = m( H , t ) + C
geometrically: m( H , t + 1) = m( H , t ) + Ct
D
•

4. • exponentially: m( H , t + 1) = Cm( H , t ) , or
superexponentially: m( H , t + 1) = Ct m( H , t )
D
•
where C and D are constants?
Holland’s K-armed bandit argument shows that, regardless of the distribution of noise
surrounding the feature’s fitness, one should allocate the apparent best feature
exponentially, to insure minimum loss. Moreover, Holland's Schema Theorem of GAs
shows that a reproductive (EC) plan does just such an allocation.
This leads to Holland's Building Block Hypothesis, which is the central heuristic in GA
search. This heuristic assumes that if one can propagate advantageous features (building
blocks) in a population of agents, they will combine (build) into improved agents, and
agent system behaviors. Although this heuristic is clearly not adequate for all situations,
it seems appropriate, given a general system with a distributed character.
Moreover, Holland shows that in considering a population of agents, one implicitly
applies the k-armed bandit argument, through the schema theorem, to a large number of
building blocks to which the building block hypothesis might accurately apply. Clearly,
one cannot expose each and every feature to the same exponential decision process, since
the space of features (where a feature is any combination of elements with an agent) is far
larger than the space of possible agents. . However, Holland’s implicit parallelism
argument shows that for moderate population sizes N, the number being processed is on
the order of N3.
Given these empirical and theoretical arguments, Holland’s genetic adaptive plans seem
the appropriate way to propagate features through a system of agents. Moreover, these
EC processes implicitly exploit parallelism, while remaining trivial to explicitly
parallelize. Therefore, EC methods are one of the most natural machine learning
techniques to transfer general-purpose adaptive capabilities to agent-based systems.
Philosophical Motivation
The previous sections outline motivations for exploring EC in agent-based systems. In
many ways, all research on coevolving EC systems is apropos to agent-based systems.
However, EC systems that use actual software agents have been rare. One possible
exception is work on Tierra (Ray, 1990), which involves the coevolution of pieces of
computer code. However, Tierra "agents" are not standard software components. They
cannot perform the actions of general software agents, and require an EC specific
software environment in which to run.
It is important to investigate EC (and other distributed AI techniques) within an
environment of real-world, standards-based software components, for reasons of
embodiment (Brooks, et al., 1998). In the area of robotic artificial intelligence,
embodiment means that a robot's sensory-motor systems cannot be fully disconnected
from behavior, capability to adapt, and intelligence. Internal representations, reasoning,
and behavior are ultimately grounded in the physical nature of the robot, and its physical
world. Therefore, Brooks and others are addressing the issue of robotic intelligence
through hardware robots, where embodiment is considered directly.

5. Similar arguments can be made for artificial intelligence in agent-based systems. To
consider the embodiment of adaptation, behavior, and intelligence in such agents, one
must examine real sensors, effectors, and environments along with those agents. Thus, it
is important to consider EC in systems of standards-based software agents, that interact in
a generalized computing environment, and are not specifically designed for EC
experiments..
Pre-BT Fellowship Results
Until recent work by the author, no standards-based, EC agent framework existed. Before
the BT fellowship, the author had developed an agent-based EC framework. There are
several potential advantages to this framework. The primary is advantage is that such a
framework could allow several users or agents, at distributed locations, to interact in EC
processes in a standardized way, without a priori knowledge of these processes. Potential
applications include collaborative search, and collaborative problem solving. Details of
the framework itself are included in this report's Appendix (Page 16).
The framework was tested on problems where coevolution was not necessary (i.e.,
optimization problems). Results from one such experiment are shown in Figure 2.
30
25
Agent Fitness
20
15
10
5
0
0.0E+00
1.0E+06
2.0E+05
4.0E+05
6.0E+05
8.0E+05
1.2E+06
Tim e (m illiseconds)
Figure 2: A result with the EC agent framework applied to a simple optimization task.
In this experiment, each agent was set with the simple task of finding fit mates. Every
agent advertises (through a Plumage object) its own fitness as the number of ones in an
internal, 32-bit chromosome. Every agent trusts the fitness advertised by other agents,
and as a result the number of agents with the "all ones" chromosome gradually increases.
Note that such trust is not necessary for the general operation of the framework.
In other words, this experiment is an agent-based implementation of an EC benchmarking
scenario known as the 32-bit "counting ones" problem. All agents play both a paternal
and maternal role. Once an agent "mothers" a child, it dies, thus insuring a constant

6. population size. One also can think of each matrilineal trace as being a single agent that
can modify its own characteristics. The result in Figure 2 is similar to those of standard
EC systems, despite the asynchronous, decentralized, agent-based nature of the system.
The distributed nature of the experiment is better illustrated in Figure 3
30
25
Agent Fitness
20
15
10
5
0
160
78050
123250
184000
237990
315540
376180
441490
523820
618400
689860
803830
860240
938730
987340
Agent Generated
(Tim e Labels)
(m illiseconds)
Figure 3: A single maternal line, from the experiment shown in Figure 2.
The figure shows the fitness of a single "maternal line". Imagine that this line of agents
remains the "property" of a single user. This user could have genetic interactions with
other users, through the framework. Such interactions would improve those agents
belonging to the user.
Although these results are encouraging, they do not consider a key issue in both EC and
agent-based systems research. The fitness value of each agent in the previous experiment
is independent of the fitness values of the other agents in the population. Moreover, their
only interactions are genetic. The goal of the BT fellowship research was to begin a
deeper investigation, where evolving agents were interdependent. Such efforts should
relate to previous centralized genetics-based machine learning systems (like those in
Smith & Dike, 1995), and more realistic agent based systems. Moreover this research will
being to address the watchwords of modern EC research, including niches, coevolution,
and emergence.
There are proven theories on how an EC system can implicitly balance a diverse,
cooperative population, while exploiting Holland’s theories (Deb, 89; Smith, Forrest, &
Perelson, 95; Horn & Goldberg, 96). However, these issues have yet to be evaluated in a
standard framework of asynchronous software agents. The BT work has set the stage for
this examination, in a producer/consumer agent world.

7. The Producer/Consumer Interaction Framework
Much work has been done in the area of coevolution in EC systems (Rosin & Belew,
1997). However, many of the systems in past research are difficult to parameterize for
detailed examination of various effects. In consultation during the BT fellowship, Paul
Kearney, Walter Merlat, and the author developed a much more tractable system for
these examinations, based on producer/consumer economics.
The problem setting is as follows. Any number of agents interact in a specific economy.
Each agent in the system has a pre-specified number of workers at its disposal. These
workers can be allocated to any one of M possible technologies. There are N possible
goods in the economy. Each technology converts one set of goods into another set.
Technologies are a pre-specified aspect of the economic world.
From an agent's perspective, the goal is to allocate its workers, and maximize its profit.
An agent must perform this task, given that the price of each good varies with time, with
the action of other agents, and with external market forces. Each of these price variations
takes place through a market process of supply and demand.
From the perspective of the overall system, one goal is to demonstrate the evolutionary
emergence of interesting, productive economic interaction amongst selfish, co-evolved
agents. Another goal is to examine variations in emergent behavior, based on variations
in system parameters, and individual agent behavior.
Software Structure
The system designed during the BT fellowship is intended as a platform for a wide
variety of ongoing experiments. Therefore, it was important to maintain an extensible,
object-oriented design. The following are the key classes in that design. Javadoc and
source code files are available within the BT Intelligent Business Systems Research
Group. Like the EC Framework, the agent classes here are built around IBM's Aglets.
However, they could be easily transferred to other agent systems. Each class is extensible
for future experimentation.
The marketAgent class
Agents of this class process orders for goods from producer/consumer agents, and credit
these agents’ cash account.
The economicWorld class
This class defines the "rules" of the economic world, including
• M, the number of technologies
• N, the number of goods
• the set of technologies, and what they do, and
• external consumers and producers (of end goods and raw materials, respectively).
The producerAgent class
This is the key agent class. Each producerAgent responds to “Current Prices”
messages from agent(s) of the marketAgent class. The producerAgent determines a

8. worker allocation based on this current information, and its own Genotype object (see
details of the EC agent framework in the Appendix, Page 16). The producerAgent then
submits an order message to the marketAgent, with negative and positive quantities
representing sell and buy orders for each good.
When triggered by an internally defined condition, a producerAgent searches out
mates, and attempts to produce child producerAgents.
Each of these basic classes is extensible for a number of market and genetic behaviors.
Also, there are a set of utility classes that allow for graphical interaction with an
experiment. Screen shots are shown in Figure 4.
Figure 4: Screen shots from the Java-based producer/consumer system.
Preliminary Experiments
To illustrate the basic operation of the producer/consumer economic framework, a
simplified economic world was constructed.
The Market
In this world, there is only one marketAgent, which maintains a store of each good.
Each store can have positive and negative quantities, and represents the excess supply of
a good. The effect of supply and demand is simulated, by basing each good's price, P, on
a simple function of the amount in the marketAgent's excess store of that good, S:
P = exp( − λ S )

9. where λ is a parameter.
Prices are only recalculated at the end of a trading day. A trading day begins with the
marketAgent broadcasting a current price message to every producerAgent. The
market then asynchronously processes orders from each producerAgent that received
this current price message. The trading day is complete when all these orders are
processed. Prices are then recalculated, and a new trading day begins.
The Agents
In this preliminary experiment, the producerAgents are also simplified. In its
Genotype, each producerAgent has a BooleanGene for each of the M
technologies. A producerAgent allocates its workers evenly to each technology that is
marked true in its Genotype.
In the preliminary experiment, producerAgents trade for 10 trading days, then they
(asynchronously) broadcast and receive Plumage and Sperm objects, to locate mates.
In this case, a Plumage object contains the broadcasting agent's evaluation of its own
profit per trading day, which is implicitly trusted by the other agents. Note that such trust
is an aspect of this experiment only, and is not, in general, a necessary aspect of the
framework. When five or more Plumage objects are received by an agent, the best is
picked, and the associated Sperm object is used to create a producerAgent child.
Then the maternal parent agent dies. Note that in the paternal role, an agent can have
many children, based on it's relative profit potential, while in the maternal role, an agent
only has one child.
The Economic World
In the preliminary experiment, it was desirable to have a set of technologies and goods
that interacted in a uniform, extensible manner. Therefore, the following scheme was
employed. Goods were "stacked" into R rows and C columns, as shown in Figure 5.

10. End goods out
good(R,0) good(R,1) good(R,2)
... good(R,C)
... ... ...
...
good(1,0) good(1,1) good(1,2)
... good(1,C)
good(0,0) good(0,1) good(0,2)
... good(0,C)
good(0,C)
Raw materials in
Figure 5: The simplified economic world used in preliminary experiments.
In this figure, the unfilled arrows represent technologies. Note that each technology takes
two goods in one row, and converts them into a single good in the next row. Such
conversions are conservative, such that x quantity of one good, combined with y quantity
of another good, yields (x+y) quantity of the third good. Note that the world "wraps
around" at the right and left edges. Thus, the world is expandable via the R and C
parameters, and goods in the R-2 intermediate rows are treated identically.
The filled arrows in Figure 5 represent external suppliers of raw materials, and consumers
of end goods. The raw material goods (in the lowest row) and the end goods (in the
highest row) are controllable through the supply and demand of external suppliers and
consumers. Note that the total number of goods N=R*C, and the total number of
technologies M=(R-1)*C. The preliminary experiment presented here is the minimal
meaningful form of this world. There is one layer of intermediate goods (R=3). For
simplicity, C=3, giving a total of N=9 goods and M=6 technologies. External supplies
and demands are manipulated such that there is a fixed price for the raw materials, and
another fixed price for end goods.
Results
In the first experiment, the prices of these raw materials and end goods are both
maintained at a value of one (i.e. the external supply and demand adjust so as to maintain
S = 0 for these goods). A population of 50 producerAgents interacts in the market, and
in the EC mating process.
Prices of the three intermediate goods in this simulation are shown in Figure 6. Variations
in these prices are due to evolution of the agents involved in the market. As prices for
intermediate goods rise (due to low supply and high demand), agents evolve to produce
these high-profit goods. This causes supply to increase, demand to fall, and prices to fall.

11. Then the agents evolve towards being consumers of the low-cost intermediate goods.
Note that the agents involved have no memory of price history. All changes in behavior
occur through the action of EC.
18
16
14
12
10
Prices
8
6
4
2
0
1
169
337
505
673
841
1177
1345
1513
1681
1849
2017
2185
2353
2521
2689
2857
1009
Trading Day
Figure 6: Prices of three intermediate goods as a function of trading day in an evolving
economy of producers and consumers.
In a second experiment, the price of raw materials remains fixed at 1, but the price of end
goods is raised to the fixed value of 5. Thus, a fixed price gradient is maintained from
raw materials to end goods. Prices of the three intermediate goods in this simulation are
shown in Figure 7. Once again, all price variations are due to agent evolution.
50
45
40
35
30
Price
25
20
15
10
5
0
364
727
1
122
243
485
606
848
969
1090
1211
1332
1453
1574
1695
1816
Trading Day
Figure 7: Prices of three intermediate goods as a function of trading day in an evolving
economy of producers and consumers, with a fixed, positive price gradient.

12. Analysis
Clearly, these are only preliminary results from the system. However, they do point to the
advantages of the framework developed during the BT fellowship. Unlike much of the
extant research on coevolving systems (for instance, the fighter plane research shown in
Figure 1), this particular setting yields results can be analyzed. For instance, consider a
Fast Fourier Transform of the data in Figure 6, shown as a power spectral density in
Figure 8. In this figure, the power spectrum for each good is normalized, such that the
maximum power equals one.
1
Relative
0.5
Power
0
1024
512
341
256
205
171
146
128
114
102
93
1/Frequency
(wavelength in
trading days)
Figure 8: A power spectral density of prices for the three intermediate goods shown in
Figure 6. Power values are normalized, such that the maximum power for each good
equals one.
The graph shows the complexity of the price fluctuations for each good (i.e., there are
many significant frequency components) Note that, given the symmetries of the
economic world, there is no clear reason that each good should have different price
variations. However, although the goods all share some frequencies of variation in
common (chiefly those around 1/300 trading days), there are broad variations in the
secondary frequency components. This is most likely due to subtle, "genetic drift" effects
in the EC system.
Contrast these results to those shown in Figure 9, which is a similar power spectral
density for the experiment shown in Figure 7. Recall that in this experiment, there is a
positive price gradient maintained from the raw materials to the end goods. The power
spectral density shows that the price variations in this experiment are much less complex
than those in the previous experiment (i.e., there are fewer pronounced secondary
frequencies). Moreover, the price variations in the intermediate goods (although different
in scale) are more similar than those of the previous experiment (i.e., there are fewer
differences between the spectra of the different goods).

13. 1
Relative
0.5
Power
0
1024
512
341
256
205
171
146
128
114
102
93
1/Frequency
(wavelength in
trading days)
Figure 9: A power spectral density of prices for the three intermediate goods shown in
Figure 7. Power values are normalized, such that the maximum power for each good
equals one.
Future Study
Clearly, these experiments only begin to scratch the surface of a large area for
investigation that is of importance to both the EC and the agent-based systems
communities. The research environment presented here is uniquely suited to this
investigation. It has the advantages of
• embodiment of the agents in a generalized, fully asynchronous agent system (IBM's
Aglets),
• a flexible, object-oriented design, well-suited to modifications, extensions, and
further experimentation,
• possible extension into real-world applications, and
• coevolved results that manifest themselves in a clear, analyzable form.
The last point is key: previous research efforts on co-evolved systems have used
experimental environments where clear-cut examination of agent interactions was
difficult or impossible.
Several further experiments with the current system present themselves immediately,
including those discussed below.
Problem Variations
In the preliminary experiments shown here, supply and demand are manipulated to
maintain constant prices for raw materials and end goods. However, a more realistic
simulation would consider fixed flow rates for raw materials and end goods, or flows that

14. are dependent on prices. This is a fertile area for exploration, that is easily supported by
the current framework.
Another problem variation lies in the complexity of technology interactions. In the
preliminary experiments presented here, there is a uniformity of the economic world,
induced by the similarity of all technologies. Breaking symmetries in the economic
world, through the introduction of more complex technologies, is an interesting area for
investigation. Once again, this extension is easily introduced into the current framework.
Agent Variations
The most significant simplification in the preliminary experiments is that the agents
assign their workers blindly, as dictated by their genetic code. All reactivity to price
patterns occurs on the evolutionary time scale.
An obvious next step is to build simple price reactivity to the agents. As a first step, this
would involve no learning of price patterns during the agent's lifetime. The agent would
only have the capacity to shift worker allocation in reaction to current prices, due to a
strategy dictated by its genetic code. This closely parallel's previous work by the author
on coevolution in learning classifier systems (Smith, 1995).
A second step would by to allow the agents to adapt internal parameters during their
lifetimes. This would involve memory of past price patterns within the agent, and
resulting lifetime learning. Through such learning, an individual agent would have the
capacity to predict and anticipate price patterns. Once again, this closely parallels
previous work by the author (Smith, 1995).
Note that as the complexity of the agent's strategies increases, issues of embodiment
become more important. An agent’s ability to sense, calculate, and execute are implicitly
tied to its computational interactions with other agents, and interactions with its
computational environment. In these situations, the sort of framework presented here is
key to future investigation. The current framework will easily support these extensions.
EC Variations
The modifications suggested above lead to important issues in EC. Chief among these is
the Baldwin Effect. This effect is the interplay of lifetime learning and genetic evolution.
If an agent is evolved to learn a particular behavioral strategy, that agent's descendents
are more likely to learn that strategy as well, although the strategy is not specifically
inherited genetically. This interplay of genetic learning and lifetime learning is key to
desirable emergent behavior in agent-based systems. The current system is well poised
for an examination in this area.
Many other more technical EC issues can be examined within the current system,
including:
• more complex coding of strategies, including genetic programming techniques
• mating restrictions to promote distinct species of agents
• diploidy and dominance to improve tracking of time variations in the agent-based
system (Smith, 92)

15. Given the object based nature of the EC agents framework, each of these extensions is
straight forward.
Final Comments
The system of producer/consumer agents developed during the BT fellowship, coupled
with the author's previous work on constructing a framework for EC in agent-based
systems, presents a clear opportunity for further research. The current system is
analyzable, while maintaining a realistic, embodied character of the agents. Preliminary
results show the promise of investigations using this system. Given that EC seems a
theoretically and empirically appropriate technique for developing social, adaptive
behavior in agents, further investigations with this system are promising.

16. Appendix: Details of the EC Agents Framework
Design Philosophy
The structure of a typical EC software system is shown in Figure 10. In such systems, a
central program manipulates data structures that comprise the GA population. Although
these structures may act as agents, through interactions with an external world, they do
not function autonomously. All interactions are dictated by the central GA program. Even
in distributed GAs (Kapsalis, Smith & Rayward-Smith, 1994) this structure is
maintained, though the central GA program may be replicated at several locations.
Centralized GA program
Evaluation/Interaction
population
w/ external applications
Selection
Recombination
Mutation
Individuals are
simple genetic encodings
of parameters
Figure 10: The structure of typical GA (EC) software systems.
A truly agent-based GA structure is shown in Figure 11. In this system, the population is
comprised of standardized software agents, that interact with each other, and with the
external world, autonomously. There is no central GA. Agents can interact in general
ways, not just genetically. When interacting genetically, these agents exchange standard
objects that allow them to select mates, exchange genetic material, and construct
children, all based on their own internal goals and procedures.

17. Generalized Agent Environment
contains genetic and
non-genetic agents
Agent’s Evaluation
and Selection of Mates
Agent’s
genetic and non-genetic Recombination Strategy
message interactions
with other agents Agent’s
and the “outside world” Mutation Strategy
“Child” Agent
Figure 11
Implementation Specifics
The EC agents framework was implemented in Java, for its machine independence. The
framework is also based on IBM's Aglets, for agent infrastructure, relationship to
emerging systems standards, and the possibility of agent mobility. Javadoc is available
for the entire framework, from the Intelligent Business Systems Group at BT Labs. The
following sections outline the basics of the framework.
The EC agents framework consists of two main packages. The ECbasics package
defines a set of extensible objects for EC. It is not specific to any particular agent system.
The GAglet package defines of set of EC extensions and utilities for IBM's Aglets.
These packages are discussed in greater detail below.
The ECbasics Package
This package defines a number of key interfaces for the objects exchanged in EC
interactions. These include:
Randomizable
This interface defines methods that allow one to generate a "random" instance of an
object. What this means depends on the particular class that implements the interface. For
instance, for a BooleanGene (see below), this interface implements methods that
generate a gene with an equiprobable, random (true or false) value.

18. Mutatable
This interface defines methods that cause a random change in an object. What this means
depends on the particular class that implements this interface. For instance, in a
BooleanGene (see below), this interface implements methods that change a true
valued gene to a false valued gene, or vice versa.
Gene
This interface trivially extends Randomizable and Mutatable.
Genotype
This interface extends Randomizable and Mutatable, but also defines methods to
generate objects that implement Sperm or Egg interfaces (see below).
Gamete
This interface defines methods that can "get" and "set" segments of an object. What this
means depends on the class that implements this interface.
Sperm
This interface extends Gamete, but only trivially.
Egg
This interface extends Gamete, but also defines routines that can combine the
implementing class with a class implementing the Sperm interface. The result is an
object implementing the Genotype interface (i.e., the genotype of a potential child).
Example Objects
BooleanGene implements Gene
This class is the typical Boolean gene often used in GAs.
FloatGene implements Gene
This class is a floating point gene, like those often used in evolutionary programming. Its
mutation operator is a Gaussian "creep" around the current gene value.
VectorGenotype extends Vector, implements Genotype
This class is the typical chromosome string often used in GAs, but it also inherits all the
functionality of the Java Vector class.
VectorEgg and VectorSperm extend Vector, implement Egg or Sperm,
respectively
These classes implement the recombination of string chromosomes typically used in
GAs.
Other Possibilities
Within the framework, Genes, Genotypes, Sperm, and Eggs can be any type of object.
For instance, one could easily implement classes like
• HashtableGenotype

19. • TreeGenotype
• IntegerGene
• VectorGene
etc.
The GAglet Package
This package implements EC functionality for agents within the IBM Aglet framework.
This key classes of this package are discussed below.
There are two key utility classes:
Plumage
Objects from this class are used by agents to "advertise" themselves for mating to other
agents.
Phylum
Objects from this class are used to initialize the basic genetic behaviors and intentions of
a "first generation" agent, including its preferences in mates.
There are also two key agent classes:
GAglet extends Aglet
This class is a basic Aglet agent, with methods that allow it to be initialized with either
Phylum objects, or objects that implement the Genotype interface.
Egglet extends GAglet
This class implements methods that provide the agent with mate seeking behavior. These
methods broadcast Plumage and Sperm objects, and call appropriate methods to create
new child agents.

20. References
Aglets Workbench. http://www.trl.ibm.co.jp/aglets/
Brooks, R.A., C. Breazeal (Ferrell), R. Irie, C. Kemp, M. Marjanovic, B. Scassellat and
M. Williamson (1998). Alternate essences of intelligence. To appear in Proceedings of
AAAI-98 Conference. AAAI Press.
Deb, K. and Goldberg, D. E. (1989). An investigation of niche and species formation in
genetic function optimization. Proceedings of the Third International Conference on
Genetic Algorithms. p. 42--50.
Franklin and Graesser (1997). Is it an agent, or just a program? : A taxonomy for
autonomous agents. In Proceedings of the Third International Workshop on Agent
Theories, Architectures, and Languages. Springer-Verlag. pp. 21-35.
Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine
learning. Addison-Wesley. Reading, MA
Holland, J. H. (1975). Adaptation in natural and artificial systems. The University of
Michigan Press. Ann Arbor, MI.
Horn, J. and Goldberg, D. E. and Deb, K. (1994). Implicit niching in a learning classifier
system: Nature's way. Evolutionary Computation, 2(1). p. 37-66.
Kapsalis, A., Smith, G. D. and Rayward-Smith, V. J. (1994). A unified paradigm for
parallel genetic algorithms. In T. Fogarty (ed.) Evolutionary Computing AISB Workshop.
Springer-Verlag. 131-149.
Koza, J. R. (1992). Genetic Programming: On The Programming of Computers By
Means of Natural Selection. MIT Press.
Ray, Thomas (1990). An approach to the synthesis of life. In Langton, C., Taylor, C.
Farmer, J. D. and Rasmussen, S. (eds.) Artificial Life II. Addison-Wesley. p. 371-408
Rosin, C. D. and Belew, R. K. (1997). New methods in competitive coevolution.
Evolutionary Computation. 5(1). pp. 1-29
Smith, R. E. (1995). Memory Exploitation in Learning Classifier Systems. Evolutionary
Computation, 2(3). p. 199-220
Smith R. E. and Dike, B. A. (1995). Learning novel fighter combat maneuver rules via
genetic algorithms. International Journal of Expert Systems. 8 (3). 247-276.
Smith, R. E., Forrest, S. and Perelson, A. S. (1993). Searching for diverse, cooperative
populations with genetic algorithms. Evolutionary Computation, 1(2). p. 127-149.
Smith, R. E. and Goldberg, D. E. (1992). Diploidy and dominance in artificial genetic
search. Complex Systems, 6(3). p. 251-285.
Smith, R. E. and Taylor, N. (1998).A framework for evolutionary computation in agent-
based systems. In C. Looney and J. Castaing (eds.) Proceedings of the 1998
International Conference on Intelligent Systems. ISCA Press. p. 221-224
Wooldridge and Jennings (1996). Software agents. IEE Review. January 1996. 17-20.