Yusuke Goto (iwate Pref. Univ.) and Shingo Takahashi (Waseda Univ.) Agent-Based Simulation Analysis of Performance Measurement Systems Considering Uncertainties of a Learning Model The 25th European Conference on Operational Research July 9, 2012 (Vilnius, Lithuania)

1. Agent-based Simulation Analysis of
Performance Measurement Systems
Considering Uncertainties of a Learning Model
EURO XXV
The 25th European Conference on Operational Research
July 9, 2012
Vilnius, Lithuania
Yusuke Goto* and Shingo Takahashi**
*Iwate Prefectural University, Japan
**Waseda University, Japan
1

2. Introduction
•Performance Measurement System (PMS) is
a major tool of management control
•PMS indirectly controls member s behavior
by evaluation
•We have little knowledge about which PMS
design is effective in a given situation
•Effect of PMS is highly contingent on the
situation:
- organizational culture
- environmental factors
- individual characteristics ...
•Traditional positivistic approaches play only
a limited role
Simulation analysis is a promising approach
2

3. Agent-based Simulation
To simulate the effect of PMS...
•We must capture individual behavior (micro level)
and organizational performance (macro level)
•We must develop a human behavioral model
•We must consider the variety of individual value
and attitude in the behavioral model
Agent-based simulation (ABS):
•Micro-macro link of the organization system is
considered
•Members in an organization are autonomous
agents
•Agents determine their own behavior by
referring to their internal decision-making model
3

4. Modeling agent s behavior
Behavioral framework:
•Agents have their own attitude for their actions
•Their actions are based on their attitude
•Their attitude has variety
•They learn their attitude to improve their evaluation
by the PMS
agent s
attitude
PMS
evaluation
agent s
behavior
learn / change
Uncertainties in agent s learning model
•When do agents learn? Always?
•Who learns? Everybody?
•How do agents learn? Are there some noises?
4

5. Purpose and Method
Research questions:
•Does agent s learning model have an affect on the
effect of PMSs?
•Is this affect contingent to PMS design?
Research method:
•Modeling the target organization: a sales organization
•Simulation analysis:
8 different learning models
3 different PMS designs
•Discussion
•Summary
5

6. Sales organization
Sales organization
•The sales division has 10 groups
•Every group has 10 agents
•Every group has 100 customers initially
Agent (sales person)
•capability： cp ( 0 cp 1)
agent s sales capability is equal to the probability
of sales success
•attitude：
•aggressiveness: ag {0,1,..., 7}
•cooperativeness: co {0,1,..., 7}
•innovativeness: in {0,1,..., 7}
6

7. Flowchart
Sales related activities by agents
1.Visit
2.Market cultivation
3.Education
4.Training
Evaluation by the PMS
Learning of agents attitude
7

8. Sales related activities
Agent s activities
1.Visit：
•selling of a good to a customer
•The probability of sales success is equal
to agent s sales capability cp
2.Market cultivation：
•seeking new customers for a good mi + +
3.Education of teammates:
•increase the cp of all teammates by EO ( 0 EO 1)
4.Training:
•increase sales person s cp
Attitude
•aggressiveness: ag
•cooperativeness: co
•innovativeness: in
by ES ( 0
ES
1)
Behavior
•market cultivation
•education of teammates
•training
8

9. PMS (Performance Measurement System)
•PMS evaluates every agent based on his/her behavior
and result
•PMS intends to control agent s behavior and
organizational performance by the evaluation for agents
1.Individual sales:
2.Group sales:
(agent i belongs to group j)
3.Behavioral control:
9

10. Learning model
Basic algorithm
•Learning as an imitation process based on
Genetic Algorithm (GA)
•Agents imitate another agent s attitude
whose evaluation is higher
•Sometimes agents changes the attitude randomly
Parameters in a learning model
1.Who?
Number of agents who learn their attitude
2.When?
Agents learn their attitude until their evaluation
th
value meets their learning threshold
3.How?
Probability of random change
10

11. Simulation experiment
8 different learning models:
= {100, 90}
1.Who?: Number of learners
2.When?: learning threshold
3.How?
th
= {1000, 1.1}
= {0.005, 0.001}
3 different PMSs
1.Individual sales:
2.Group sales:
3.Behavioral control:
•1,000 trials for each pattern
•Performance to control: agent s sales
11

12. Result (1/3): Individual Sales
30
25
20
15
Avg. # of sales
0
5
10
25
20
15
10
0
5
Avg. # of sales
30
35
PMS (individual sales):
P1_1000_100_0.001
35
P1_1000_100_0.005
0
10
20
30
40
50
60
0
10
20
30
40
50
60
40
50
60
25
20
60
0
10
20
30
30
25
20
15
Avg. # of sales
20
15
0
5
10
10
25
30
35
P1_1.1_100_0.001
35
Cycle
5
Avg. # of sales
50
30
30
10
20
30
40
50
60
0
10
20
30
P1_1.1_90_0.005
P1_1.1_90_0.001
30
25
20
15
Avg. # of sales
20
15
0
5
10
0
5
10
25
30
35
Cycle
35
Cycle
Avg. # of sales
40
15
Avg. # of sales
20
0
0
•Width of possible outcomes
60
5
10
Cycle
•Bifurcation
50
0
0
P1_1.1_100_0.005
Model dependent trend:
40
10
30
25
20
15
Avg. # of sales
10
0
•Sales decline gradually
•Sales vary considerably
60
P1_1000_90_0.001
5
General trend:
50
35
P1_1000_90_0.005
40
Cycle
35
Cycle
0
10
20
30
Cycle
40
50
60
0
10
20
30
Cycle
12

13. Result (2/3): Group Sales
30
25
20
15
Avg. # of sales
0
5
10
25
20
15
10
0
5
Avg. # of sales
30
35
PMS (group sales):
P11_1000_100_0.001
35
P11_1000_100_0.005
0
10
20
30
40
50
60
0
10
20
30
40
50
60
40
50
60
30
25
20
15
Avg. # of sales
20
30
40
50
60
0
10
20
30
30
25
20
15
Avg. # of sales
20
15
0
0
5
10
10
25
30
35
P11_1.1_100_0.001
35
Cycle
P11_1.1_100_0.005
Avg. # of sales
60
5
10
Cycle
0
10
20
•Some declinations
30
40
50
60
0
10
20
30
P11_1.1_90_0.005
P11_1.1_90_0.001
30
25
20
15
Avg. # of sales
20
15
0
5
10
0
5
10
25
30
35
Cycle
35
Cycle
Avg. # of sales
50
0
0
5
Model dependent trend:
40
10
30
25
20
15
Avg. # of sales
10
0
•Sales improve gradually
•Sales variation
60
P11_1000_90_0.001
5
General trend:
50
35
P11_1000_90_0.005
40
Cycle
35
Cycle
0
10
20
30
Cycle
40
50
60
0
10
20
30
Cycle
13

14. Result (3/3): Behavioral control
30
25
20
15
Avg. # of sales
0
5
10
25
20
15
10
0
5
Avg. # of sales
30
35
PMS (behavioral control):
P9_1000_100_0.001
35
P9_1000_100_0.005
0
10
20
30
40
50
60
0
10
20
30
40
50
60
40
50
60
30
25
20
15
Avg. # of sales
20
30
40
50
60
0
10
20
30
30
25
20
15
Avg. # of sales
20
15
0
5
10
0
5
10
25
30
35
P9_1.1_100_0.001
35
Cycle
P9_1.1_100_0.005
Avg. # of sales
60
5
10
Cycle
0
10
20
30
40
50
60
0
10
20
30
P9_1.1_90_0.005
P9_1.1_90_0.001
30
25
20
15
Avg. # of sales
20
15
0
5
10
0
5
10
25
30
35
Cycle
35
Cycle
Avg. # of sales
50
0
0
and saturate
•Sales converge
40
10
30
25
20
15
Avg. # of sales
10
0
•Sales improve gradually
60
P9_1000_90_0.001
5
General trend:
50
35
P9_1000_90_0.005
40
Cycle
35
Cycle
0
10
20
30
Cycle
40
50
60
0
10
20
30
Cycle
14

15. Discussion
Learning model effect
1.width of possible outcomes
2.bifurcation
3.declination
• Learning models have an effect on organizational
behavior
• The learning models effect is complicated
PMS effect
1.Individual sales:
Different learning models generate different outcomes
2.Group sales:
Different learning models generally generate same outcomes
3.Behavioral control:
All learning models always generate same outcomes
Effects of learning models are contingent on the PMSs
15

16. Summary
• We developed learning models of agents
behavior under a PMS
• 8 different learning models and 3 typical
PMSs are simulated
• Learning models have an effect on organizational
behavior
• Effect of learning models are complicated and
contingent on the PMSs
The uncertainties of agent s learning model
have an impact on PMS s effects
16

17. References
Deguchi, H. (2009) Dawn of Agent-Based Social Systems Sciences. In
Deguchi, H. and Kijima, K. (Eds.) Manifesto: Agent-based Social
Systems Sciences. Keiso-Shobo (in Japanese).
Goto, Y., S. Takahashi, and Y. Senoue (2009) Analysis of Performance
Measurement System for Knowledge Sharing under
Intraorganizational Competition. Journal of the Japan Society for
Management Information 18(1): 15-49 (in Japanese).
Hales, D., J. Rouchier, and B. Edmonds (2003) Model-to-Model Analysis.
Journal of Artificial Societies and Social Simulations 6(4).
North, M. and C. M. Macal (2007) Managing Business Complexity:
Discovering Strategic Solutions with Agent-Based Modeling and
Simulation. Oxford University Press.
Otomasa, S. (2003) On Use of Performance Measurement Indices in
Japanese Companies. Rokkodai-Ronshu Management Series 49(4):
19-54 (in Japanese).
Richiardi, M., R. Leombruni, N. Saam, and M. Sonnessa (2006) A
Common Protocol for Agent-Based Social Simulation. Journal of
Artificial Societies and Social Simulation 9(1).
17