Research presentation at INSEAD, June 20, 2008.

1. Experimental Results
of Online Information
Aggregation Markets
for Sales Forecasting
Eric van Heck
Rotterdam School of Management
Erasmus University
evanheck@rsm.nl
www.rsm.nl/evanheck
INSEAD Presentation
Fontainebleau, 20 June 2008
© Eric van Heck, 2008.

2. Menu
1. What are information aggregation markets (or also
called prediction markets)?
2. State-of-the-art in practice
IOWA Political Markets
Hollywood Stock Exchange
3. State-of-the-art in theory
Project 1 with Mathijs van der Vlis
4. State-of-the-art in theory and practice
Project 2 with Annie Yang et al. and anonymous
company
5. Conclusions

3. Introduction
How many passengers can travel
with the Silja Symphony?
Helsinki – Stockholm v.v.

4. Aggregation and Averaging
Bo: 2,700
Heli: 2,640
Virpi: 3,005
Ari: 3,050
Helsinki – Stockholm v.v.
Pekka: 2,502
Mika: 2,600
Jyrki: 2,845
Szymon: 2,777
Ralph: 2,799
Esko: 3,592

5. Aggregation and Averaging
Bo: 2,700
Heli: 2,640
Virpi: 3,005
Ari: 3,050
Helsinki – Stockholm v.v.
Pekka: 2,502
Mika: 2,600
Jyrki: 2,845
Szymon: 2,777
Ralph: 2,799
Esko: 3,592
Average: 2,851

6. Aggregation and Averaging
Bo: 2,700
Heli: 2,640
Virpi: 3,005
Ari: 3,050
Helsinki – Stockholm v.v.
Pekka: 2,502
Mika: 2,600
Jyrki: 2,845
Szymon: 2,777
Ralph: 2,799
Esko: 3,592
Average: 2,851
Correct answer: 2,852

7. Aggregation and Averaging
Bo: 2,700
Heli: 2,640
Virpi: 3,005
Ari: 3,050
Helsinki – Stockholm v.v.
Pekka: 2,502
Mika: 2,600
Jyrki: 2,845
Szymon: 2,777
Ralph: 2,799
Esko: 3,592
Average: 2,851
Correct answer: 2,852
The average is a very good predictor – wisdom of crowds.
Jyrki is closest to the correct answer!

8. What are information markets?
1. A group of people that buy and sell stocks.
2. Stocks represent the potential outcome of the subject to be
forecasted (number of Silja passengers, future demand of
mobile telephones, winner soccer game, etc).
3. Market mechanism is a double auction.
4. Market price of a particular stock represent the probability that
that potential outcome will happen – for example: stock Italy (in
the game Italy – NL) is 0,80 cent (range 0 – 100 cents) =
probability that Italy wins is 80%.
5. The market aggregates information by the aggregation of the
individual beliefs of the players.

9. State-of-the-art in practice
Some applications in practice:
IOWA Political Markets
Hollywood Stock Exchange
Internal Information Markets for example by HP,
Google, and external Information Markets such as
NewsFutures, Foresight Exchange.

10. IOWA Political Markets

14. Lessons Learned (Berg et al, 1996, 2000)
• IOWA political markets perform better than
polling results
• Presidential election markets perform
better than (lower profile) congressional,
state, or local elections
• Markets with more volume near the
election perform better
• Markets with fewer contracts (i.e. fewer
candidates or parties) predict better

15. Hollywood Stock Exchange

16. Trading in MovieStocks

17. Trading in StarBonds

18. Lessons Learned
• Prices of securities in Oscar, Emmy, and
Grammy awards correlate well with actual
award outcome frequencies, and prices of
movie stocks accurately predict real box
office results (Pennock, 2001).

19. Hype Cycle for Emerging Technologies 2006

20. State-of-the-art in theory
Market Characteristics
Market Efficiency
Incentive Mechanism
Transaction Costs Trader Anonymity
Market Information/Signals
Prediction Metric (last trading price, avg price)
Trading Mechanism
Contract Type (binary, spread, index) Liquidity/Market Size Selling short/portfolios
Information Cascades/Market Bubbles
Frequency of information update
Trader Characteristics
TraderType
Biases/Bounded Rationality Trader Demographics
Characteristics of the to-
be-predicted event Information Source Homogeneity/Heterogeneity
Inherent Predictability Trading Experience/Knowledge Wealth Risk Attitude
Aggregate Certainty Private Information
Cheating/Collaboration/Manipulation
Information Availability/Costs
Time Scope

21. Main Theories
• Mechanism Design Theory (Hayek 1945)
Markets are an appropriate mechanism for the purpose of efficient
information aggregation and decision making due to the incentives for
information discovery.
• Double Auction Theory (Plott and Sunder, 1982, 1988)
Prediction markets have the ability to aggregate dispersed private
information held by individuals as the double auction mechanism has the
ability to disseminate private information among traders.
• Rational Expectation Theory (Lucas 1972, Grossman 1981)
The price observed in a prediction market is a sufficient statistic for all
information available to traders
• The Wisdom of Crowds (Surowiecki 2004)
Small and large groups of people seem to do better at decision making than
individuals.

22. Mechanism Design Theory

23. Project 1 - with Mathijs van der Vlis :
What is the impact of the number of traders,
the distribution of wisdom, and monetary incentives
to the outcome of information markets?

24. Hypotheses
1. Number of traders (Surowiecki, 2004)
More traders will increase the level of aggregation and the level
of prediction accuracy
2. Distribution of wisdom (Anderson and Holt, 1997; Hansen,
Smith, and Strober, 2001; Hanson and Oprea, 2004)
Uneven distribution among traders will increase the level of
aggregation and the level of prediction accuracy
3. Monetary incentives (Servan-Schreiber, Wolfers, Pennock, and
Galebach, 2004)
Monetary incentives will not increase the level of aggregation
and the level of prediction accuracy

25. 128 Laboratory Experiments to forecast future demand
of mobile telephones

26. Results Experiments (N = 128)

27. Hypotheses
1. Number of traders (Surowiecki, 2004)
More traders will increase the level of aggregation and level of
No prediction accuracy
Yes
2. Distribution of wisdom (Anderson and Holt, 1997; Hansen,
Smith, and Strober, 2001; Hanson and Oprea, 2004)
Uneven distribution among traders will increase the level of
Yes aggregation and the level of prediction accuracy
No
3. Monetary incentives (Servan-Schreiber, Wolfers, Pennock, and
Galebach, 2004)
Monetary incentives will not increase the level of aggregation
Yes and the level of prediction accuracy
Yes

28. Lessons Learned
• Results indicate that even in the presence of a small
number of traders there tends to be aggregation, while
only in the presence of a large number of traders are
accurate predictions generated.
• When wisdom is unequally distributed there is
aggregation (wise people lead markets), yet the markets
do not produce more accurate predictions (wise people
can potentially mislead markets).
• Monetary incentives impact neither the level of
aggregation nor the level of accuracy.

29. State-of-the-art in theory and practice
Some applications:
Internal Information Markets for example at a
financial company
Helsinki – Stockholm v.v.

30. Project 2 - with Annie Yang, Maarten Colijn, Willem
Verbeke, Mathijs van der Vlis and anonymous
company
What is the performance of information markets in
forecasting the overall sales of a product
over several regions in the Netherlands?

31. Hypotheses
• Market Size – Number of Traders (Surowiecki 2004, Hansen
2003)
H1a: A prediction market with more traders is likely to
aggregate sooner and more significantly.
H1b: A prediction market with more traders is likely to forecast
more accurately.
• Monetary Incentives (Servan-Schreiber et al. 2004)
H2: An offer of monetary incentives does not affect the
activeness of traders’ participation in a prediction market.
• Time Horizon (Berg et al. 2003)
H3: A prediction market forecasts more accurately in a short
run than in a long run.

32. Trading Web Page

33. 1st Prediction Market 2nd Prediction Market
Subject to be predicted Annual sales of a financial product Periodical sales of a financial product
Contracts Spread contracts (in million euro) Spread contracts (in million euro)
Description of Prediction Markets
Description of traders Regional sales managers Regional sales managers
Number of stocks 10 9
Number of traders 34 34
Number of active traders 31 18
Number of very active
8 3
traders
Total number of bids (incl.
604 461
demand and sell)
Total number of completed
368 275
bids (buy and sell)
Time of markets 24 hrs / 7 days 24 hrs / 7 days
Market duration 12 calendar days (Feb 2007) 12 calendar days (June 2007)

34. Aggregation and Forecasting Results
Historical Stock Prices in 1st Prediction Market
80
110-120 Actual sales
70 121-130 i.e. 133
60 131-140
Stock Price (in point)
141-150 Market forecast
50 151-160
40 161-170
Top-down
171-180
30 forecast
181-190
20 191-200
201-210
10
0 Trading Day
Trade
1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th

35. Aggregation and Forecasting Results
Historical Stock Prices in 2nd Prediction Market
120 Market forecast
19 - 22
100 22 - 25
Top-down
25 - 28
forecast
Stock Price (in point)
80 28 - 31
31 - 34 Actual sales
60 34 - 37 i.e. 28.6
37 - 40
40 40 - 43
43 - 46
20
0 Trading Day
Trade Day
1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th

36. Forecasting Accuracy
1st Prediction Market 2nd Prediction Market
% error % error
Actual results 133 28.6
Prediction market forecast 141-150 +6% - 13% 19-22 -23% - 34%
Top-down forecast 150-160 +13% - 20% 27 -6%

37. Hypotheses
• Market Size – Number of Traders (Surowiecki 2004, Hansen
2003)
H1a: A prediction market with more traders is likely to
No aggregate sooner and more significantly.
H1b: A prediction market with more traders is likely to forecast
Yes more accurately.
• Monetary Incentives (Servan-Schreiber et al. 2004)
H2: An offer of monetary incentives does not affect the
Yes activeness of traders’ participation in a prediction market.
• Time Horizon (Berg et al. 2003)
H3: A prediction market forecasts more accurately in a short
No run than in a long run.

38. Lessons Learned
1. Market size, in terms of the number of traders, does not
necessarily influence market aggregations but the
accuracy of predictions. A thicker market is more likely
to forecast accurately.
2. Monetary incentives are not effective to motivate traders
to trade in internal prediction markets – time for trading
is a constraint.
3. Markets predict more accurately in a long run than in a
short run. Interesting because the impact of the
worldwide mortgage crises was predicted very well
4. Traders are sensitive to the prices of contracts, learning
from signals and constantly updating their beliefs.
However, this yields that traders could be easily misled,
particularly in a thin market.

39. Conclusions
1. “Information Aggregation” is a Key Critical
Component for Firms - online markets can
improve the information aggregation capability of
a firm!
Helsinki – Stockholm v.v.
2. Several issues need to be solved for example:
details of the market design
incentive structure of players
3. Do you want to know more: please join!