2. Agenda
• Introduc;ons
• What makes Enterprise Markets work
• Why markets as a medium for predic;ng the future
• Enterprise Do’s and Don’ts
• Discussion
3. What Makes Markets Work
Buy Low, Sell High
Will price
rise or fall?
“Pays $1 if
Obama wins” sell
E[ price change | ?? ]
buy
price
sell Lots of ?? get tried,
buy price includes all!
5. What Makes Markets Work
Beats Alterna%ves
• Vs. Public Opinion
– I.E.M. beat presiden;al elec;on polls 709/964 (Berg et al ‘08)
– Re NFL, beat ave., rank 7 vs. 39 of 1947 (Pennock et al ’04)
• Vs. Public Experts
– Racetrack odds beat weighed track experts (Figlewski ‘79)
• If anything, track odds weigh experts too much!
– OJ futures improve weather forecast (Roll ‘84)
– Stocks beat Challenger panel (Maloney & Mulherin ‘03)
– Gas demand markets beat experts (Spencer ‘04)
– Econ stat markets beat experts 2/3 (Wolfers & Zitzewitz ‘04)
• Vs. Private Experts
– HP market beat official forecast 6/8 (Ploh ‘00)
– Eli Lily markets beat official 6/9 (Servan‐Schreiber ’05)
– Microsoi project markets beat managers (Proebs;ng ’05)
6. What Makes Markets Work
Example: Iowa Electronic Market
Item 1988 1992 1996 2000 2004 All
# big 59 151 157 229 368 964
polls
Poll 25 43 21 56 110 255
“wins”
Market 34 108 136 173 258 709
“wins”
% 58% 72% 87% 76% 70% 74%
Market
P-value 0.148 0.000 0.000 0.000 0.000 0.000
“Prediction Market Accuracy in the Long Run”
Joyce Berg, Forrest Nelson and Thomas Rietz, Jan. 2008.
7. Why Markets as a Medium for Forecas%ng
Advantages
Incen(ves
• Numerically precise Self‐Selec(on
• Consistent across many issues Correct Biases
• Frequently updated
• Hard to manipulate
• Need not say who how expert when
– Issue is not experts vs. amateurs
• At least as accurate as alterna;ves
11. Why Markets as Medium for Forecas%ng
Factors Might Influence Sales
E[Sales|Factor]
P[Factor]
• Economy recovers fast?
• Compe;tors introduce new version?
• We do big promo;on?
• We lower prices? They lower prices?
• We add distribu;on channel?
• We add feature F? They add feature F?
• Our defect rate very low?
12. Why Markets as Medium for Forecas%ng
Issues
Input: Contribu%ons Output: Forecasts, Scores
• What info can express? • What ques(ons can ask?
• How account for costs? • How account for value?
• Who let in where? • Use or validate system?
• Enough Incen;ves • Should adjust outputs?
– T‐shirts enough?
– Zero‐sum scoring? • Who let see outputs?
• Limit Costs • Sabotage & manipula;on
– Awkward Interface • Legal, P.R. risks?
– Wait for offer accept
– Retribu;on
13. Why Markets as Medium for Forecas%ng
Requirements
Use: • Ques%ons really want answered now
– Will eventually know answer (or parts)
– Suspect not geong frank info via usual channels
– Don’t mind par;cipants knowing best es;mates
• People who have or can get key info
– Their ;me is the main cost
– Lihle penalty for invite many don’t know
• Incen%ves to en;ce careful par;cipa;on
– Money, ahen;on, influence can legally offer
– Valued when ques;ons answered
Validate: • Many fast ques;ons, status quo es;mates
14. Enterprise Do’s and Don’ts of Enterprise
Markets
Customer
• Do
– Integrate into Enterprise processes
– Nurture execu;ve sponsorship
– Make accessible to all
– Customize to your business
– Make it Part of your Value Prop
14
15. Integrate Into Enterprise Processes Customer
Challenge Ac;on Result
• Facing too many • Deployed internal • Increased accuracy
unknowns, leaders market, integrated from market has
were planning for all into exis;ng streamlined short
possible futures Enterprise and mid‐term
processes and planning
• Costly and ;me‐
dashboards
consuming • An;cipated
• Appropriate ;me improvements in
horizons matched metrics, e.g. cash
Enterprise management,
deadlines balance sheet
15
16. Nurture Execu%ve Sponsorship Customer
Challenge Ac;on Result
• Market introduced • Sought exec air • Balanced top‐down
by self‐ini;a;ng cover, not mandate endorsement with
team, not C‐suite • Targeted key bohoms‐up culture
• Fit entrepreneurial leaders as early • Grassroots growth
culture adopters more sustainable
• S;ll needed to drive • Champion is one of • Consulta;ve
awareness, top users rela;onship with
par;cipa;on, and • Opened market to key leaders paves
execu;ve usage everyone way for followers
16
17. Make It Accessible To All Customer
Challenge Ac;on Result
• Accelerate the • Incorporated • Improved overall
process of market into ;me to market,
priori;zing ideas broader innova;on speed of idea
and product plaqorm ac;on‐ by 100 days
features • Deployed market • 5‐10X ROI
• Engage more globally • Tripled
employees in • Feed ideas to every collabora;on and
innova;on business unit cut duplicate ideas
in half
17
18. Customize It to Your Business Customer
Challenge Ac;on Result
• Looking for new • Market well‐named • Made it familiar
ways to engage and branded and instantly
their customer base • Seamless accessible to target
• Seeking integra;on with group
differen;a;on from company website • Drove par;cipa;on
the significant • Develop custom and accelerated
compe;;on in their interface to match engagement
market user community • Differen;ated from
compe;;on
18
19. Make It Part of Your Value Prop Client
Challenge Ac;on Result
• Customers • Recognized market • Provide clients with
represent broad as amplifier of beher data via
array of industry exis;ng informa;on public market
stakeholders and service • Enable client
• Universal client offerings specific experience
need: predic;ng • Integrated market • Provide superior
behaviors into all aspects of services and ;mely
• Tradi;onal research how they create insights, create
& forecas;ng value and conduct compe;;ve
insufficient given business advantage
industry changes
19
20. Enterprise Do’s and Don’ts of Enterprise
Markets
Customer
• Don’t
– Run a 30 day pilot
– Use a similar group of par;cipants
– Run a market without promo;ng it
– Assume leaders will act on insights
20
