1. The Death of Polling Version 1 Public 1
Death of
Polling?
The
#ipsosmorilive

2. The Death of Polling Version 1 Public 2
Ben Page, Ipsos MORI
Agenda
Julia Clark, Ipsos USA
David Ahlin, Ipsos Sweden
Nando Pagnoncelli, Ipsos Italy
Darrell Bricker, Ipsos Canada
Q&A

3. The Death of Polling Version 1 Public 3
Ben Page
Ipsos MORI
ben.page@ipsos.com
#ipsosmorilive

4. 4The Death of Polling? Version 1 Public
Sadly not everywhere looks like this …
Average candidate error in US Presidential Elections (INDIV CAND SHARE)
1936
6%
1.5%
2012

5. 5The Death of Polling? Version 1 Public
There’s a global
Ipsos has carried out election
conversation going
on about polling
polling in c30 countries since 2007

6. 6The Death of Polling? Version 1 Public
And pollsters all around the world are
How do we achieve a
representative sample?
How do we predict
turnout accurately?
having to face up to hard questions
How do we make best
use of the increasing range
of methodologies open to us?
How do we ensure our polls
are reported well and
understood among the
media, politicians
and the public?How do we avoid other
biases, such as social
desirability?

7. 7The Death of Polling? Version 1 Public
... although our answers
Size/diversity of population
Turnout levels/compulsory voting
Stable party system or new insurgents?
Generational declines in party loyalty
Societal/cultural context Media expectations vs budgets
and levels of
polling transparency?
Survey researchers or
moving towards big
data modelling?
National polls vs
local/state-level polling?
Traditional face-to-face
methods vs new
or mixed modes?
And in any one country
likely to be good and
bad examples of each!
may be very different!

8. The Death of Polling Version 1 Public 8
UK
experience
The

9. 9The Death of Polling? Version 1 Public
British Polling Council
Selection of samples
(their main explanation)
Representativeness of sample
(key among explanations)
Correction for likelihood
of voting (less important)
Late swing (some signs,
but not that important)
Herding
(unproven)
Inquiry Interim findings

10. 10The Death of Polling? Version 1 Public
Our final poll – all parties less than 2% points away
How would you vote if there were a General Election tomorrow?
from actual – except Labour, overestimated
36%
35%
11%
5%
8%
5%
Ipsos MORI final poll GB final result
Conservative lead = +1 Conservative lead = +6.5
CONSERVATIVE
LABOUR
UKIP
GREEN
LIB DEM
OTHER
37.7%
31.2%
12.9%
3.8%
8.1%
6.4%

11. 11The Death of Polling? Version 1 Public
‘Shy Tories’ not our problem – instead too
How would you vote if there were a General Election tomorrow?
many Labour voters and not enough non-voters
Conservatives
Labour
Non voters
11.3m
12.5m
9.3m
12.2m
20.5m
15.4m
Actual
Implied from final poll

12. The Death of Polling Version 1 Public 12
Ipsos MORI’s view:
We need to take a two-pronged approach
– tackle the problem of more politically
engaged taking part and also making
sure we can detect differential shift
in turnout over-claim between parties

13. The Death of Polling Version 1 Public 13
Ipsos MORI’s view:
Healthy scepticism
not the death of polling

14. 14The Death of Polling? Version 1 Public
We need to improve representation of
No easy answers to this
(especially bearing in
mind budget and time
Constraints)
politically disengaged/non-voters in our samples
Already introduced newspaper weighting (for example to reduce
proportion of broadsheet readers). In four months from
September to December 2015 this:
• Reduced the proportion of claimed likely voters by an average of 3
percentage points a month
• Primarily at the expense of the Labour share (down on average by 3 points,
Conservatives up by 1.75 points)
But will continue other experiments (for example changes in quotas, and so on)

15. 15The Death of Polling? Version 1 Public
Random samples after election though do not appear
to be hugely more representative than our quota ones
Election
result
Final telephone
polls (average)
Final Ipsos MORI
poll
British Social
Attitudes
British Election
Study1
Ipsos MORI post-
GE past vote
(June-July)
Voting
intentions2
Difference
from result
Voting
intentions
Difference
from result
Report of
vote
Difference
from result
Report of
vote
Difference
from result
Report of
vote
Difference
from result
% % % % % %
Con. 37.7 34.5 -3.2 36 -1.7 39.7 +2.0 40.6 +2.9 37.9 +0.2
Labour 31.2 34.3 +3.1 35 +3.8 33.6 +2.4 32.7 +1.5 32.5 +1.3
Other
parties 31.2 31.2 0.0 29 -2.2 26.7 -4.5 26.7 -4.5 29.6 -1.6

16. The Death of Polling Version 1 Public 16
The Death of Polling?
So much more than
a horse race

17. 17The Death of Polling? Version 1 Public
So much more
richness in the
polls to help
us understand
public opinion

18. 18The Death of Polling? Version 1 Public
Not to mention giving voters a
chance to express their views

19. 19The Death of Polling? Version 1 Public
Or using twitter analytics to get live, real-time reactions
to the big events: 239,000 tweets in the 2nd debate
Twitter ‘worm’ – real time
analysis of reaction to
second leader debate
(2,500+ per minute)

20. 20The Death of Polling? Version 1 Public
And using new digital techniques to get closer to voters
7426
posts across
340 forum
topics
c.2000
members
from across
the UK
Over
7500
survey
responses

21. The Death of Polling Version 1 Public 21
Julia Clark
Ipsos USA
julia.clark@ipsos.com
#ipsosmorilive

22. THE THEME IS:

23. More Data Than Ever
0
20
40
60
80
100
120
Jan-60
Dec-60
Nov-61
Oct-62
Sep-63
Aug-64
Jul-65
Jun-66
May-67
Apr-68
Mar-69
Feb-70
Jan-71
Dec-71
Nov-72
Oct-73
Sep-74
Aug-75
Jul-76
Jun-77
May-78
Apr-79
Mar-80
Feb-81
Jan-82
Dec-82
Nov-83
Oct-84
Sep-85
Aug-86
Jul-87
Jun-88
May-89
Apr-90
Mar-91
Feb-92
Jan-93
Dec-93
Nov-94
Oct-95
Sep-96
Aug-97
Jul-98
Jun-99
May-00
Apr-01
Mar-02
Feb-03
Jan-04
Dec-04
Nov-05
Oct-06
Sep-07
Aug-08
Jul-09
Jun-10
May-11
Apr-12
Number of Polls per month - 1960 - 2013Number of Polls per Month – 1960 - 2013
17,058 polls in 2012

24. Current US Political Polling Methodologies
PHONE
RDD
Lists
IVR (Robo)
TRADITIONAL
ONLINE
Probability
Panel Only
Lists
Blended
NONTRADITIONAL
ONLINE
SurveyMonkey
Google

25. Great Variation Among Pollsters
2
5

26. Constantly
Changing
Questionnaire
(daily)
Election
Day ‘Exit
Poll’ of
40,000
Voters
Continual Survey: 11,000/month (24/7/365)
Daily Assessing Events Same-Day (Parsing)
26
State-level
polls (2k)
with rolling
reporting
What Does Ipsos Do?

28. A VERY QUICK
WORD ON THE US
ELECTION…

29. ANTI-ESTABLISHMENT THEME:
But Differential Framing of Problems & Causes
The rich are to blame
Middle Class Economics
Americans First
Restarting American
Exceptionalism
VS.

30. Trump & Sanders are the Response
17%
5% 5% 6%
13%
3% 3%
29%
19%
Trump
Cruz
Carson
Otheroutsiders(Paul,
Fiorina,etc)
Establishment(Bush,
Rubio,etc)
Wouldn'tvote
O'Malley
Clinton
Sanders
Republicans (46%) Democrats (51%)
46% of Americans are
supporting “nontraditional”
candidates

31. Political Fundamentals Speak to a Republican Year
65% 60% 55% 50% 45% 40% 35%
Government Approval Rating
OddsofWinning
Incumbent Party’s Odds of Winning White House
SUCCESSORS
INCUMBENTS
(2012)

32. … AND LOW TURNOUT BENEFITS THE REPUBLICANS TOO
Source: Reuters / Ipsos Poll; Sept-Oct 2014
30% 35% 40% 45% 50% 55% 60% 65%
51
50
49 49
48
47 47 47
44 44
45
46 46
48
Republican Vote
Democratic Vote
2012 TurnoutActual 2014 Turnout
GenericCongressionalVoteShare
Turnout Level (by Likely Voter Model)
2014 Generic Congressional Ballot by Turnout Levels

33. The Death of Polling Version 1 Public 33
Nando Pagnoncelli
Ipsos Italy
nando.pagnoncelli@ipsos.com
#ipsosmorilive

34. 34 © 2015 Ipsos.
THE PREDOMINANCE OF COMPLEXITY
THE ELECTORAL SCENARIO IN ITALY - 2013
169 parties and
movements
2

35. 35 © 2015 Ipsos.
A GAME CHANGER EFFECT
THE EVOLUTION OF THE ELECTOR
ELECTORATES become fluid, reactive to the political offer, they lose
their sense of belonging
2013 is the end of a 20-year period of almost perfect turnover
between centre-left and centre-right, during which
• neither coalition was in office for two turns in a row
• voters switches between the two coalitions were residual
3

36. 36 © 2015 Ipsos.
A GAME CHANGER EFFECT
MOVIMENTO 5 STELLE
in 2013 we experienced a political earthquake or, a tsunami
4

37. 37 © 2015 Ipsos.
A GAME CHANGER EFFECT
MOVIMENTO 5 STELLE
… with victims, politicians …
… and pollster alike
they got a shock
we got it wrong
5

38. 38 © 2015 Ipsos.
EVERYBODY BECAME A POLLING EXPERT, except the experts
THE POLLSTERS UNDER ACCUSATION
Discussions went on for weeks on the limits of polls
 Methodologies
 Sample size
 Coverage
 Bias
 …
were discussed and commented and criticised by anybody and all on TV, in
newspapers, online forum, social networks, workplaces and cafés
6

39. 39 © 2015 Ipsos.
Tecnè Demos Ipsos
diff vs. actual results
RIVOLUZIONE CIVILE 0,9% 0,9% 1,0%
SEL 0,0% 0,4% 0,4%
PD 4,9% 3,7% 4,9%
ALTRI CENTRO SINISTRA -0,4% -0,5% -0,4%
TOTALE CENTROSINISTRA 4,5% 3,6% 4,9%
CON MONTI PER L'ITALIA -0,9% 2,0% 0,4%
UDC 0,8% 1,0% 0,7%
FLI 0,0% 0,2% 0,2%
TOTALE CENTRO -0,1% 3,2% 1,3%
LEGA NORD 1,0% 1,2% -0,1%
PDL -0,3% -1,5% -1,4%
ALTRI CENTRO DESTRA -0,7% -0,7% 0,6%
TOTALE CENTRODESTRA 0,0% -1,0% -0,9%
MOVIMENTO 5 STELLE
BEPPEGRILLO.IT -5,8% -6,6% -5,6%
POLLSTERS WERE INACCURATE
THE POLLSTERS UNDER ACCUSATION
WHAT WENT WRONG WHY IT WENT WRONG
For all the major agencies in Italy
• overestimated the Democratic Party’s
• underestimated the M5S result
• No past voters behaviour on M5S, which is a key
component of the weighting process
• The difficulties of intercepting the potential M5S
voters
• reticence in centre-left supporters to declare their
intention
• last minute swing (25% estimated to have decided 2
days before)
• A high refusal rate, affecting differently the
various groups of respondents (10 contacts to
get a valid interview)
• Respondents lie
7

40. 40 © 2015 Ipsos.
BUT SOME WERE MORE ACCURATE THAN OTHERS
THE POLLSTERS UNDER ACCUSATION
WHAT WENT RIGHT WHY IT WENT RIGHT
IPSOS correctly described the scenario
• The high level of abstention (26,5%)
• The dramatic growth of M5S
• The victory of the centre-left at the
Chamber of Deputies
• The hung Senate
• All the other parties’ results
Because
• A mixed sampling method was used
• Sample sizes were large enough
• We had been polling continuously –
weekly, even daily – for a long time
• We had a wealth of data
• We had lots of analyses on targets,
geographical areas, voters switch
dynamics
8

41. 41 © 2015 Ipsos.
THE PREDOMINANCE OF MADNESS
THE POLLSTERS CHALLENGE
Our stakeholders show a schizophrenic attitude
• one minute, they attack us, because we couldn’t predict the future as
exactly as we are expected to
• the minute after, they ask for data, comments, explanations, because
they need our expertise in interpretation
We would like to see expectations decrease, towards the predictive nature of
our work
but we know it’s inconceivable
9

42. 42 © 2015 Ipsos.
THE FlashForward Effect
THE POLLSTERS CHALLENGE
… and everyone acted consequently.
Polls act as an activator of two powerful, contrasting forces
in electors
• Self-fulfilling prophecies
vs.
• Self-defeating prophecies
“On October 6, the planet blacked out for two minutes and seventeen seconds. The whole world saw the future...”
10

43. The Death of Polling Version 1 Public 43
David Ahlin
Ipsos Sweden
david.ahlin@ipsos.com
#ipsosmorilive

44. 2,9
(2006)
5,7
(2010)
12,9
(2014)
0
10
20
30
Electoral support for the Sweden Democrats 2006-2016
2006 2010 2014

45. 0
30
60
Left-Right Parties Election 2006 Left-Right Parties February 2016
From two political blocks to three – in ten years

46. Immigration most pressing issue and Voter support
for Sweden Democrats
5
8
13
20
20
40
6
7
9
12
14
17
0
20
40
June-2010 June-2014 Aug-2014 Jan-2015 Jun-2015 Jan-2016
Immigration/integration most pressing issue Voter support for the Sweden Democrats

47. Immigration most pressing issue and google searches
for “refugee” in Swedish
8 13
20 20
40
0
20
40
60
80
100
June-2014 Aug-2014 Jan-2015 June 2015 Jan-2016
Number of people say immigration/integration issues are most pressing issue
The number of searches on the word
”Refugee” on Google in Swedish
during the same time period

48. 17.2
23.9
0
20
Average voter support Telephone based methods Average voter support Web based methods
Voter support Sweden Democrats based on
method of measurement Feb 2016

49. 49 © 2015 Ipsos.
The state of polling 2016
SWEDEN
Social effects /
Interviewer effects
Higher share of non-
response in strong
Sweden Democrat
regions
From 45 % – 25 % in
the last six years
SOCIAL EFFECTS AND
NON-RESPONSE
DROPPING
RESPONSE RATES
Less than 50 per cent
with younger Swedes
Not a problem with
seniors
SAMPLE COVERAGE
Telephone calls only
for close friends and
family? (Pew 2015)
Web based and
telephone based give
different answers
Transparency!
Mobile first strategy?
NEW BEHAVIORS
NEW METHODS

50. The Death of Polling Version 1 Public 50
Darrell Bricker
Ipsos Canada
darrell.bricker@ipsos.com
#ipsosmorilive

51. 51 © 2015 Ipsos.
Ipsos Final Poll vs. Actual Election Results—National
ELECTION DEBRIEF
Ipsos Final Poll Election Results
38% 40%
31% 32%
22% 20%
4% 5%
4% 4%
Base: Final Call Poll (Decided Voters, Leaners Included n=2,226; Online n=1,328; CATI n=898)
Weighting: 50/50 telephone/online, education, region, thumb

52. 52 © 2015 Ipsos.
Online/Telephone Research (Call Poll)
ELECTION DEBRIEF
2 waves consisting of minimum sample sizes of:
1/3
1/3
1/3 Ampario
Allocated
Reallocated
1,000 Telephone Completes

1,000 Online OMNI

roughly: 40% cellphone
3-day
field
window

 Sample blended together to create 4 quadrants:
1) telephone landline; 2) telephone cell; 3) online panel, and 4) online non-panel
- Result somewhere in the middle…
and
KEY BENEFITS:
 Telephone poll to act as a gut check on our online  greater confidence in our figures
 Triangulation among various sample sources mitigates bias of any one methodology
 Allows us to formulate a weighting scheme to mitigate panel bias

53. 53 © 2015 Ipsos.
Call Poll Sample Breakdown
ELECTION DEBRIEF
Base: Final Call Poll (Total n=2,503; Online n=1,502; CATI n=1,001)
16%
24%
13%
47%
CATI Mobile
CATI Landline
Online Ampario
Online Panel 60%
Online
40%
CATI

54. 54 © 2015 Ipsos.
What We Learned
54 © 2015 Ipsos.

55. 55 © 2015 Ipsos.
Lessons Learned and Next Steps
ELECTION DEBRIEF
Mixed mode methods show promise.
Ipsos will continue to explore and share what we learn.
Offset biases
associated
with specific
modes of data
collection with
minimum
weighting.

56. 56 © 2015 Ipsos.
Lessons Learned and Next Steps
ELECTION DEBRIEF
Online and offline methods consistently under and over-estimate specific
demographic groups. Also, over and under-estimate specific voter mindsets.
Evidence building that:
…online overestimates progressive
voters
…telephone overestimates
conservative voters.

57. 57 © 2015 Ipsos.
Lessons Learned and Next Steps
ELECTION DEBRIEF
LIKELY VOTER ESTIMATES,
while logically attractive,
work better in theory than
in practice.
What’s missing? How do we
make them better?

58. 58 © 2015 Ipsos.
Likely Voter Model
A REGRESSION MODEL BASED ON…
PAST BEHAVIOUR
INTENT TO VOTE
INTEREST IN ELECTION

59. 59 © 2015 Ipsos.
Final Weighted Call Poll by Likely Voter
ELECTION DEBRIEF
38%
37%
37%
37%
37%
31%
32%
32%
33%
32%
22%
22%
22%
22%
22%
4%
2%
2%
2%
3%
4%
5%
5%
5%
4%
Final
55% Turnout
60% Turnout
65% Turnout
70% Turnout
Liberals Conservative NDP Green Bloc
Base: Final Call Poll (Online Total Decided Voters n=1,328; I-Say Allocated n=427; I-Say Re-Allocated n=624; Ampario n=277)
Weighting: 50/50 telephone/online, education, region, thumb

60. The Death of Polling Version 1 Public 60
Thanks for listening
Q&A
#ipsosmorilive