Anúncio

Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612

CEO em Magellan Strategies
20 de Jun de 2014
Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612
Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612
Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612
Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612
Anúncio
Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612
Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612
Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612
Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612
Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612
Próximos SlideShares
RedC Poll, Richard Colwell- Autumn Seminar 2010RedC Poll, Richard Colwell- Autumn Seminar 2010
Carregando em ... 3
1 de 9
Anúncio

Mais conteúdo relacionado

Apresentações para você(20)

Similar a Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612(20)

Anúncio

Mais de Magellan Strategies(20)

Último(20)

Anúncio

Magellan Strategies 2012 Internal Survey Research Summary Memorandum 120612

  1. December 6, 2012 MEMORANDUM TO: INTERESTED PARTIES FR: DAVID FLAHERTY, CEO MAGELLAN STRATEGIES RE: 2012 POST-ELECTION INTERNAL SURVEY ANALYSIS This memorandum is a review of Magellan Strategies' internal surveys from the 2012 election cycle. This document shares insights to adjustments that we made during the cycle to improve the accuracy of our surveys. The intent of this memorandum is to provide political professionals and observers with information about how Magellan Strategies conducted its survey research in the 2012 Presidential election.   THE CHALLENGES OF SURVEY RESEARCH IN 2012 Measuring voter opinion and candidate support was a challenge for many researchers in the 2012 election cycle because younger and minority voters made up a larger than expected portion of the overall voting population. To get it right, researchers had to make correct assumptions about what the final voter turnout demographics would be, and make sure their phone sample coverage of the final voting population was sufficient. If one or both of these assumptions is incorrect, then the survey will be inaccurate. To understand what we are talking about when discussing demographics, we are referring to the percentage of a survey's total interviews broken out by gender, age or age group, race and geographic region (county, media market, etc.). Another important demographic that is used by some researchers, but not all, is party identification. SURVEY SAMPLE COVERAGE CHALLENGES Survey sample "coverage" refers to the percentage of the voting population that can be reached by a phone call. For the survey to be accurate, minimum "coverage" sample requirements need be met to ensure enough voters have an opportunity to participate in the survey. For example, if I am conducting a survey in a legislative district that contains 100,000 voters, I want to be sure the sample used sufficiently covers the total population. If the sample covers 85% of the population, that would be considered acceptable to most researchers.
  2. Page 2 of 9   Magellan Strategies | 1685 Boxelder Street, Suite 300 | Louisville, CO 80027 www.MagellanStrategies.com | (303) 861-8585 But if the 15% of the voting population that I am unable to contact by phone is predominantly younger or minority voters and those subgroups are contributing 30% to the overall vote, then the survey will be inaccurate. The decline of the household or "landline" telephone created additional challenges for researchers in the 2012 election cycle. To maintain a sufficient level of sample coverage, most researchers will add cell phone sample. A difficult decision that a researcher needs to make if they are using landline and cell phone sample is what the percentage each set of sample should be. MAGELLAN STRATEGIES SURVEY RESEARCH LESSONS LEARNED During the 2012 election cycle, Magellan Strategies regularly fielded surveys in all Presidential battleground states starting in early May. All of our surveys at the start of the cycle were fielded using an automated voice recorded method of interviewing. Automated surveys have the benefit of a lower cost and a larger number of interviews, and political professionals like the ability to “drill down” among voter subgroups with more confidence than a survey with fewer interviews. In early June we had concerns that the automated interviewing approach was not accurately measuring minority and younger voter populations. This made us skeptical of the results, so we began including cell phone and live landline interviews of minority voters in almost all of our surveys. MAGELLAN STRATEGIES APPROACH TO SURVEY DEMOGRAPHICS FOR 2012 The 2008 Presidential election produced a much larger than expected voter turnout among younger voters, aged 18 to 34 in states across the country. Heading into the 2012 Presidential election, Magellan Strategies, like many other political researchers believed it was unlikely that younger voters aged 18 to 34 would turnout in the same percentages as they did in the 2008 election. However, we believed these younger voters would vote in higher numbers than they did in the 2004 election. Turnout demographics from the 2010 election among younger voters were higher than they were in the 2006 election. Taking this into account, we set our 2012 turnout demographics closer to 2008 than 2004.
  3. Page 3 of 9   Magellan Strategies | 1685 Boxelder Street, Suite 300 | Louisville, CO 80027 www.MagellanStrategies.com | (303) 861-8585 MAGELLAN STRATEGIES SURVEY RESULTS BY OBAMA WIN MARGIN Two weeks before the 2012 election, we fielded surveys in the battleground states that merged automated voice recorded, cell phone, and live landline interviews of likely voters. In the final week before the election, we fielded only automated voice recorded surveys without any cell phone or landline live interviews. The following tables show how well both sets of surveys performed against the actual results. It should be noted the Presidential ballot test for each survey included the qualified third party candidates in each state. To measure the accuracy of the surveys, the following two tables show the actual vote percentage "Obama final win margin" and the survey "Magellan Obama win margin.” Oct. 23rd - 24th ALC* Survey Obama Win Margin Comparison State Obama Actual Final % Romney Actual Final % Obama Final Win Margin % Magellan ALC* Survey Obama Win Margin % Obama Final Win Margin - Magellan Obama Win Margin Difference Florida 50.0% 49.1% 0.9% 0.8% 0.1% Pennsylvania 52.1% 46.7% 5.4% 4.9% 0.5% Colorado 51.5% 46.1% 5.4% 4.5% 0.9% Michigan 54.2% 44.7% 9.5% 10.8% 1.3% Wisconsin 52.8% 45.9% 6.9% 4.3% 2.6% Ohio 50.1% 48.1% 2.0% 4.8% 2.8% Nevada 52.3% 45.6% 6.7% 3.6% 3.1% Virginia 51.1% 47.2% 3.9% 0.8% 3.1% North Carolina 48.3% 50.3% -2.0% 1.9% 3.9% New Hampshire 52.0% 46.4% 5.6% 1.9% 3.7% Iowa 51.9% 46.1% 5.8% -0.1% 5.9% *ALC refers to the survey interview method that included automated voice recorded, live landline, and live cell phone interviews. Oct. 31st - Nov. 1st Automated Only Survey Obama Win Margin Comparison State Obama Final % Romney Final % Obama Final Win Margin Magellan Auto Survey Obama Win Margin Obama Final Win Margin Magellan Obama Win Margin Difference Colorado 51.5% 46.1% 5.4% 5.6% 0.2% Ohio 50.1% 48.1% 2.0% 1.3% 0.7% North Carolina 48.3% 50.3% -2.0% -3.2% 1.2% New Hampshire 52.0% 46.4% 5.6% 3.5% 2.1% Florida 50.0% 49.1% 0.9% -1.3% 2.2% Iowa 51.9% 46.1% 5.8% 3.1% 2.7% Nevada 52.3% 45.6% 6.7% 2.4% 4.3% Wisconsin 52.8% 45.9% 6.9% 0.5% 6.4% Virginia 51.1% 47.2% 3.9% -3.0% 6.9%
  4. Page 4 of 9   Magellan Strategies | 1685 Boxelder Street, Suite 300 | Louisville, CO 80027 www.MagellanStrategies.com | (303) 861-8585 SURVEY ACCURACY MEASURING OBAMA AND ROMNEY SUPPORT The next set of tables looks at how well each survey measured support for Barack Obama and Mitt Romney. As the tables indicate, the surveys were more accurate in measuring Mitt Romney's support than Barack Obama's. We believe the “under measurement” of Barack Obama’s support is due to the challenges of getting younger and minority voters to participate in a survey. It is also clear surveys that included cell phone and landline interviews of minority voters did a better job than just automated interviews alone. We believe measuring Mitt Romney’s strength was less challenging the measuring Barack Obama’s because Romney supporters were older and less racially diverse. Magellan Strategies Survey Obama Support Comparison Table State Obama Final % Magellan 10/23 ALC* Survey Obama % Magellan 10/23 ALC Survey Difference Magellan 10/31 Auto Survey Obama % Magellan 10/31 Auto Survey Difference Ohio 50.1% 50.1% 0.0% 48.6% 1.5% North Carolina 48.3% 48.5% 0.2% 46.9% 1.4% Michigan 54.2% 53.3% 0.9% N/A** N/A** Florida 50.0% 47.8% 2.2% 47.7% 2.3% Pennsylvania 52.1% 49.9% 2.2% N/A** N/A** Colorado 51.5% 48.4% 3.1% 51.1% 0.4% New Hampshire 52.0% 48.9% 3.1% 50.2% 1.8% Wisconsin 52.8% 49.7% 3.1% 48.9% 3.9% Nevada 52.3% 49.1% 3.2% 47.4% 4.9% Virginia 51.1% 47.9% 3.2% 46.4% 4.7% Iowa 51.9% 46.5% 5.4% 49.4% 2.5% *ALC refers to the survey interview method that included automated voice recorded, live landline, and live cell phone interviews. **“N/A” indicates a survey was not fielded in the final week prior to the election. We did not think Michigan was competitive, and for Pennsylvania we were concerned homes without power from Sandy would significantly impact survey response rates. Magellan Strategies Survey Romney Support Comparison Table State Romney Final % Magellan 10/23 ALC Survey Romney % Magellan 10/23 ALC Survey Difference Magellan 10/31 Auto Survey Romney % Magellan 10/31 Auto Survey Difference Nevada 45.6% 45.5% 0.1% 45.0% 0.6% Virginia 47.2% 47.1% 0.1% 51.0% 3.8% Iowa 46.1% 46.6% 0.5% 46.3% 0.2% Wisconsin 45.9% 45.4% 0.5% 48.4% 2.5% New Hampshire 46.4% 47.0% 0.6% 46.7% 0.3% Pennsylvania 46.7% 45.0% 1.7% N/A N/A Florida 49.1% 47.0% 2.1% 49.0% 0.1% Colorado 46.1% 43.9% 2.2% 45.5% 0.6% Michigan 44.7% 42.5% 2.2% N/A N/A Ohio 48.1% 45.3% 2.8% 47.3% 0.8% North Carolina 50.3% 46.6% 3.7% 50.1% 0.2%
  5. Page 5 of 9   Magellan Strategies | 1685 Boxelder Street, Suite 300 | Louisville, CO 80027 www.MagellanStrategies.com | (303) 861-8585 MAGELLAN SURVEY EXIT POLL COMPARISON The survey tables that follow compare the Magellan survey results to the 2012 exit polling data by voter subgroup. To understand how to read the data in the tables, the exit polling results for each candidate are in the "Exit" row and the Magellan survey candidate results are in the "Mag" row. The "Difference" row indicates the difference between the exit polling results and the Magellan survey results for the voter subgroup. The comparison of the two different survey interview methods shows the challenges of an automated survey to accurately measure minority voter opinion. Colorado Oct. 23rd - 24th Automated, Landline and Cellphone Survey Total*^ Men Wom White Wom Ind Hisp 65+ Difference 3.1% 3% 2% 1% 1% 11% 9% Obama (Exit Poll) 51.5% 51% 51% 46% 45% 75% 42% Obama (Mag) 48.4% 48% 49% 45% 44% 64% 51% Difference 2.2% 6% 1% 1% 11% 7% 10% Romney (Exit Poll) 46.1% 46% 48% 53% 49% 23% 57% Romney (Mag) 43.9% 40% 47% 52% 38% 30% 47% *The Total (Exit) column shows the final candidate vote percentage as reported on November 29th. ^ The actual Obama win margin was 5.4%, the survey Obama win margin was 4.5% with 3.6% of respondents undecided. The Magellan survey interviewed 746n likely voters, for a margin of error of +/- 3.58%. The exit polling data interviewed 1,029 voters, for a margin of error of +/- 3.05%. Colorado Oct. 31st - Nov. 1st Automated Only Survey vs. Exit Polls Total*^ Men Wom White Wom Ind Hisp 65+ Difference 0.4% 4% 4% 10% 10% 24% 8% Obama (Exit) 51.5% 51% 51% 46% 45% 74% 42% Obama (Mag) 51.1% 47% 55% 56% 55% 52% 50% Difference 0.6% 2% 4% 9% 13% 19% 8% Romney (Exit) 46.1% 46% 48% 53% 49% 23% 57% Romney (Mag) 45.5% 48% 44% 44% 36% 42% 49% *The Total (Exit) column shows the final candidate vote percentages as reported on November 29th. ^ The actual Obama win margin was 5.4%, the survey Obama win margin was 5.6%, with 0.6% of respondents undecided. The Magellan survey interviewed 1,033n likely voters, for a margin of error of +/- 3.05%. The exit polling data interviewed 1,029 voters, for a margin of error of +/- 3.05%.
  6. Page 6 of 9   Magellan Strategies | 1685 Boxelder Street, Suite 300 | Louisville, CO 80027 www.MagellanStrategies.com | (303) 861-8585 Florida Oct. 23rd - 24th Automated, Landline and Cellphone Survey Total*^ Men Wom White Wom Ind Hisp Black 65+ Difference 2.2% 0% 3% 0% 0% 6% 9% 3% Obama (Exit) 50.0% 46% 53% 41% 50% 60% 95% 41% Obama (Mag) 47.8% 46% 50% 41% 50% 54% 86% 38% Difference 2.1% 4% 0% 2% 9% 0% 5% 1% Romney (Exit) 49.1% 52% 46% 58% 47% 39% 5% 58% Romney (Mag) 47.0% 48% 46% 56% 38% 39% 10% 57% *The Total (Exit) column shows the final candidate vote percentage as reported on November 29th. ^ The actual Obama win margin was 0.9%, the survey Obama win margin was 0.8% with 3.8% of respondents undecided. The Magellan survey interviewed 774n likely voters, for a margin of error of +/- 3.52%. The exit polling data interviewed 4,225 voters, for a margin of error of +/- 1.51%. Florida Oct. 31st - Nov. 1st Automated Only Survey vs. Exit Polls Total*^ Men Wom White Wom Ind Hisp Black 65+ Difference 2.3% 2% 5% 3% 6% 15% 25% 6% Obama (Exit) 50.0% 46% 53% 41% 50% 60% 95% 41% Obama (Mag) 47.7% 48% 48% 44% 44% 45% 70% 47% Difference 0.1% 1% 2% 3% 6% 13% 20% 6% Romney (Exit) 49.1% 52% 46% 58% 47% 39% 5% 58% Romney (Mag) 49.0% 51% 48% 55% 41% 52% 25% 52% *The Total (Exit) column shows the final candidate vote percentage as reported on November 29th. ^ The actual Obama win margin was 0.9%, the survey Romney win margin was 1.3% with 0.4% of respondents undecided. The Magellan survey interviewed 888n likely voters, for a margin of error of +/- 3.29%. The exit polling data interviewed 4,225 voters, for a margin of error of +/- 1.51%. The Florida October 24th survey that included cellphone interviews and landline interviews of African American and Hispanic voters predicted the overall outcome very well with a win margin of 0.1%, but the final automated survey missed the mark, predicting a close Romney win. We think this is a good example of how an automated interview approach can have coverage and penetration problems among younger and minority voters, thereby under-measuring Democrat candidate strength.
  7. Page 7 of 9   Magellan Strategies | 1685 Boxelder Street, Suite 300 | Louisville, CO 80027 www.MagellanStrategies.com | (303) 861-8585 Ohio Oct 24th - 25th Automated, Landline and Cellphone Survey vs. Exit Polls Total*^ Men Wom White Wom Ind Black 65+ Difference 0% 5% 4% 0% 1% 11% 4% Obama (Exit) 50.1% 45% 55% 46% 43% 96% 44% Obama (Mag) 50.1% 50% 51% 46% 44% 85% 48% Difference 2.8% 6% 1% 2% 6% 8% 8% Romney (Exit) 48.1% 52% 44% 53% 53% 4% 55% Romney (Mag) 45.3% 46% 45% 51% 47% 12% 47% *The Total (Exit) column shows the final candidate vote percentage as reported on November 29th. ^ The actual Obama win margin was 2.0%, the survey Obama win margin was 1.6% with 4.4% of respondents undecided. The Magellan survey interviewed 713n likely voters, for a margin of error of +/- 3.67%. The exit polling data interviewed 3,992 voters, for a margin of error of +/- 1.55%. Ohio Oct. 31st – Nov. 1st Automated Only Survey vs. Exit Polls Total*^ Men Wom White Wom Ind Black 65+ Difference 1.5% 2% 5% 3% 7% 28% 5% Obama (Exit) 50.1% 45% 55% 46% 43% 96% 44% Obama (Mag) 48.6% 47% 50% 49% 36% 68% 49% Difference 0.8% 3% 2% 5% 3% 23% 7% Romney (Exit) 48.1% 52% 44% 53% 53% 4% 55% Romney (Mag) 47.3% 49% 46% 48% 56% 27% 48% *The Total (Exit) column shows the final candidate vote percentage as reported on November 29th. ^ The actual Obama win margin was 2.0%, the survey Obama win margin was 2.5% with 4.5% of respondents undecided. The Magellan survey interviewed 1,002n likely voters, for a margin of error of +/- 3.10%. The exit polling data interviewed 3,992 voters, for a margin of error of +/- 1.55%. Both Ohio surveys predicted the final outcome fairly well, with both being within the margin of error. As in other surveys with larger minority populations, you can see how the automated interview approach struggles to get a solid, accurate measurement of African American voter opinion.
  8. Page 8 of 9   Magellan Strategies | 1685 Boxelder Street, Suite 300 | Louisville, CO 80027 www.MagellanStrategies.com | (303) 861-8585 Nevada Oct. 23rd - 24th Automated, Landline and Cellphone Survey Total*^ Men Wom White Wom Ind 65+ Hisp Black Difference 3.2% 5% 3% 2% 0% 3% 7% 33% Obama (Exit) 52.3% 48% 57% 46% 43% 44% 71% 92% Obama (Mag) 49.1% 43% 54% 48% 43% 47% 64% 59% Difference 0.1% 1% 1% 3% 8% 6% 1% 25% Romney (Exit) 45.6% 49% 41% 52% 50% 55% 24% 8% Romney (Mag) 45.5% 50% 42% 49% 42% 49% 25% 33% *The Total (Exit) column shows the final candidate vote percentage as reported on November 29th. ^ The actual Obama win margin was 6.7%, the survey Obama win margin was 3.6% with 2.8% of respondents undecided. The Magellan survey interviewed 754n likely voters, for a margin of error of +/- 3.57%. The exit polling data interviewed 4,388 voters, for a margin of error of +/- 1.63%. Nevada Oct. 31st - Nov. 1st Automated Only Survey vs. Exit Polls Total*^ Men Wom White Wom Ind 65+ Hisp Black Difference 4.9% 5% 6% 2% 3% 7% 12% 17% Obama (Exit) 52.3% 48% 57% 46% 43% 44% 71% 92% Obama (Mag) 47.4% 43% 51% 44% 40% 51% 59% 75% Difference 0.6% 4% 2% 1% 2% 7% 1% 5% Romney (Exit) 45.6% 49% 41% 52% 50% 55% 24% 8% Romney (Mag) 45.0% 53% 39% 53% 40% 48% 25% 13% *The Total (Exit) column shows the final candidate vote percentage as reported on November 29th. ^ The actual Obama win margin was 6.7%, the survey Obama win margin was 2.4% with 0.7% of respondents undecided. The Magellan survey interviewed 844n likely voters, for a margin of error of +/- 3.37%. The exit polling data interviewed 4,388 voters, for a margin of error of +/- 1.63%.
  9. Page 9 of 9   Magellan Strategies | 1685 Boxelder Street, Suite 300 | Louisville, CO 80027 www.MagellanStrategies.com | (303) 861-8585 North Carolina Oct. 22nd - 23rd Automated, Landline and Cellphone Survey Total*^ Men Wom White Wom Ind 65+ Black Difference 3.7% 2% 10% 14% 6% 11% 2% Romney (Exit) 50.3% 54% 49% 67% 57% 64% 4% Romney (Mag) 46.6% 56% 39% 53% 51% 53% 6% Difference 0.2% 6% 5% 9% 5% 5% 5% Obama (Exit) 48.3% 45% 51% 33% 42% 35% 96% Obama (Mag) 48.5% 39% 56% 42% 37% 40% 91% *The Total (Exit) column shows the final candidate vote percentage as reported on November 29th. ^ The actual Romney win margin was 2.0%, the survey Obama win margin was 1.9% with 4.9% of respondents undecided. The Magellan survey interviewed 769n likely voters, for a margin of error of +/- 3.53%. The exit polling data interviewed 4,341 voters, for a margin of error of +/- 1.49%. North Carolina Oct. 31st - Nov. 1st Automated Only Survey vs. Exit Polls Total*^ Men Wom White Wom Ind 65+ Black Difference 0.2% 0% 2% 5% 3% 8% 8% Romney (Exit) 50.3% 54% 49% 67% 57% 64% 4% Romney (Mag) 50.1% 54% 47% 62% 54% 56% 12% Difference 1.4% 2% 1% 4% 3% 6% 11% Obama (Exit) 48.3% 45% 51% 33% 42% 35% 96% Obama (Mag) 46.9% 43% 50% 37% 39% 41% 85% *The Total (Exit) column shows the final candidate vote percentage as reported on November 29th. ^ The actual Romney win margin was 2.0%, the survey Romney win margin was 3.2% with 1.1% of respondents undecided. The Magellan survey interviewed 1,006n likely voters, for a margin of error of +/- 3.09%. The exit polling data interviewed 4,341 voters, for a margin of error of +/- 1.49%. CONCLUSION We believe the 2012 election cycle was challenging for survey researchers because of assumptions made that it was unlikely that younger voters would turnout in the same percentages as they did in the 2008 election cycle. We also believe that the difficulty of getting younger voters on the phone, even with significant cell phone sample, contributed to lower sample coverage rates. We see this in our own surveys that are detailed in this memorandum. CONTACT MAGELLAN STRATEGIES Any questions regarding this memorandum should be directed to David Flaherty at 303-861-8585 or by email at Dflaherty@magellanstrategies.com. This memorandum can be downloaded on the front page of the Magellan Strategies website at www.MagellanStrategies.com.
Anúncio