Increasing Voter Knowledge with Pre-Election Interventions on Facebook
â˘
2 gostaramâ˘265 visualizaçþes
As part of our Data Science to Solve Social Problems series, Facebook Data Scientist Winter Mason presented on efforts to increase online civic engagement.
Recomendados
Recomendados
Mais conteĂşdo relacionado
Mais procurados
Mais procurados (19)
Semelhante a Increasing Voter Knowledge with Pre-Election Interventions on Facebook
Semelhante a Increasing Voter Knowledge with Pre-Election Interventions on Facebook (20)
Ăltimo
Ăltimo (20)
Increasing Voter Knowledge with Pre-Election Interventions on Facebook
- 1. Increasing voter knowledge with pre-election interventions on Facebook MIT GovLab November 2017
- 2. tl;dr: moving voter knowledge is hard but possible Overview ⢠Civic Engagement at Facebook ⢠Goals & values of team ⢠Research strategy ⢠Social Ballot ⢠Impact on U.S. knowledge ⢠Perspectives Pivot ⢠Impact on GB & FR knowledge
- 3. Our research team! Monica Lee Devra MoehlerWinter Mason Samidh Chakrabarti Justin Grimmer Funda Kivran-Swaine
- 4. Our research team (extended)! Nicole Bonoff Christy SauperSahar Massachi Annie Franco Brian Goldman Eric Anderson
- 5. Civic engagement is one of five major pillars of how FB seeks to realize its mission https://www.facebook.com/notes/mark-zuckerberg/building-gobal-community/10154544292806634/ Giving people a voice is a principle our community has been committed to since we began. As we look ahead to building the social infrastructure for a global community, we will work on building new tools that encourage thoughtful civic engagement. Only through dramatically greater engagement can we ensure [that] political processes reflect our values. -- Mark Zuckerberg, February 2017
- 6. âPolitical Efficacyâ is a North Star for âbetterâ Political Efficacy definition /pÉËlidÉk(É)l efÉkÉsÄ/ noun In political science, political efficacy is the citizensâ belief that they can understand and influence political affairs. It is commonly measured by surveys and is used as an indicator for the broader health of civil society. 0 10 20 30 40 50 60 70 80 ANES Political Efficacy Index 1956 20121984 Sources: Wikipedia (for definition). ANES (for chart).
- 7. We think hard about the values driving our work ⢠Be Selfless: Serve people's interests first, not Facebook's interests ⢠Be Protective: Keep people safe (including from personal risk) ⢠Be Fair: Provide same opportunities to everyone ⢠Be Representative: Strive towards broadly inclusive products ⢠Be Constructive: Build empathy and defuse acrimonious polarization ⢠Be Conscious: Know our impact (both positive and negative)
- 8. Qualitative research is our starting point What weâve done so far: ⢠In-depth interviews with people in 12 countries, including multiple U.S. cities ⢠Group interviews with U.S. Senate & Congressional staffers ⢠Interviews with social media managers of world leaders What weâve found: ⢠Elections are the main vehicle to get voices heard ⢠Universal desire exists to connect with representatives, especially to get a sense of their achievements while in office ⢠Limited awareness of civic activities one could do outside of national elections or an immediate crisis ⢠Skepticism that individual voices matter ⢠People sometimes worry that online political conversations online can lead to personal risk ⢠Perception of account security is an important driver for peopleâs willing to engage in political discourse online â... I don't know that [my voice] was individually heard, but I think as a whole it was.â
- 9. Analyzing interaction patterns on Facebook helps us understand engagement with politicians⌠Comments on Senator Pages: ⢠Red lines = from constituents ⢠Blue lines = from out of state Comments arenât just from constituents⌠there is also a vigorous national dialogue
- 10. Analyzing interaction patterns on Facebook helps us understand engagement with politicians⌠Discussion also spikes when an issue is part of the national conversation Trending issues are central to engagement and drive a lot of interactions w/ reps
- 11. Analyzing interaction patterns on Facebook helps us understand engagement with politicians⌠Constituents and non-constituents often comment about different things on politiciansâ Pages Even online, constituency matters in political interactions
- 12. ⌠and gaps in political engagement on Facebook Political Commenting Across FB By Age
- 13. Self-declared political ideology of US users on their Profiles Perhaps most importantly, observational analyses help us stay true to core principles (like fairness) Self-reported political ideology of US users based on surveys
- 14. Voter Knowledge in U.S. 2016 Presidential Election
- 15. Voting Plan was our flagship product to enhance peopleâs knowledge of their ballot
- 16. See a sample of all the races and referenda that will be on your ballot on election day
- 17. Learn more about a candidate or pick your favorites if youâve made a decision
- 18. Go in depth on a candidateâs platform as well as what theyâve been talking about recently
- 19. Ask your friends about their favorites if theyâve chosen to share them with you
- 20. Our goal for the US 2016 election was to increase voter knowledge, voter turnout, and ongoing connections Voter Turnout Voter Knowledge Ongoing Connections
- 21. Our usage and participation rates exceeded our expectations, but weâre here to talk about real impact! [Usage] ~100M ballot contests shown to people [Participation] ~10M unique people visited the product [Impact] Lift in knowledge? Keeping you in suspenseâŚ. Voter Turnout Voter Knowledge Ongoing Connections
- 22. We conducted a large scale survey to measure our impact on both knowledge and key attitudes Important to look both for beneficial effects on some metrics and also avoid harmful effects on others Real World Outcome Expected Impact Actual Impact Knowledge Contests on ballot Examples: State Legislature, Attorney General, etc. Increase Candidates for a contest Examples: Kamala Harris, Loretta Sanchez, etc. Increase Attitudes Affective polarization Neutral
- 23. The survey was dynamically customized for each person so they could be asked about their own ballot Contest Knowledge Candidate Knowledge
- 24. Random control groups were used to causally determine the impact of these products Access Points to Voting Plan Treatment Control (1%) Search Bookmark Newsfeed PromoFriendsâ Posts Search Bookmark Friendsâ Posts Random assignment of promotion, not product (i.e., the products were accessible to everyone, but a small control group had less promotion)
- 25. Survey Methods ⢠2016-11-09 to 2016-11-15 ⢠51,388 Participants ⢠Median age: 37 ⢠63.7% Female ⢠All 50 states
- 26. Our products increased the accuracy of peopleâs knowledge of the contests on their ballot by 6% 6% is equivalent to the ballot knowledge delta between people in late high school and people in college! 90% confidence 95% confidence 99% confidence
- 27. But we were not able to measurably lift peopleâs knowledge of which candidates are on their ballot We continue to try to improve 90% confidence 95% confidence 99% confidence
- 28. As an important check, thankfully we also did not make problems such as polarization any worse No measurable impact on other attitudes either (e.g., political efficacy, strength of party identification, etc.) 90% confidence 95% confidence 99% confidence
- 29. Our key lesson is that while improving peopleâs ballot knowledge is hard, it is possible to do at scale! Real World Outcome Expected Impact Actual Impact Knowledge Contests on ballot Examples: State Legislature, Attorney General, etc. Increase Increase Candidates for a contest Examples: Kamala Harris, Loretta Sanchez, etc. Increase Neutral Attitudes Affective polarization Neutral Neutral This is just the beginning! Our efforts continue!
- 30. Voter Knowledge in France and U.K. in 2017
- 31. Issues tab for Politicians & Parties
- 32. Election Perspectives http://www.huffingtonpost.fr/2017/04/11/facebook-lance- perspectives-son-comparateur-de-programmes-des_a_22034911/ What happenedafter you clicked on the Feed unit during the UK Election? ⢠On the comparison page, people could toggle between different issues, and see what the parties had filled out in Issue Cards on their party pages (pictured to the right). ⢠This was a joint effort with the Civic Engagement team to encourage candidates to complete these issue cards. Vision for the Election spectives Pivot the Election es Pivot is to people to diverse es when they nformation about ues. We believe success in this provide a light- to discover spectives in the ght when people ming content ssue.
- 33. Engagement with Pivot Reach France Round 1 â 7.5% DAP France Round 2 â 15% DAP UK â 10.1% DAP CTR (per person) France Round 1â 8% France Round 2â 11% UK â 9.1% Time spent on landing page: France Round 2â 119 seconds UK â 181 seconds
- 34. Engagement with Issues France U.K. Clicks Shares Clicks Shares
- 35. Survey Research - France - Perceptions Method ⢠N = 5341 French-speaking respondents ⢠57% Male ⢠Mean Age = 32yr (SD = 16.35) Responses were collected ⢠Round 1: April 12th â April 20th (from the launch of the Pivot until a few days prior to the first round) ⢠Round 2: April 27th â May 7th (the days from the launch of the Pivot until the final day of the election) Participants were sent one of 6 ârapid feedbackâ surveys: ⢠All surveys first asked respondents how much they wanted to see the pivot in their News Feed ⢠The second question rotated through different versions, each asking about how informing/useful they felt the pivot was UXEvaluation Framework for the Election Perspectives Pivot In addition to metrics-based evaluations (e.g., CTR and time spent), we also evaluated the success of the Election Perspectives Pivot with a series of Rapid Feedback Surveys that people were prompted to complete after visiting the candidate comparison page.
- 36. Survey Research - France - Perceptions
- 37. Survey Research - France - Knowledge Method ⢠Round 1 N = 8,261 ⢠Round 2 N = 7,195 ⢠People who were eligible to see the pivot, with an oversampling of a holdout group. Responses were collected ⢠Round 1: April 16th â April 23rd ⢠Round 2: April 30th â May 10th Participants saw an invitation to participate in a survey in News Feed
- 38. Survey Research - France - Knowledge
- 39. Survey Research - UK - Perceptions ⢠N = 2007 British respondents ⢠58% Male ⢠Mean Age = 32yr (SD = 15.88) Responses were collected ⢠June 1st â June 8th (the days from the launch of the Election Perspectives Pivot until the day of the Election) Participants were sent one of 3 rapid feedback surveys: ⢠All surveys first asked respondents how much they wanted to see the pivot in their ⢠The second question rotated through different versions, each asking about how informing they felt the pivot was UXEvaluation Framework for the Election Perspectives Pivot In addition to metrics-based evaluations (e.g., CTR and time spent), we also evaluated the success of the Election Perspectives Pivot with a series of Rapid Feedback Surveys that people were prompted to complete after visiting the candidate comparison page.
- 40. Survey Research - UK - Perceptions 66% wanted to see 60% learned something new
- 41. User Research UK (6/5-6/9): Focus groups, 1:1 in-depth interviews, and intercepts with a total of 69 people
- 42. Survey Research - UK - Knowledge Method ⢠N = 23,000 ⢠People who were eligible to see the pivot, with an oversampling of a holdout group. Responses were collected ⢠June 6th â June 11th Participants saw an invitation to participate in a survey in News Feed
- 43. Survey Research - UK - Knowledge
- 44. ⢠Itâs hard (but possible) to improve voter knowledge at scale ⢠Positive impact in U.S. 2016 election and 2nd round France ⢠Neutral impact in 1st round France & U.K. ⢠A focus on societal impact (vs usage metrics) is a core value of the team ⢠Research is central to ensuring we are staying true to our values Conclusion: This is a tough yet worthwhile endeavor
- 46. Voter Knowledge Ongoing Connections We also served a comprehensive system of voting reminders to encourage more people to vote Voter Turnout
- 47. 2Mvoters registered in 2016 (estim.) Voter Turnout Voter Knowledge Connected Connections This work moved the needle on voter participation
- 48. Election results served as a springboard to encourage people to connect to their new reps Voter Turnout Voter Knowledge Ongoing Connections
- 49. We limited the types of analyses to only include those that would improve the product Outcomes Researched Information explored Y # of selections Y Privacy level of selection Y Candidate selected N Cuts Researched Demographics (gender) Y Location (state) Y Party affiliation N Political ideology N Improve utilization among broad population, regardless of peopleâs political beliefs
- 50. Who should be able to see your selections? FriendsOnly You More Comfort For Private People More Conversation For Everyone People are either split on who to share with or they may not fully understand this privacy control Same number of candidates selected in both cases A Defaults to âOnly Youâ (Can Switch to Friends) B Defaults to âFriendsâ (Can Switch to Only You)
- 51. Created a novel 2-step privacy model: no default and requires explicit privacy selection for each contest ⢠More complex ⢠More drop off ⢠Non-standard ⢠But still worth doing!
- 52. Requiring a choice ultimately improved both privacy and engagement on the final product in November ⢠Both settings used a lot! ⢠âFriendsâ selected more frequently down ballot, increasing conversation ⢠Visual tutorial on privacy model further boosted usage by 70%
- 53. During user interviews testing early mocks, a few people initially thought this might be a way to cast a vote We made radical design & content changes to try to eliminate the possibility of confusion with voting Early Mock Final Version ⢠Big red bar (abhorrent to FB designers) ⢠From âselectâ to âfavoriteâ and from âvoteâ to âplanâ ⢠Follow up notification to go to polls on election day
- 54. What info should we show? How to be fair to all candidates when varying amounts of info are available? UX interviews showed people thought it would be fairest to⌠⢠Show all available info even if not all candidates have it ⢠Link offsite to website ⢠Be sure to get word out to candidates about this feature
- 55. In the end people found all of these types of info to be valuable, validating the decision to be expansive
- 56. Gender Disparities: From TICTeC 2016âŚ
- 57. We got closer to gender parity due to privacy model Product Metric Usage by Women Click rate on product promo 28% higher (than men) âFriendsâ privacy selection % 32% lower âOnly Meâ privacy selection % 19% higher Average # of selections 20% lower Women were more interested than men in the product but would have used it far less than men if not for the privacy model
- 58. How to order the contests: federal first or local first? Federal -> Local ⢠Matches actual ballot so less likely to create confusion ⢠Grabs attention and helps people understand product Local -> Federal ⢠Less acrimonious/polarizing ⢠Increases exposure to low-information races
- 59. Interviews drove a âFederal firstâ approach for Voting Plan⌠Federal -> Local ⢠Matches actual ballot so less likely to create confusion ⢠Grabs attention and helps people understand product Local -> Federal ⢠Less acrimonious/polarizing ⢠Increases exposure to low-information races
- 60. However, in our election results product, we were able to default to the âlocal firstâ approach Post-Election Results ⢠Election is over so no risk of ballot confusion ⢠May attenuate polarization at a time when acrimony peaks ⢠Provides the most constructive pathway for staying involved
- 61. Most importantly our community loved these features âI appreciate the interactive guide to the election. I was more prepared to vote because of Facebook.â
Notas do Editor
- [For Transition OutâŚ] Just as important as what a product does is *how* it does it. So beyond achieving these goals, we wanted build the product in a way that honored a core set of values.
- XXX We also included attitude measures, most notably polarization âattitudes about supporters of different parties We did not expect to affect polarization, but we felt it is critical to measure our impact on it It is one of the biggest problems for democracy in the US today and so it was necessary to make sure the product werenât exacerbating this problem
- The survey was customized for each person left is the question for the first type of knowledge right are examples of the questions for the second type of knowledge Menlo Park NOW that Iâve described the outcomes we measured, let me tell you about the research design we used The key question was how could we evaluate the causal impact of the product while still allowing everyone to use the product The standard thing to do is to withhold the product from a randomly selected group of people We took a different approach where everyone could use the productâŚ
- We randomly spit the population into a treatment and small control group, on the order of 1% Both groups could access the exact same product through Search , Bookmark, Friends posts. But the treatment group additionally saw a promotion to use the product This newsfeed banner â happy valentines day but it invited you to preview your ballot encouragement design. Just by randomly varying the likelihood of people seeing the product, you can rigorously measure impact What Iâm going to show you next is a comparison of everyone in the treatment group versus everyone in the control group intention to treat analysis. It likely underestimates the real effect given that some people in the control group used the product
- Fortunately, our products did significantly increase peopleâs knowledge of the contests on their ballot The difference between the treatment and control group was 6%. 6% is roughly equivalent to the difference in knowledge between people in high school, and people in college And this is the lift in knowledge for everyone in the treatment group including both those who interacted with the product and those that didnât In this figure, the red line shows the estimated 6% difference between treatment and control. The blue shows the confidence intervals. Dark blue is 90% confidence, Medium blue is 95% confidence Light blue is 99% confidence
- Unfortunately, we didnât have a measurable impact that that second kind of knowledge XXX
- Third, we didnât significantly effect affective polarization. Fortunately, the product did not seem to exacerbate polarization Confidence intervals are large. Measuring Attitudes is hard
- XXX This evaluation was necessary to learn what we need to work on.
- Increase exposure to issues tab on pages Increase knowledge about candidates / issues Increase exposure to different ideas
- Things people are interested in are not always the same as what people want to share
- Explain rapid feedback.
- Explain questions on survey including knowledge questions about issues Explain how to interpret plot & how they are created
- Explain hold out groups Main point is no difference on interest (no surprise) and most people were somewhat interested (higher than U.S.)
- Too often we talk of impact in terms of impact on usage metrics, and not societal outcomes. If we donât have this focus, then what are we all doing here? I understand this can be a scary thing to do: - Expensive - Methodology still in its infancy & inconclusive results are the norm - Afraid of what we might find out⌠maybe our impact was negligible? But the most important reason to do this, despite these fears, is that research is central to ensuring civic tech is values-conscious Still have a lot of work to do: privacy models⌠Many of you already realize this, but itâs an exceptionally rare thing for an organization like ours to give a behind the scenes look⌠tough tradeoffs, what worked and what didnât work. But itâs important to us to have this dialogue with the community! Want to work together to have a real impact.
- XXX In order to respect the fact that political beliefs are sensitive, we didnât look do any analysis using party affiliation XXX
- One question was, what kind of privacy setting should be used. In the interviews, some people said that they wanted to keep their selections private Others said they wanted to share with friends to initiate conversations about the candidates [click] XXX We never considered public sharing since in the interviews, people did not express interest in sharing outside their friend network [click] To quantify the tradeoff and help us decide what to do, we conducted an AB test What we found was that people on average shared the same number of candidates in both cases This means that either people are spit on who to share with, or they may not fully understand the product
- XXX
- XXX
- XXXX People still made fun of us upon launch⌠but worthwhile to keep people safe.
- XXX Still providing this many choices might have decreased choice for some. Sometimes more choices leads to a paradox of choice. When faced with too many choices people just donât chooose.
- Fortunately this was not the case People did examine all the kinds of information we offered, thus validating the decision to be expansive
- Last year, we shared some data showing that women are less active producers of political content on FB. Just as they tend to be offline. But weâve wanted to do better, so weâve been monitoring and trying to improve this gender disparity in our products this year.
- Women clicked on the promotion to see the product more than men suggesting they were more interested in seeing what would be on their ballot Fortunately there was no gender gap in exposure to the product However there was a difference in how women and men used the product When it came to selecting candidates, women shared with friends less than men, and kept selections private more than men âŚchoice of a private mode decreased the gender gap from what it would have been otherwise. If we defaulted to sharing, the gap probably would have been closer to 32% The ability to choose reduced the gender gap in selection to 20% âŚproducts that allow people to interact with the product in private, will help to reduce the gender gap BUT, we have more work to do
- XXX
- Interviews showed that Federal to local was the order was most intuitive and would lead to the least confusion
- But for our product launched election eve, we were able to reverse the order XXX