Recomendados
Recomendados
Mais conteúdo relacionado
Destaque
Destaque (8)
Semelhante a Making Data Work For You - The Data Assemblyline
Semelhante a Making Data Work For You - The Data Assemblyline (20)
Mais de Lauren-Glenn Davitian
Mais de Lauren-Glenn Davitian (20)
Último
Último (20)
Making Data Work For You - The Data Assemblyline
- 1. Making Data Work for You The Data Assembly Line 5/1/14 Lauren-Glenn Davitian, host Shelagh Cooley, Common Good Vermont Michael Moser, Vermont State Data Center 1
- 2. Logistics Watch on your computer. Interaction – We want to hear from you! Comments and Questions – Chat Box Slides and Recording Afterwards. Satisfaction Survey. 2
- 3. The Data Assembly Line Setting the Stage Data Audit Data Collection Data Management Data Analysis Data Utilization 3
- 4. Setting the Stage Why is data important? What results do you seek? Who is your audience? How will they use the information? 4
- 5. Question Cat Who is YOUR audience? 5
- 6. QuickTime™ and a decompressor are needed to see this picture. “I can honestly say that not a day goes by when we don’t use those evaluations in one way or another.” 6
- 7. Example: Foodshelf Result: Food security for people in the region. Audience: Board, funders, policy makers, staff 7
- 8. PERFORMANCE MEASURES Foodshelf # Clients Served lbs. of Food Provided % Nutritious Food % Clients Satisfied with food choice Staff-Client Ratio # Clients not returning -after 6 months - after 1 year % of Clients not returning -after 6 months - after 1 year QUANTITY (#) QUALITY (%) 8
- 9. Question Cat Who is YOUR audience? 9
- 10. The Data Assembly Line Setting the Stage Data Audit Data Collection Data Management Data Analysis Data Utilization 10
- 11. Data Audit - People What people do you have? How are they involved? Do they have “will” & “skill”? Is it in their job description? 11
- 12. Data Audit - Data • What data do you already have? • What data do you need? • What data is most useful? 12
- 13. QuickTime™ and a decompressor are needed to see this picture. 13
- 14. Data Source Tracking Sheet Fremework Evaluation Question Data Source Indicator Food How much food do we serve? Inventoroy # pounds of food/day Clients How many clients do we serve Registration Forms #clients/month Satisfaction How satisfied are clients with the food they receive? Satisfaction Survey % of very satisfied clients/month Quality of Food How nutritious is the food? Receipts to Clients % of Food distributed that is nutritious Food Security How many clients are food secure after 6 months? 1 year? # Clients Receiving less than 5 lbs of food per month 14
- 15. Question Cat Where do you collect your data from? 15
- 16. Data Sources •Calendars: Events, Meals, Clinics •Internal Databases: Clients •Attendance Sheets: Youth •Log Sheets: Services Received •Surveys: Satisfaction 16
- 17. Question Cat Where do you collect your data from? 17
- 18. The Data Assembly Line Setting the Stage Data AuditData Audit Data Collection Data Management Data Analysis Data Utilization 18
- 19. Data Collection Data You Already Collect Data Development Agenda Data Audit 19
- 20. Data Collection Value of the Performance Measure System Buy-in Standard 20
- 21. Question Cat What tools do YOU use to collect dat? 21
- 23. Benefits of Using Tools • Reduce human error, increase consistency. • Standardizing data reduces time “cleaning data” • Online (remote access for multiple sites). • Automatic reports and organized data. • Automatically backed up. • It’s EASIER!!! 23
- 24. Question Cat What tools do you use to collect data? 24
- 25. The Data Assembly Line Setting the Stage Data AuditData Audit Data Collection Data Management Data Analysis Data Utilization 25
- 26. Data Management Is your data secure? What is your time frame? Do you have back-ups? Who is responsible? 26
- 28. Data Management Don’t mix up your data types. 28
- 29. PERFORMANCE MEASURES Foodshelf # Clients Served lbs. of Food Provided % Nutritious Food % Clients Satisfied with food choice Staff-Client Ratio # Clients not returning -after 6 months - after 1 year % of Clients not returning -after 6 months - after 1 year QUANTITY (#) QUALITY (%) 29
- 30. Data Management Example January 1-15 January 16-31 January Average February 1-15 February 16-28 February Average 2008 258 275 266.5 350 375 362.5 2009 245 272 258.5 324 370 347 2010 242 267 254.5 356 368 362 2011 235 265 250 333 362 347.5 2012 222 260 241 332 360 346 2013 216 262 239 313 345 329 2014 224 252 238 310 370 340 30
- 32. Data Management Example January 1- 15 January 16- 31 February 1- 15 February 16- 28 March 1- 15 2008 258 275 350 375 297 2009 245 272 324 370 297 2010 242 267 356 368 279 2011 235 265 333 362 234 2012 222 260 332 360 285 2013 216 262 313 345 262 2014 224 252 310 370 155 32
- 34. Data Management Best Practices 1. Training 2. Buy-In 3. File Naming 4. Security Practices 5. Code Book 34
- 35. Stretch Break Send in Your Questions! 35
- 36. The Data Assembly Line Setting the Stage Data AuditData Audit Data Collection Data Management Data Analysis Data Utilization 36
- 37. Analyzing the Data Remember, who is your audience? How often will the data be analyzed? Time Series vs. Non-Time Series Data 37
- 38. Example of Survey Monkey 38
- 39. Trend Lines 39
- 40. The Data Assembly Line Setting the Stage Data AuditData Audit Data Collection Data Management Data Analysis Data Utilization 40
- 41. Data Utilization Decision Making Fundraising Community Building Education 41
- 42. Data Utilization Convene your audience. What story does the data tell? Are we achieving the results we want? What will we do differently? 42
- 43. Why Visualize Data? “A picture is worth a thousand words” Higher Engagement from audience. Broader audience. 43
- 47. Sources: http://thegrio.com/2013/06/07/hunger-in-america-food- insecurity-disproportionately-affects-african- americans/#s:foodinsecurity2 http://www.vtfoodatlas.com/plan/chapter/4-1-food-security-in- vermont Percentage Food Insecurity Very Low Food Security 1999-2001 9.1 1.8 2001-2003 8.9 3 2003-2005 9.5 3.9 2005-2007 10.2 4.6 2007-2009 13.6 6.2 2009-2011 12 5.4 Infographic Example: -Simple -Key information is highlighted -Effective Messaging 47
- 49. Rules to Live By Continue to adapt & improve measures Learn from others Be willing to invest if the information has value to you. Keep it Simple. Be realistic & practical 49
- 50. Assembly Line Not Automatic 50
- 51. PERFORMANCE MEASURES HOW MUCH ARE WE DOING? HOW WELL ARE WE DOING IT? HOW WELL ARE WE DOING IT? HOW WELL ARE WE DOING IT? QUANTITY (#) QUALITY (%) 51
- 52. PERFORMANCE MEASURES Afterschool # Youth served # Hours of Programming (academic vs. non- academic) # of Enrichment activities % High Quality Interaction % Youth Satisfaction % Parent Satisfaction Staff-youth Ratio # Youth Honor Roll # Youth 90% School Attendance # Youth leading activities % Youth Honor Roll % Youth with 90% School Attendance % Youth leading activities QUANTITY (#) QUALITY (%) 52
- 53. PERFORMANCE MEASURES Mental Health # Clients Served Size of Waiting list Average time to next appointment # of Clients in school or working # of Clients into institutional care # of Clients to less restrictive care % of Clients in school or working % of Clients into institutional care % of Clients to less restrictive care QUANTITY (#) QUALITY (%) 53
- 54. Upcoming Events Let’s Talk Shop: Regional Mixers - Brattleboro 6/19 Leadership Vermont Luncheons -Burlington 5/21 -Brattleboro 6/20 -Barnet 10/1 54
- 55. Resources Idealware http://www.idealware.org/ Helping Nonprofits Make Smart Software Decisions Youtube Video: How to Add a Trend Line in Excel: https://www.youtube.com/watch?v=svFSKnmAlKQ An Introduction to Regression Analysis: Alan O. Skyes: http://www.law.uchicago.edu/files/files/20.Sykes_.Regression.pd f CGVT Know More, Do More with Data Visualization http://blog.commongoodvt.org/2013/10/video-storytelling-with- data/ CGVT Surveys and Spreadsheets https://www.cctv.org/watch-tv/programs/making-data-work-you- survey-spreadsheets 55
- 56. References Trying Hard is Not Good Enough, Mark Friedman http://resultsleadership.org/product/trying-hard-is-not-good- enough-by-mark-friedman/ Essentials of Utilization-Focused Evaluation (Sage, 2012) by Michael Quinn Patton Edward Tufte, Data Visualization: http://www.edwardtufte.com/tufte/index American Evaluation Association – Tools, Tips, Trainings http://www.eval.org/ Also, check out courses offered at area colleges. 56
Notas do Editor
- The Agenda for Today.
- How do you know you are making a difference, if you don’t measure the change? What change do you want to see? How will you know when you get there? Planning the Data Process – involve stakeholders - Be sure to Involve stakeholders in the process. Example: Include staff, executive director, executive board, clients and the public in planning the data process. Ask these important questions: Who are the end-users? (program staff, executive board, funders, etc) What is the end-use? (direct policy, allocate resources, seek funding, annual report) Is the data relevant to them!!!
- How do you know you are making a difference, if you don’t measure the change? What change do you want to see? How will you know when you get there? Planning the Data Process – involve stakeholders - Be sure to Involve stakeholders in the process. Example: Include staff, executive director, executive board, clients and the public in planning the data process. Ask these important questions: Who are the end-users? (program staff, executive board, funders, etc) What is the end-use? (direct policy, allocate resources, seek funding, annual report) Is the data relevant to them!!!
- You want the information to be USEFUL!!!! What format will you be using this data? In a report that nobody reads?
- More important questions What data will be most relevant to our program evaluation? What data is missing? Does that data exist somewhere else (another agency, government, etc)? Any data missing goes on your DATA DEVELOPMENT AGENDA. Remember- Who is your audience? Include your stakeholders in this process. If you could know three things about the work you do that you don’t know, what would they be?
- Checking back in…What did people say?
- The Agenda for Today.
- What people do you have? - Who are your team members? How are they involved? - What are their responsiblities currently and how will they change going forward? Do they have “will” & “skill”? - Get buy-in (helps to include them from the beginning) and also recognize when you need to avoid nay-sayers Is it in their job description? - is “data” explicit to their job? Is should be!
- More important questions What data will be most relevant to our program evaluation? What data is missing? Does that data exist somewhere else (another agency, government, etc)? Any data missing goes on your DATA DEVELOPMENT AGENDA. Remember- Who is your audience? Include your stakeholders in this process. If you could know three things about the work you do that you don’t know, what would they be?
- Sometimes it’s a balance between funders needs and programming needs. Hopefully, with conversations and time, these needs can be aligned but it can be a balancing act. We recognize that you are being pulled in multiple directions.
- One place where all the indicators are kept track of either in excel or google docs. MealsHow many meals do we serve?Kitchen Calendar#meals/month ClinicsHow many clinics do we do?Mobile Van Calendar#clinics/month How many patients see a doctor?Doctor Log Sheet#patients/month Case ManagementHow many clients receive case management?Internal DatabaseTotal # clients
- List the sources of data - where does your data come from?
- Other examples of Data Sources
- List the sources of data - where does your data come from?
- Next we will talk about Data Collection.
- From the data audit you will get clear about what data you already collect and how versus what you would like to collectj--- the data on your Data Development Agenda
- Not All Performance Measures are Created Equal What is the simplest way to collect the data? Low/cost or no-cost. Do you have data source that could also collect the information you want? Make sure you have buy-in. Collect consistent and standardized data. - Electronic data collection tools can be very efficient and reduce errors.- Standardize
- What don’t you know about your work that would help make a bigger impact?
- Idealware - great resource to help nonprofits choose tools NTEN- Nonprofit Technology Network Techsoup
- Here are some of the advantages of using structured survey applications as input tools.
- Check-in- Do you have a data development agenda? What’s on it?
- Next we will talk about Data Collection.
- What should you backup: The simple answer is, “everything.” Or at least, everything that’s critical, valuable, irreplaceable, or important to your organization.
- The Dashboard can be an effective way to reduce frustration and confusion. The dashboard is an inventory of your data sources with the website link, the username, password, and any notes about what the website is used for. This is especially useful is there is staff turnover or staff at multiple sites.
- Don’t mix up your data types.
- Here’s the same table with out the averages. You can put those in another tab on your excel spreadsheet.
- Look at what other people are doing. Go to a training at American Evaluation Association or Burlington College online Get buy-in from your stakeholders File naming- spreadsheets of data with no context. (include: Project, Data Source, Your Name, Final or Edits, Date) Personally identifiable information- you need to have someway to keep that data secure If you have lots of data passing hands, it can be great to have “code book” or a place to describe what each indicator means, where it comes from and if there are different values for them.
- Next we will talk about Data Collection.
- Do you have the skill set to accomplish this? Does the person analyzing know what the end-users want to know? Is average number of clients per month more useful than total number clients or do they need both? Create a Schedule ( second staff meeting of every month we will look at the data). Pre/Post Comparative Trend Qualitative Analysis Tools for analysis Excel SPSS
- Whyi using survey tools can save you a lot of time! A lot of tools like lime survey, survey monkey, google analytics have a built-in reporting function. Look at what they have already, you can save yourself a lot of headache if what they offer is sufficient for your needs.
- Does the data make sense? – Have more than one person look at the results.
- Next we will talk about Data Collection.
- Here are some reasons that you will use data in your work.
- For Decision making--- Who will be using the data?- What process will be driven by the data? Data Visualization/Presentation is important. Graphs with narrative and quotes is more powerful than just numbers alone. Give then numbers context. Keep it simple!
- On Facebook, videos are shared 12x more than links and text posts combined. Photos are liked 2x more than text updates.
- Tips for Visualizing Data: Above all else show data Maximize the data-ink ratio Erase non-data-ink Erase redundant data-ink Revise and edit
- Tips for Visualizing Data: Above all else show data Maximize the data-ink ratio Erase non-data-ink Erase redundant data-ink Revise and edit
- Comparing these to methods of visualization. Each are ok. Consider your audience. Do you want to share the table on facebook? Maybe the table is ok for internal work. I would recommend make a bar chart out of these numbers. Then if you want to share publicly you can visualize the data like above.
- Think of creating graphics as showing the world what you do and why you do it, in a compelling, interesting and easy-to-share way! If you want to know more about visualization, then you can watch Lauren Glenn’s show.
- There is no one way to measure or evaluate a program or organization. There is no “Gold Standard”. You will continue to adapt your evaluation to meet your needs. There is a distinction between research and evaluation and that comes from the rigor of the methods. For evaluation, you need to decide what is important to you and how you will be using the data. -ADD START TODAY!
- All of this takes communication. Leadership needs to be invested and willing to invest and above all this is not an automatic process, there are people that need to be include and involved.
- The source for