Data Fluency for Dummies
•Transferir como PPTX, PDF•
1 gostou•1,149 visualizações
A recent Bersin survey pointed out that just 28 percent of organizations have “good” or “very good” levels of proficiency in basic data literacy skills. And that makes sense, because it often feels like you need a statistics degree to understand HR analytics. But the truth is, you don’t need a degree. You just need to know what to look for and how to turn that into meaningful conclusions. Degreed and Watershed are here to help.Join us for Data Fluency for Dummies.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Data Fluency for Dummies
Semelhante a Data Fluency for Dummies (20)
Mais de David Blake
Mais de David Blake (11)
Último
Último (20)
Data Fluency for Dummies
- 1. @watershedLRS @degreed Data Fluency for Dummies Tim Dickinson Director, Learning Analytics Strategy watershedLRS.com @watershedLRS James Densmore Director of Data Science Degreed.com @degreed
- 2. @watershedLRS @degreed What is Data Science?
- 3. @watershedLRS @degreed Why is it so confusing in the first place? • Hype cycle • VCs and startups! • A catchall for a fast-growing set of competencies
- 4. @watershedLRS @degreed One way to consider data science is as an evolutionary step in interdisciplinary fields— such as business analysis that incorporate computer science, modeling, statistics, analytics, and mathematics. - NYU Source: Calvin Andrus Common definitions of data science @watershedLRS @degreed
- 5. @watershedLRS @degreed A Broad Field Data Scientists are people with some mix of coding and statistical skills who work on making data useful in various ways. Source: Michael Hochster Type A Data Scientist: Analysis Type B Data Scientist: Building These scientists have some statistical background, are very strong coders, and may be trained software engineers. These scientists are concerned primarily with making sense of data or working with it in a fairly static way.
- 6. Stuff that data scientists actually do @watershedLRS @degreed
- 7. @watershedLRS @degreed How does data fit into learning?
- 8. @watershedLRS @degreed Learning Analytics © Watershed Systems, Inc 2018
- 9. @watershedLRS @degreed Understanding Learning Analytics: Categories Understand an overall learning program. Is this initiative helping to meet business objectives? Understand a learner or group of learners. Ensure organizational readiness and compliance. Understand more about a specific learning activity. Maximize effectiveness and spot problems. Learning Experience Learner Learning Program
- 10. @watershedLRS @degreed Understanding Learning Analytics: Complexity MEASUREMENT What are people doing? How are people interacting with your learning content? How are they performing? EVALUATION Is that good or bad? How are people, resources, and programs performing against benchmarks? Are they better or worse than they were before? ADVANCED EVALUATION Why? What are the reasons for good and bad performance? What’s different about your most successful people, resources, and programs? PREDICTIVE & PRESCRIPTIV E What would happen if I do X? Based on your data, can you predict what a successful person, resource, or program will look like? What do you need to do differently?
- 11. @watershedLRS @degreed@watershedLRS @degreed EXAMPLE: Complexity in Learning Experience MEASUREMENT What are people doing? Each month, our people watch an average of 2,764 videos of less than 3 minutes in length. EVALUATION Is that good or bad? Videos under 3 minutes receive 50% more views than videos longer than 3 minutes. ADVANCED EVALUATION Why? Videos longer than 3 minutes are too long; people are dropping out before completing, and people are less likely to even start. PREDICTIVE & PRESCRIPTIV E What would happen if… If we reduce the length of longer videos to under 3 minutes, we can increase completion rates by at least 50%.
- 12. @watershedLRS @degreed Link learning design to measurement Have they learned it? What do people need to learn? Are people doing that? What do people need to do? Did they complete it? What training is required? Was the goal achieved? Business goal Learning Design Measurement
- 13. @watershedLRS @degreed What’s it look like in the real world?
- 14. @watershedLRS @degreed Key Metrics Time to first drug Time to defibrillation Time to chest
- 15. What is the time to first drug? What is the time to defibrillation? What is the time to chest? Was time to chest under 90 seconds? Why or why not? What happens when we change X? Why or why not? Why or why not? What happens when we change X? What happens when we change X? Was time to defib under 180 seconds? Was time to first drug under 120 seconds? @watershedLRS @degreed
- 16. @watershedLRS @degreed Applying Benchmarks and Investigating Issues
- 17. @watershedLRS @degreed@watershedLRS @degreed How does training affect sales dollars?
- 18. @watershedLRS @degreed@watershedLRS @degreed Training completed in May 2017 How does training affect sales dollars?
- 19. @watershedLRS @degreed Your turn. Tips and tricks to use data.
- 20. @watershedLRS @degreed Keys to using data and data science in learning 1. Understand what learners want to learn. 2. Use data to help them learn. 3. Measure their progress.
- 21. @watershedLRS @degreed 1. Understand what they want to learn. You can ask them, or infer based on activity and preferences. 1. What do they say they want to learn? 2. What are they actively learning already? (These two things often do not match!)
- 22. @watershedLRS @degreed@watershedLRS @degreed Spot opportunities in the data. EVERYONE TALKS ABOUT DATA, DESIGN THINKING, AND AGILE—BUT ARE WE DOING ENOUGH TO DEVELOP OUR BIZ OPS SKILLS?
- 23. @watershedLRS @degreed 2. Help them learn. Match needs with resources. • Intelligent recommendations • Focus on skills rather than volume of content • Leverage their network Identify skills to build. • Understand their roles (or desired roles) in the organization • Focus them on skills associated with those roles, then on the content needed
- 24. @watershedLRS @degreed@watershedLRS @degreed Get them content – from anywhere. Content completions by provider type 64,107 SOURCES 63% 31% 6% Paid Internal Open @watershedLRS @degreed
- 25. Improve Recommendations Items influenced by a user’s social network are 22% more likely to be clicked than items that are not based on a user’s social network. YOUR NETWORK’S Recommendations Interests Tags YOUR GROUPS’ Interests Related topics Degreed, ClickZ, Wordstream, Signupto.com social feed non-social 5.5% 4.5% 1.9% 1.6% 0.1% 0.03% CLICK-THROUGH RATE @watershedLRS @degreed
- 26. @watershedLRS @degreed 3. Measure their progress. Investigate if learners are… …Focused on the correct skills …Improving or making progress over time …Exhibiting skill gaps or opportunities compared to their peers Don’t get stuck on the volume of content consumptions.
- 27. @watershedLRS @degreed@watershedLRS @degreed Identify Skill Gaps @degreed
- 28. @watershedLRS @degreed Learning Data Fluency Cheat Sheet Identify the analytics categories: • Learning experience • Learner • Learning program • Understand what learners want to learn. • Look for opportunities in the data. • Match needs with resources. • Identify what skills to build. • Provide options for content access. • Improve recommendations. • Measure progress. • Identify skills gaps. Know your data Use your data Measurement (start here) Evaluation Advanced evaluation Predictive and Prescriptive Start with measurement
- 29. @watershedLRS @degreed Data Fluency for Dummies Tim Dickinson Director, Learning Analytics Strategy watershedLRS.com @watershedLRS James Densmore Director of Data Science Degreed.com @degreed
Notas do Editor
- Why is there so much confusion over what data science is? Hype cycle VCs and startups! (data science in general a few years back, now AI, Deep Learning and ML) The problem is that it’s become a catch all for a fast growing set of competencies.
- You could spend a lifetime trying to find a standard definition of data science. Try it for yourself on Google, Quora, or even at a bar with a bunch of data scientists I like to ask companies and data science leaders how THEY define it when I’m going to work with them. There are some common themes though A combination of business analysis, computer science, statistics, and general mathematics A growing segment of data science is veering into specific artificial intelligence skill sets as well
- Data Scientists are people with some mix of coding and statistical skills who work on making data useful in various ways. In my world, there are two main types: Type A Data Scientist: The A is for Analysis. This type is primarily concerned with making sense of data or working with it in a fairly static way… Type B Data Scientist: The B is for Building. Type B Data Scientists share some statistical background with Type A, but they are also very strong coders and may be trained software engineers.
- You can think of learning analytics from the perspectives of a particular learning experience, of a learner of group of learners, or of a learning program where a group of learners work through a collection of resources and experiences. Within each of those categories, you can do anything from simple measurement of what’s happening right up to complex predictive and prescriptive analytics. This can also be seen as a maturity model and we encourage people to start with measurement in just one category and get value from that, before expanding into other categories or moving outwards in the triangle. So how do you get started?
- Then thinking about complexity, here’s some of the questions you should ask at each level. Notice how they build on one another, so you can’t just dive straight in to predictive and prescriptive; you have to start with measurement. I’ll give you a moment to read those questions. [PAUSE for 30 seconds] Also bear in mind that each of these questions could be something that’s answered automatically by analytics software, but they could also be questions that are answered outside of the software. For example, your analytics software might tell you what people are doing and how that compares to benchmarks, and then you do more qualitative research of talking to the people involved and reflecting to cover the higher levels of analysis.
- Here’s an example of moving through the levels of complexity around the length of videos. In this example, you could imagine reporting on video length, number of times each video is watched and the point at which people drop off watching the video. Again, I’ll give you a moment to read that example yourselves. [PAUSE for 30 seconds]
- So when you’re designing learning, whether that’s a single course or a whole program, start with a business goal in mind. Then, working with your subject matter experts, identify what people need to do differently or better to meet that goal. Then identify what they need to learn and the training and resources required for them to learn that. As you can see from the bottom half of the slide, designing the learning in this way gives you clear measures you can report on at each stage of your logic from the training to the business goal.
- In medical training, evaluation of training effectiveness is especially important. The real-world tasks clinicians are being trained to perform are high stakes and need to be done right first time. Clinicians can’t practice on real patients while they figure it all out; to do so would put lives at risk. Instead, they must reach competency in the training environment, before applying that training in the real world. Not only do the clinicians have to be competent, but MedStar needs to be confident in their competence. Assessments need to be rigorous and effective so MedStar knows which clinicians are ready. Another unique aspect of medical training is that the skills learnt relate to very practical and physical tasks. Training and assessments, therefore, also need to be practical and physical. While it is important to understand the theory of how the heart works, what really matters is to have the skills to perform the tasks needed to resuscitate the patient.
- Here we see an example of how we can learn more from the data by asking further questions of it. The top row shows examples of questions that straightforward measurements can answer. To continue going further to evaluate the learning we need to ask questions that help us understand what are data describes.
- A measurement and evaluation example: This dashboard is constructed to display activity measurements of Instructor Led Training in a physical classroom environment. If we focus on the chart on the left hand side, we can initially see the measurement of the NPS scores for 5 different courses. By applying a benchmark (identified by the broken horizontal line), we’re able to add context to this data so we can evaluate it. We now know whether what we are looking at is good or bad. Looking at this chart, it would be reasonable to investigate further into Courses 3 and 5 to learn why people are becoming increasingly dissatisfied with them.
- Another measurement vs evaluation example: This line chart displays data measuring the revenue generated by two different groups of sales associates: those who have completed a particular sales training course and those who have not. At first glance, it’s easy to think that the training was ineffective because the green group, who did not complete the training, outperform the blue group.
- However, in order to truly evaluate the effectiveness, we need to understand the complete context. If we know that the training was only completed by the Blue Group in May, and we show that they were already underperforming, we can demonstrate with confidence that the training was effective at bringing the underperforming group up to parity with their peers. In this example, the product in question was discontinued at the end of August, hence the large drop off.
- An example – Use your data to see what leaners on the platform are actively looking for. In this example (again, Degreed but we can generalize) you can see the growth in what learners are searching for year over year. Is everyone aware of these trends? Are we making sure to keep up with them as far as content, training, etc?
- In Degreed, we found that the majority of the content learner’s are using comes from “open” sources (free from across the internet). It’s not just about formal learning – we can use this insight to help learner’s find good content from all over. We’d never know if we didn’t look into thedata
- An example on recommendations We can also use data to better understand how users learn as well as to improve the learning product itself. This is the goal of many Data Science teams. At Degreed we use data to feed back into the recommendation models, but also to see why certain recommendations do better than others. In this example, content that ends up in a learner’s feed (the home screen on Degreed) are far more likely to be clicked on if they made the feed due to a social connection of the learner. Average CTRs LinkedIn 0.025% https://www.clickz.com/what-is-a-good-click-through-rate-for-ppc/44829/ Facebook 0.090% http://www.wordstream.com/blog/ws/2017/02/28/facebook-advertising-benchmarks Twitter 1.64% https://www.signupto.com/news/permission-marketing/twitter-marketing-what-results-should-you-expect-infographic/ Google 1.91% (intent) http://www.wordstream.com/blog/ws/2016/02/29/google-adwords-industry-benchmarks
- An example from Degreed data, but we can generalize – Compare an the skill level of employees in your organization to an industry or set of competitors. Track these over time, not just once! Can talk through data collection on basic visualization in an Excel chart