Moving Forward on Learning Analytics - A/Professor Deborah West, Charles Darwin University and Ms Nicole Wall, Blackboard ANZTLC15
•Transferir como PPTX, PDF•
2 gostaram•1,104 visualizações
Learning analytics is a 'hot topic' in education with many institutions seeking to make better use of the data available via various systems. One of the key challenges in this process is to understand the business questions that people working in various roles in institutions would like to be able to answer. However, it is also important that these questions are appropriately structured and specific in order to gather the relevant data. This session builds on the workshop run at last year's Blackboard Learning and Teaching conference where participants explored business questions and use cases for learning analytics from a range of perspectives. Delivered at Innovate and Educate: Teaching and Learning Conference by Blackboard. 24 -27 August 2015 in Adelaide, Australia.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Moving Forward on Learning Analytics - A/Professor Deborah West, Charles Darwin University and Ms Nicole Wall, Blackboard ANZTLC15
Semelhante a Moving Forward on Learning Analytics - A/Professor Deborah West, Charles Darwin University and Ms Nicole Wall, Blackboard ANZTLC15 (20)
Mais de Blackboard APAC
Mais de Blackboard APAC (20)
Último
Último (20)
Moving Forward on Learning Analytics - A/Professor Deborah West, Charles Darwin University and Ms Nicole Wall, Blackboard ANZTLC15
- 1. Moving Forward on Learning Analytics A/Professor Deborah West, Charles Darwin University Ms Nicole Wall, Blackboard
- 2. Today’s session • Brief background • Review the information gathered on the business questions from last year • Present an analysis of which of these questions can be answered via specific Analytics for Learn reports • Review a set of reports to check their validity in relation to the use case • Explore the development of ‘good’ questions which can be used to further refine reporting. 2
- 3. ‘Learning Analytics is about collecting traces that learners leave behind and using those to improve learning’ (Duval, 2012) 3
- 4. Our Focus • collecting traces that students and staff leave behind and using those to improve the student experience 4
- 5. Different types of data and affordances • Functionality of LMS • Analytics packages for the LMS • Data warehouses 5
- 6. Benefits of Learning Analytics Able to provide data to: • Create a more structured and effective intervention to improve student outcomes • Understand workload planning • Value add over time through student tracking • Improve support, target training workshops • Identify successful strategies and evaluate teacher effectiveness • Strategy and planning 6
- 7. Question Themes • Curriculum/course design and improvement • Student behaviour • Improve retention • Resourcing/planning at institutional level: • Academic actions/behaviour 7
- 8. Curriculum/Course Design Questions • How active are students in course content overall and at an individual content level? • What types of content are included in courses? How does this relate to what is most actively used and the results students achieve? • Can I couple student’s ratings of content/activities with their use of that content / activities with their results? • Is there a correlation between different course ‘styles’ (e.g. instructivist vs. constructivist) and student achievement within a given course or faculty? 8
- 9. Student Behaviour Questions • When and how are students engaged with the LMS and associated tools? – what times/lengths of day – spread of access throughout academic calendar, particularly key milestone dates? – what types of devices and where from? • When did a student last access email? When did a student last access Wi-Fi? • Is there correlation between messaging and test attempts? 9
- 10. Retention Questions • What kind of pre-enrolment interactions lead to “better” outcomes for our students? • Analytics for identifying, monitoring, managing and responding to students at risk. • What are the indicators? • Who are the most at risk? • What interventions are more likely to be effective? • What trends can we see in terms of at risk students 10
- 11. Institutional level planning/resourcing • Gauging impact of new initiatives/introduction of new technologies • Where are students connecting with “the university” in their learning journey? • Is the time and/or money/effect that has been put into something worthwhile? Are there particular things which are over or underrepresented? 11
- 12. Academic Behaviours • Which academics and which “schools” need support to become better/fuller users of Bb Learn? • What’s the frequency or % of academic attention per student within a subject? • How engaged are academics with the full Bb Learn toolset? • How much time instructors are spending and where the peaks of activity are? • Usability and Ease of Access by lecturers. 12
- 13. Miscellaneous Questions • Is this data: reliable, meaningful, and accurate? • If we have the “answer” are we morally, ethically and legally required to act? • How much inference can be drawn from the level of engagement a student has with an online environment? 13
- 14. Getting Answers from Analytics for Learn 14 Questions Themese #Qs Report Available Report Can be Built No Direct Match Curriculum/Course design & improvement 19 47% 21% 31% Student behaviour 7 42% 57% Improve Retention 10 100% Resourcing/planning at institutional level 8 38% 13% 50% Academic actions/behaviour 6 50% 33% 17% Questions from Different Perspectives 20 60% 40%
- 15. Writing good questions for your reports • Focus on Questions That Are Aligned With Your Business Strategy • Keep your questions specific and related to opportunities for applying the insights • Be definitive - One of the most valuable questions asked is how can you engage your students better? But what is your definition of ‘engagement’. 15
- 16. Writing good questions for your reports 16 Grade & Outcome Results Course Attributes Student Attributes Final Grades Student Information System Reports Instructo r AttributesUser Activit y Data Course Item Data
- 17. Writing good questions for your reports • What are you trying to measure? – Interactions, Accesses, Minutes, Submissions • How are you trying to measure it? – Average, Percentage, Count • To what? – Blackboard, Courses, Items 17
- 18. Writing good questions for your reports • How do you want to filter or group the information – Course Attributes – Instructor Attributes – Student Attributes – Performance – Time Frame 18
- 19. Writing good questions for your reports 19 How many logins occurred in Semester 1 and 2 by Faculty Which Faculty had the highest logins in the last 2 semester How is engagement with Blackboard changing ✔ ✗ ✔
- 20. Activity • Refine the question – Be specific; incorporate the measures and dimensions that are available – Consider the purpose of the question – what you want to know and for what purpose – Might break down into more than one • On the butchers paper draw what the report would look like – E.g. graph, table, pie chart 20
- 21. But… “Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted.” Albert Einstein 21
Notas do Editor
- Course Design – hard to measure most of the questions e.g best way to structure the content –how do we measure this? Student Behavior – many questions outside the scope of A4L e.g. email, device, IP Improve Retention - Analytics can help to track indicators/those at risk/if interventions change behaviour but it does not define them Resource Planning – many questions were about how to use the data not what you want to answer
- Why and how to do it Specific - It has to be clear what the KPI exactly measures. There has to be one widely-accepted definition of the KPI to make sure the different users interpret it the same way and, as a result, come to the same and right conclusions which they can act on. Measurable - The KPI has to be measurable to define a standard, budget or norm, to make it possible to measure the actual value and to make the actual value comparable to the budgeted value. Achievable - Every KPI has to be measurable to define a standard value for it. It is really important for the acceptance of KPI’s and Peformance Management in general within the organization that this norm is achievable. Nothing is more discouraging than striving for a goal that you will never obtain. Relevant - The KPI must give more insight in the performance of the organization in obtaining its strategy. If a KPI is not measuring a part of the strategy, acting on it doesn’t affect the organizations’ performance. Therefore an irrelevant KPI is useless. Time phased - It is important to express the value of the KPI in time. Every KPI only has a meaning if one knows the time dimension in which it is realized. The realization and standardization of the KPI therefore has to be time phased.
- Why and how to do it Measures are the first step
- Why and how to do it Dimesnions help us to slice and dice te information
- Why and how to do it Dimesnions help us to slice and dice