1. 1
EARLY WARNING SYSTEM
AND PERFORMANCE
DASHBOARD
June 2012 Subtitle

2. Outline of session
2
 My experience
 Monitoring students and engaging with large
numbers
 The problems
 The context
 Potential
management and engagement tools:
EWS and Performance Dashboard
 Examples from NUR219 Learnline site
 Discussion/questions
June 2012

3. My Experience: Self-portrait (circa 2009)!
3
June 2012

4. The Problems: NUR219 Mental
4
Health Nursing
 Disconnection
 Unable to establish effective interpersonal relationships with students
 Minor unit management errors on my part became exponentially
magnified through the effect of high volumes, technology, and isolation
from campus activity
 Complexity of teaching across multiple modes increases risk of
confused information
 Non-traditional student cohort
 NUR219: 367 students; 11% internal, 1% full-time (“Area 52” data, 21st
May)
 Unable to access traditional feedback from students as a
check
 Minimal ability to obtain direct feedback from students; difficult to
evaluate the quality of feedback
 Difficult to set up expectations and norms as these are not set by on-
campus socialisation or effective face-to-face routine contact. We now
compete with other powerful expectation-setting contexts (e.g.
employers, past education experiences, workplace inservices, etc.)
June 2012

5. Feedback, Engagement, and
5
Management
 Lecturer -> -> Student (large
no’s)
 Students -> -> Lecturer
 Student -> -> Peer Community
My view of
NUR219 students
June 2012

6. “Invisible” online students
6
 Students who are losing track or not-
progressing for other reasons are not detected
in a timely manner.
 Leads to a high risk of failure
 Which then leads to students blaming a lack of
support, guidance, confusion, etc.
June 2012

7. Identifying students who need
7
support
Online
 Early Warning System
 Performance Dashboard
“My maths teacher” (circa
late-ish 1900’s)!
 Track progression data, then
 Assertively provide help and support
(asynchronously).
June 2012

8. Unit Design Considerations
8
 Tracking progression involves decisions about
what “items” that will generate the data needed.
 This decision will guide how to set up learning
materials and assessment items.
 EWS and Performance Dashboard…
 uses two main types of data: Review Status (Learning
Materials) and Due Date Status (Assessment Items)
 allows the Unit Coordinator to interrogate students’
progress
 the unit coordinator can then identify significant
groups of students and respond en masse
June 2012

9. Data-driven Unit Design
9
 Early Warning System
uses “due dates” data from any assessment
items. Notifications can be sent x days before
or after a due date, or if poor unit access, etc.
 Performance Dashboard
uses “Review Status” data from content items.
The number of items marked as Reviewed by
the student can be observed.
June 2012

10. EWS example
10
 Example: Notification to students with late
attempt at Week 1 self-assessment quiz.
June 2012

11. Performance Dashboard
11
example
 Example: Identifying students who have not
reviewed Learning Materials.
June 2012

12. Unit Design: Thinking About
12
Data
W Topic (Review items) Self-assessment Assessment
k (due date items)
1 Orientation to NUR219 Quiz 1
2 Theory as a basis for practice Quiz 2
3 Neuroscience as a basis for Quiz 3
practice
4 Tools of psychiatric nursing Quiz 4 Census date
5 Culture and ethnic Quiz 5 Assignment 1
considerations
6 Law, ethics and mental health Quiz 6
7 The client experiencing crisis / Quiz 7
anxiety
8 The client experiencing Quiz 8 Assignment 2
schizophrenia
June 2012
Etc…

13. Progress and Performance Data
13
for NUR219
 weekly self-assessment quizzes with
automated marking and feedback
 provides weekly “due date” data for the EWS
 can also develop an EWS notification for poor scores if you wish!
 weekly Learning Modules
 provides a sequential weekly structure
 With Review Status and Adaptive Release
switched on to generate “Reviewed” content data.
June 2012

14. Example: Early warning
14
notifications
June 2012

15. Early Warning System
15
 A student progression management tool, but
EWS Notifications are a mechanism for
“assertive engagement” with students
June 2012

16. Performance Dashboard: Tracking
16
progress
 Example:
 Sort
on Review Status column (starting at zero)
 Show all
June 2012

17. NUR219 Example: Tracking
17
progress (Review status)
June 2012

18. Example: Tracking progress
18
student detail
June 2012

19. Communicating expectations
19
 Monitoring progression; who is falling behind
and who is becoming “High Risk”?
 Assertive engagement using notifications
generated from EWS
 Frame up helpful notifications that express clear
boundaries
 Audit trail
June 2012

20. Example NUR219 notification to
20
students from EWS
June 2012

21. Example NUR219 notification
21
(absence from unit)
June 2012

22. Example NUR219 notification
(insufficient progression through
22 Learning Materials)
June 2012

23. Example NUR219 notification
(insufficient progression through
23 Learning Materials)
June 2012

24. High Volumes
(Advanced Performance
24
Dashboard)
 Can’t send notifications directly from Performance
Dashboard
 Use Grade Centre to send notifications if needed to
appropriate students en masse
 Copy Dashboard to Excel for advanced filtering and
format as a table
 In Dashboard, select Show All.
 Select all the cells in the Dashboard table including column
headings.
 Use “Paste Special” (choose Unicode) to dump data into
Excel sheet.
 Then Format as Table in Excel
June 2012