Led by Rob Wyn Jones, consultant and Shri Footring, senior co-design manager - enterprise, both Jisc. With contribution from Rebecca Davies, pro vice-chancellor and chief operating officer, Aberystwyth University. Connect more in Wales, 7 July 2016

1. Implementing analytics
LearningAnalytics and Business Intelligence
7/8/2016 1

2. Session outline
»Introductions
»Overview of the Learning Analytics Service
»Next steps / get involved
»The user voice – Prifysgol Aberystwyth University
»Overview of the Business Intelligence project
»Next steps / get involved
7/8/2016 2

3. Learning Analytics

4. What do we mean by Learning Analytics?
»The application of big data techniques such as
machine based learning and data mining to help
learners and institutions meet their goals:
»For our project:
› Improve retention (current project)
› Improve attainment (current project)
› Improve employability (future project)
› Personalised learning (future project)

5. Effective Learning Analytics Challenge
Rationale
» Universities and colleges wanted help to get started and have access to a
standard set of tools and technologies to monitor and intervene.
Priorities identified
» Code of Practice on legal and ethical issues
» Develop a basic learning analytics service including an app for students
» Provide a network to share knowledge and experience
Timescale
» 2015-16 -Test and develop the tools and metrics
» 2016-17 -Transition to service (Freemium)
» Sept 2017 – Launch. Measure impact on retention and achievement

6. Toolkit and Community
»Blog: http://analytics.jiscinvolve.org
»Reports
› Code of Practice for Learning Analytics
› The current state of play in UK higher
and further education
› Learning Analytics in Higher Education:
A review of UK and international practice
»Mailing: analytics@jiscmail.ac.uk
»Network Meetings

7. Learning Analytics Architecture

8. Current Engagement
»Expressions of interest: 85
»Engaged in activity: 35
»Discovery to Sept 16: agreed (28), completed
(18), reported (17)
»Learning Analytics Pre-Implementation: (12)
»Learning Analytics Implementation: (7)

9. Technical Progression to date…
• Deployment of our LA Data ProcessingAgreement (DPA)
• Latest student data specification (UDD v1.2.4)
• https://github.com/jiscdev/analytics-udd
• 12-36 months of UDD (student) data + aligned Activity data (VLE,Attendance etc)
• OngoingTechnicalTrials: Learner RecordsWarehouse(s), Bb & MoodleVLE
plugin(s), UDD data validation/ APIs
• Predictive Model development – service pilots start Q4 2016
• BlackBoard Learn –VLE activity data plugin evaluation for historical data capture,
with Moodle equivalent
• Student App – Beta v1.0 due for release July 2016 (iOS/Android)
• Student App evaluations – currently being formulated for 3 HEIs

10. Press Coverage

11. Future Engagement/ get involved
From Sept 2016
»“ReadinessToolkit” with a diagnostic set of
questions and support materials leading to
implementation.
»Start-up guidelines to get ready for learning
implementation.
Further details will be announced via
analytics @jiscmail.ac.uk

12. The value of Learning Analytics
LearningAnalytics at Aberystwyth University

13. Monitoring Attendance
»Recording attendance prior to 2014 involved registers
and manual entry onto SAMS (in-house Student
Attendance Monitoring System)
»SAMS simply recorded attendance and produced
limited reports on an individual student’s attendance
»The Computer Science department developed and first
ran MOPS (Monitoring of Performance System) 2013/14
› MOPS used data from SAMS to produced reports of
students with poor attendance
› MOPS then managed intervention workflows (typically
meetings) and recorded outcomes
7/8/2016 13

14. Initial Focus
»Initial focus was on
› Monitoring attendance - seen as the key indicator of
engagement
› Enabling early intervention
› Focus on student retention
»Systems were developed in-house (SAMS, MOPS)
»An accurate personalizedTimetable was an essential
pre-requisite of this work
7/8/2016 14

15. Automatic Attendance Monitoring
»But - high manual overhead of recording attendance
»Investigated card readers. Problem = high unit cost
(£500+)
»2014 Computer Science dept developed prototype
proximity card reader
»2015 CS worked with AU IS to turn this into production
»2015-16 Deployed across core teaching rooms
»By Sept 2016 will be deployed across all teaching
rooms
7/8/2016 15

16. Attendance Stats (up to end Semester 1 2016)
7/8/2016 16
2015 2016
Semester1 Semester 2 Semester 1 Semester 2
Auto = collected via card swipe. Manual = entered via SAMS. Auto Manual Auto Manual Auto Manual Auto Manual
Attendance recorded - 285,963 - 203,228 343,815 48,437 - -
Not attended - 85,811 - 100,238 108,095 14,099 - -
Statistic Details

17. Attendance Stats
»More data now collected on attendance & less work
»reporting on attendance at the student, module,
scheme, and department level – new insights
»Also being used for
› Monitoring international students to ensure compliance
withTier 4 visas
› Monitor exam attendance
› Time and attendance to record some employee working
hours - replacing time-consuming timesheets.
7/8/2016 17

18. Components of the System
7/8/2016 18

19. 3 Pillars to Successful Implementation
»A University-wide attendance policy with the
expectation that students should attend all timetabled
sessions (introduced 16/17).
»Universal automated attendance monitoring points
to capture attendance data with minimal manual
intervention.
»Systems to monitor and report attendance, alongside
managing intervention workflows.
7/8/2016 19

20. Future
Aberystwyth’s involvement with the JISC project and future plans
7/8/2016 20

21. Why the JISC Learning Analytics Project?
»Investigated commercial suppliers. Issues with
specification and price
»JISC solution designed by the sector and for the sector
»Main motivators:
› Continuous quality improvement
› Improve retention (reduce drop out, improve completion)
› Improve the student experience / satisfaction
› Improve student outcomes (degree class)
› Personalization / Student engagement with their learning
7/8/2016 21

22. Work with JISC Project so far
»Providing data from
› Student record system – in-house ASTRA
› VLE – Blackboard
› Attendance monitoring – from SAMS
»Feeding in to JISC’s UDD data specification
»Plan to pilot the JISC tools 16/17 & provide feedback:
› Test predictive model
› Unicon Learning analytics processor and dashboards
› Student Success Plan (intervention management tool)
› Student Learning Analytics App
7/8/2016 22

23. Other Plans 2016/17
»The following are being deployed University-wide from
September 2016:
› Attendance monitoring points in all teaching locations
› Consistent attendance policy
› New personal tutor system – incorporating use of analytics
in discussions
› New version of MOPS – “Tutor dashboard” showing
attendance &VLE use
› Display of attendance data in the Student Record system
(web-based app) – see next page
7/8/2016 23

24. Attendance Data on Individual’s Student Record
7/8/2016 24

25. Future Considerations
› Post Brexit
– will we need more formal monitoring for more of our
students?
– Data Protection legislation = EU law
› HE Bill in England – delay?What does this mean for the
metrics we need?
› Relationships with Student Unions / Representatives – will
they continue to support or does political uncertainty =
uncertainty in their support?
7/8/2016 25

26. HESA and Jisc Business Intelligence

27. 27
About HESA

30. Heidi Plus
 The new business intelligence service for UK Higher Education
 Replaces Heidi (which will be decommissioned in November 2016)
 Launched in November 2015 offering:
 Improved data content and functionality
 Delivery of data sets through commercial data explorer tool
 New visualisations and dashboards
 New training programme and support materials
 Available to HE institutions with a full HESA subscription
 Over 80% of current Heidi subscribers have started the Heidi Plus
application process (40% completed)

31. Heidi Lab overview

32. Secure data processing environment
Technical infrastructure bound by legal agreements to ensure data and dashboards are secure

33.  Information improvement manager UEL with;
Kent, Middlesex, Brunel, Royal Holloway
 Strategic planning and BI manager Sunderland with;
Glasgow, Glasgow Caledonian, St Andrews, Sunderland
 Director of planning, Kent with;
Birkbeck,Cardiff, Oxford, Southampton, Southampton
 Strategic Planning Manager, MMU with;
Leicester, Leicester,Cambridge, Bishop Grosseteste
Winter teams

34.  Upskilling of staff resource across sector
 Opening up of collaborative relationships
across other organisations
 Value, saving and efficiency gains from the
creation and delivery but also the actions
subsequently taken due to the insights gained
across research, student, staff and estates and
possibly internationally
 Opening up access to disparate data sets and
making sense of them in an HE context
 Possible national licensing deals for paid
access to data
Team member experiences

35. Team Laura – Q & A
Claire Daniells – University of Plymouth
Frances Leach – MMU
Natalie Butler - Leeds Beckett
NicolaWitts – MMU
Rhodri Rowlands – Sheffield Hallam
Shri Footring – JISC
ScottWilson – JISC
Laura Knox – St Andrews

36. User Stories
I want to: Understand the
destinations of my students post-
graduation (in particular further
study and employment)
So that I can: ensure the credibility
and sustainability of our curriculum
I want to: understand the
demographics of students
who progress on to further
study
So that I can: better
understand the quality and
demography of students
applying to PGT level study
at my or competitor
institutions
I want to: understand the
geographical locations of my
graduating students who enter
employment
So that I can: ensure the curriculum
is adding value and is credible in the
context of the relevant labour
market (local/national etc).
I want to: understand the gaps in
the labour market (local,
national and international)
So that I can: ensure the
curriculum is adding value and is
credible in the context of the
relevant labour market
(local/national etc).

37.  White Paper andTEF
 Graduate employment
 Highly skilled employment
 LEO dataset
 New DHLE
 Considering the need to “understand graduate migration
in greater depth, including the wider social impacts of
graduates and travel to work patterns”.
 Linked data
Additional Context

38.  DHLE Data
 NOMIS (Official Labour Market Statistics)
Possible addition:
 IPPR Burning Glass -Wheretheworkis.org
 Association of Graduate Recruiters
Data Sources

43. Dashboard: Course Market Analysis for Institutions
What is it? An Overview Movie
Purpose:
This dashboard is designed to support a university’s strategic planner in
designing course by allowing comparison across the sector.
Use case:
As a Strategic planner when working out which courses to teach I want
to examine competition to my course offerings to ensure I target
recruitment activity most effectively.
Data sources:
National Pupil Database: http://bit.ly/224CU8I
Key Information Sets: http://bit.ly/1ZYnG5z
National Pupil Database: http://bit.ly/224CU8I
HESA Data
What needs to be done and issues Time and Effort to Market
Where there is scope for improvement:
• Generally very polished
• Some work on the interface required perhaps to sign-post the features
• Licencing issues for league table data need to be negotiated.
• Data sources would need updating each year – particularly the school data.

44. Dashboard: University Finder for Students
What is it? An Overview Movie
Purpose:
This dashboard is targeted at students who are looking for a university course to
fit their needs. By needs we don't only mean course but also: cost,
employability, location and entry tariff.
Use case:
As a student when working out which university course offers best fit my needs,
I want to understand factors of relevance to me (course, cost, employability,
location, cost of living, rural/urban and entry tariff) to compare and match
offers to my circumstances.
Data Sources:
Key Information Sets: http://bit.ly/1ZYnG5z
HESA Data
What needs to be done and issues Time and Effort to Market
This dashboard supplies a unique perspective on data and services that are already available to
students. In some ways this is a crowded marked. So the unique selling point of this product
would need to be promoted – that is that the data already available to students is amalgamated
and drawn together to create a” wizard like app” for students to find courses.
What would need to be done:
• Identify appropriate vehicle for delivery
• Market uniqueness of the the product
• Negotiate data licences for league table data

45. Dashboard: Finding Comparable Institutions
What is it? An Overview Movie
Purpose:
This dashboard can be used to identify a university’s relative performance
against a benchmark of similar institutions.
Use case:
As a Planning Manager I want to select similar institutions based on metrics I
choose so that I can determine the best institutions to compare with my own
university to understand if our performance is relatively good or bad
Data Sources:
HESA data from Heidi
Key Information Sets: http://bit.ly/1ZYnG5z
League Table Data – will require licensing
What needs to be done and issues Time and Effort to Market
Where there is scope for improvement:
• Data – a relatively narrow data set was used for prototyping; a production version could
accommodate a far more comprehensive data set.
• Filters – searching and filtering could be enhanced
• Licencing – Makes use of some league table data to benchmark against entry tariff.
Licence for this need to be negotiated.

46. Library Data Labs
 Teams working on Library BI Stories at 0.2 FTE, total estimated
effort 15 days from July - Oct 2016
 Both Product Owners and Sector Data Experts invited:
 Product Owner from the sector to steer which stories are of interest
 Sector Experts to understand what data sources are available & what is in the data
 Jisc Contracted Data transformation specialist
 Jisc Agile Scrum Master &Tableau User
 Teams receive experience and guidance of Agile working
 Option forTableau Desktop training to help with creating
visualisations
7/8/2016 46

47. Analytics academy – a Jisc beta service - October 2016
» Business intelligence offers value, savings and efficiencies to
Universities through data informed enhanced planning / decision
making
» Many problem spaces are commonly felt, while the data landscape to
support insights is vast.
» Some universities have little access to good BI at all, while those with
capability are often duplicating effort.
» There is no higher education focused CPD offer to train up BI expertise.
» Analytics academy addresses these problems by providing expertise
and tools for analysts (planning officers and others) to identify suitable
problem areas (student, staff, research, estates etc), exploring the data
landscape for insights and producing interactive dashboards for the
sector
7/8/2016 47

48. Keep in touch
»http://www.business-intelligence.ac.uk
»Subscribe via www.jiscmail.ac.uk/JISC-HESA-
BUSINESS-INTEL
»Twitter @HESA @jisc #hesajiscbi
7/8/2016 48

49. Questions?