FINALIZED SENIOR SEMINAR PAPER (1)

Roles of Data Analytics in Improving Student Outcomes
in Higher Education
A Course Project Presented To
Dr. Ajani
Bachelor of Science
In
Computer Information Systems
As a Partial Requirement for CSC 498 – Senior Seminar
By
Kyle Price
May, 2016

Data Analytics in Higher Education 1
Table of Contents
Table of Contents....................................................................................................................1
Abstract...................................................................................................................................3
Dedication...............................................................................................................................4
Acknowledgements.................................................................................................................5
Introduction.............................................................................................................................6
1.1 Rationale........................................................................................................................6
1.2 Framework....................................................................................................................7
1.3 Problem Leading to the Project....................................................................................10
1.4 Importance of the Problem...........................................................................................12
1.5 Purpose of the Project...................................................................................................14
1.6 Research Questions.......................................................................................................14
1.7 Objective.......................................................................................................................14
1.8 Scope of the Project......................................................................................................14
1.9 Limitations ....................................................................................................................15
Literature Review....................................................................................................................16
2.1 Data Analytics in Higher Education..............................................................................16
2.2 Three Domains of Analytics .........................................................................................17
2.3 Current Use of Data Analytics in Higher Education.....................................................18
2.4 Data Governance...........................................................................................................20
2.5 Implementation of Data Analytics in Higher Education...............................................21
Methodology...........................................................................................................................24
3.1 Introduction...................................................................................................................24
3.2 Research Design............................................................................................................24
3.3 Description of Research Population and Participants...................................................24
3.4 Research Instruments ....................................................................................................25
3.5 Procedures ....................................................................................................................25
3.6 Data Analysis ................................................................................................................26

Data Analysis and Discussion.................................................................................................27
4.1 Issues with Access to Data...........................................................................................27
4.2 Lack of Understanding How to Use Available Data....................................................28
4.3 Lack of Collaboration....................................................................................................29
4.4 Data Usage....................................................................................................................30
4.5 Discussion of Findings..................................................................................................30
4.6 Usefulness of the Study................................................................................................34
4.7 Final Remarks...............................................................................................................35
References...............................................................................................................................36
Key Terms ..............................................................................................................................39
Appendix A............................................................................................................................41
Appendix B ............................................................................................................................42
Appendix C ............................................................................................................................50

Abstract
As college becomes more expensive over time, students gaining the most out of there
education is a major concern for any student looking to continue their education. The overall
purpose of this study was to analyze the benefits of implementing data analytics into higher
education. This study took an in depth look at the advantages, challenges, and tasks involved in
implementing data analytics. Also included is information about what student data is most
valuable to an institution and how that data is used for the improvement of student outcomes.
The data collected and analyzed from research, focus groups, and personal interviews were the
basis of this study.

Dedication
For Betty Ann Barnes, Greg Price, Mark Barnes, and the rest of my family. I am
privileged to have a family that is fully supportive of my goals and that is proud of what I have
been able to accomplish. None of this would have ever been possible without their support and I
am forever grateful for the amazing family I was fortunate enough to be born into.

Acknowledgements
First and foremost I would like to thank my family and friends. This research study
would not have been possible without the hard-work, support, and encouragement that I have
received from these amazing people. Upon completion of this proposal, I have become more
aware of all that my parents and my family has had to sacrifice in order to make sure that not
only I was able to attend college, but also to make sure that I was able to succeed. I cannot put
into words how appreciative and thankful that I am for all that they have done for me. One day,
hopefully, I am able to show them how much I truly value everything that they have done.
I would like to personally thank Dr. Taiwo Ajani for his feedback, guidance, and
mentoring throughout this course, throughout this study, and throughout my undergraduate
career. I would also like to thank the academic faculty here at Ferrum College. The professors
here are extremely dedicated to their students and are more than willing to go out of their way to
help. It has been an honor to have all of you as a teacher. I am forever grateful for the knowledge
and personal experiences I have gained while being in contact with the professors here at Ferrum
College. Thank you for everything.

Chapter 1: Introduction
1.1 Introduction - Rationale
This paper will provide an insight into the benefits and risks of using data analytics to
improve student outcomes in higher education. Data analytics encompasses a wide variety of
terms and also has many different definitions that are possible. A broad definition, for the sake of
this paper, is using the plethora of current and historical data available about students to
influence decisions for the improvement of student outcomes. The National Center for Education
reports that the 2013 6-year graduation rate for first-time, full-time undergraduate students who
began their pursuit of a bachelor’s degree at a 4-year degree-granting institution in fall 2007 was
59%. The National Center for Education Statistics also states that average annual student loan
amounts for first-time, full-time degree/certificate-seeking undergraduate students receiving
loans in 2012-2013 was $7,000. A student’s average debt amount after graduation would be
approximately $28,000 and the current total national debt for students is over $1.2 trillion
(NCES, 2015). Data analytics techniques could be implemented to ensure students a maximum
return on their investments, increase graduation rates, improve student outcomes, and improve
their experiences with higher education overall.
Picciano (2014), while quoting an IBM study, stated that there are 8 categories of
possible instructional applications utilizing analytics: monitoring individual student performance,
disaggregating student performance by selected characteristics, identifying outliers for early
intervention, predicting potential, preventing attrition from a course or program, identifying and
developing effective instructional techniques, analyzing standard assessment techniques and
instruments, and testing and evaluation of curricula. With all of these possible benefits, why is it
that data analytics is something that is not implemented across the United States? Data analytics

in higher education is more of an exploratory topic right now that people are still trying to learn
more about.
There are factors that determine the effectiveness of data analytics to improve outcomes
such as the way the class is taught or what data is readily available to be accessed. If a teacher
chooses not to use a learning management system, such as Brightspace that we use on campus,
and sticks strictly to print offs and classwork it is hard to collect data on students. In order for
this method to be successful, teachers would need to use a combination of a learning
management system and also classwork. The availability of data is also important to the success
of data analytics. A lot of universities like to keep their data in as few hands as possible, for
many reasons, and that can also impede an institutions ability to have an effective data analytics
program.
Using a learning management system allows teachers to track absences, when students
access files they post, how often they access the files, and many other things that can all be used
to help improve outcomes. Teachers could begin to notice patterns such as students who login
and open assignments shortly after they are posted tend to have higher grades in the classroom.
An online system is vital to the success of data analytics and using big data to improve the
outcomes of students. The use of these techniques could revolutionize learning and create a
smarter society as a whole.
1.2 Framework
Anthony Picciano, during his graduate research on a similar topic wrote, “Big data
concepts and analytics can be applied to a variety of higher education administrative and
instructional applications including recruitment and admissions processing, financial planning,
donor tracking and student performance monitoring” (Picciano, 2014, p. 38). Picciano also

quoted the 2014 New Horizon Report in his 2014 study which states that integration of blended
and collaborative learning and the rise of data-driven learning and assessment are in the top 3
emerging technologies over the next five years. Although the use of big data is not popular, there
are universities that have already begun to implement it and reap benefits such as Purdue
University.
Purdue University developed a software called Course Signals which was “developed to
allow instructors the opportunity to employ the power of learner analytics to provide real-time
feedback to a student” (Arnold & Pistilli, 2012, p. 1). Purdue uses an algorithm, called the
Student Success Algorithm (SSA), to support Course Signals. The purpose of the algorithm is to
create a predictive model for predicting student success. The SSA consists of 4 components
which are performance, prior academic history, effort, and student characteristics. Based on the
results the student is given either a green, yellow, or red light to show a measure of performance
on their course homepage. A red light means there is a high likelihood of being unsuccessful, a
yellow light means that there is a potential problem of succeeding, and a green light means that
the student has a high likelihood to succeed in the class. This helps determine if action needs to
be taken to prevent failure and allows teachers to implement an intervention schedule to ensure
those students who are given a yellow or red light can succeed in their classes (Arnold & Pistilli,
2012).
Overall, the study by Arnold & Pistilli (2012) of Purdue’s use of learning analytics
showed an improvement in grades. There was over a 10% of the grades being A’s or B’s. Not
only did the number of overall A’s and B’s in courses increase, there was also a decrease in the
number of C’s, D’s, and F’s. The study showed over an 8% decrease in C’s and over an 8%
decrease in D’s and F’s. This shows that using methods of business intelligence in higher

education can improve student outcomes. Brightspace, Ferrum College’s current learning
management system, lacks a lot of the analytical features that are geared toward benefiting both
students and faculty. Brightspace has additional learning analytics software that colleges are
capable of upgrading to, but, the current system that Ferrum College has now does not allow for
much useful data to be collected. The additional learning analytics software tracks more data in
greater detail. An example of the analytics at work in the current system that we is the Retention
Alert System. The Retention Alert System automatically sends an e-mail to students if they are
flagged by their instructors on the online Portal. If a student’s grades are seemingly on a fast
decline or he/she is missing an excessive number of classes are examples of scenarios an alert
would be sent. This alert, or “academic warning,” is sent to them by e-mail with suggestions to
meet with their advisors and/or teachers to discuss possible learning alternatives.
Implementing data analytics into higher education does not happen for free though,
unfortunately. According to the United States Department of Education (2012), collection,
storage, development of algorithms, and interoperable administrative and learning systems are
examples of some of the necessary hardware and software costs. Not only is it necessary to buy
software that is capable of generating the information that can be utilized, but you also need
employees who know what they are doing. Given the recent nature of the field of data analytics,
it would be necessary to have employees trained to properly use the software and also trained
how to effectively read available data and run tests to determine what pieces are useful.

1.3 Problem Leading to the Project
In 2010, The National Center for Higher Education Management Systems reported that
the average retention rate for students in universities in Virginia was 78.6% (NCHEMS, 2016).
The U.S. News & World Report reported that the retention rate for Ferrum College in 2014 was
51% (2010). Although the accuracy of this data is unclear, it is not something that is
unbelievable. From personal experience, it is hard to disagree that the actual retention rate would
be much higher than that.
Oral Roberts University is an example of a success story that is in a similar situation as
Ferrum College. Both are small, private liberal arts schools and both struggled with retention
rates, but Oral Roberts University took action to increase its retention rate. Using a combination
of both Brightspace Advanced Analytics solution and Brightspace Student Success System, Oral
Roberts University not only improved its retention rate but also the success of their students
increased. Michael Mathews, the chief information officer at Oral Roberts University, stated that
after just one semester, ORU was able to develop a clearer picture of its student persistence and
retention rates. According to Mathews, the university has already seen its retention (persistence)
rate increase from 61% to 75.5% just by having accurate information at hand (Mathews, 2015).
Logically, an increased retention rate leads to an increased graduation rate. It is a win-win
situation for both students and for higher education facilities.
Data analytics in higher education has the potential to become an epidemic, which could
affect the lives of students now and in the future. Through a proper analysis of data, teachers
could learn what teaching methods are most effective in their classrooms or what students
needed the most attention, admissions would be capable of recruiting students that are the right
fit for their school, and administration could form new programs to benefit students and help

improve their outcomes. A student could be automatically flagged for teachers when he/she logs
in to his/her respective learning management system that would allow timely intervention for
help. This would allow teachers to implement some sort of intervention schedule that could help
ensure the success of students.
The problems that can be addressed are not limited to student outcomes, retention rates,
and graduation rates. Another problem that can be addressed is broader than just those things.
Applying our knowledge and technology to learning gives us an opportunity to know the best
methods to learn to ensure the most information is retained. Using this data can help create a
well-rounded, educated foundation that can will only lead to society becoming more intelligent
as a whole. All of that starts with education. There have been plenty of students who needed that
extra push during their freshman or sophomore years who went unnoticed and decided to call it
quits. Students are likely too afraid to reach out on their own and end up dropping out and giving
up, but that is a problem that can be helped with data analytics in higher education. Dr. Jaclyn
Broadbent, Lecturer in Health Psychology at Deakin University stated that using analytical
techniques, she was able to identify some of these students. “Sometimes students just need
someone to notice,” she says. “It’s so rewarding for me when I have been able to target someone
possibly ‘at risk’ of dropping out early in the semester, and then reading their assessments pieces
at the end when they’ve come so far” (Broadbent, 2015, p. 4).
In the majority of cases where students went unnoticed, teachers most likely did not have
any insight on how to connect with the struggling students or if they really needed help. They did
not have access to data that could be analyzed to reveal what the most effective teaching method
was for their class to be sure everyone was able to get the most out of the material. The use of
data analytics techniques in higher education has the potential to unlock an abundance of

information about student learning. The possibilities for improving student outcomes seem
limitless. At the end of the day, the better student outcomes are, the more students are able to
take what they have learned from their educational experiences and transfer it into the real world.
Students are able stay on the path to success because teachers are able to analyze the big data that
is collected to become more knowledgeable of student and classroom needs. The more students
we have graduating, the more intelligent we become as a society.
1.4 Importance of the Problem
Implementing data analytics techniques into higher education is important for many
reasons and is capable of helping to solve several issues within higher education facilities.
Primarily this is a topic of importance because of the impact it will have on student outcomes.
Identifying outliers and knowing when it is necessary to intervene can make the difference in
whether a student decides to return to college and continue his/her education or whether or not a
“B” becomes an “A” in the final class grade. Universities such as Deakin University are a prime
example of how using these techniques can improve retention rates. Ever since implementing
analytics into the university, retention rates sit at an impressive 90% (Broadbent, 2015).
Considering the advantages and the recent emergence of data analytics in higher
education, it is safe to assume it will become more widely used. It is important to know the
advantages, but it is also important to know that there are risks and legalities involved. The most
important thing a higher education facility should have on its agenda when considering these
techniques is to be sure it is completely transparent when it comes to the data it is going to
collect, and it must also performing a risk analysis. Students have the right to know what data is
going to be collected and how it is going to be used, and they also have the right to know that

their data is safe and is not going to be redistributed in any way without their consent. A risk
analysis is important to be sure everything is in compliance with FERPA, HIPAA, and GLB
requirements. Randy Stiles, in 2012, during his research quotes a draft guidance on risk analysis
from the Office of Civil Rights, which states “Conducting a risk analysis is the first step in
identifying and implementing safeguards that comply with and carry out the standards and
implementation specifications in the Security Rule” (Stiles, 2012, p. 21).
Being able to analyze the way students learn and unveiling the most effective methods of
teaching is knowledge that many generations to come can benefit from. In his 2014 study,
Picciano stated
By linking CMS/LMS databases with an institution’s information system, data can be
collected over time. Student and course data can be aggregated and disaggregated to
analyze patterns at multiple levels of the institution. This would allow for predictive
modeling that in turn, can create and establish student outcomes alert systems and
intervention strategies (Picciano, 2014, p. 41).
This is important because as a part of the generation of the “technology boom,” classrooms have
become more commonly integrated with an online learning management system. If learning
management systems are going to be used regardless, it would be a grave oversight to not make
use of the abundance of data that is collected and using it as an advantage towards benefiting
student outcomes.

1.5 Purpose of the Project
This leads to the purpose of the exploratory study, which will assess the effectiveness,
advantages, and risks of implementing data analytics techniques into high education to improve
student outcomes. The study will describe and compare the methods of use and the outcomes of
universities all around the globe who have implemented these techniques. The validation of the
results will be derived from research completed by individuals or universities who have
published and established results about their experiences with data analytics in higher education.
1.6 Research Questions
 What are the most important student data collected by academic institutions?
 Which of the data types are directly associated with student outcomes and what roles do
they play in student outcomes?
 How can data analytics benefit admissions or administration of a college or university?
 What are the major tasks involved in implementing data analytics into higher education?
 What are the challenges of implementing data analytics into higher education?
1.7 Objective
The intent of this exploratory research was to determine if implementing analytical
techniques as a standard in higher education would have an impact on student outcomes.
1.8 Scope of the Project
This study will be confined to the input of Ferrum College employees and a former lead
programmer, at a different university.

1.9 Limitations
The major limitations of the project are time and access to data. The amount of data
Ferrum College allowed to be accessed was limited. Some information can legally not be
released. The findings in this study may be subject to other interpretations.

Chapter 2: Literary Analysis
2.1 Data Analytics in Higher Education
Higher education is one of the most important opportunities in this world. In the 21st
century, there is an opportunity to extend our education and learn about a field of study that truly
interests us. Higher education teaches us critical thinking, provides us with skills on how to
properly research, and also trains us how to use those skills towards accomplishing whatever
goal we seek to achieve. Clearly, higher education is an important foundation for scholars and
should be implemented in the most efficient way possible. Data analytics is a way to ensure that
higher education facilities are running in the most efficient manner and students are receiving the
best possible outcomes. Picciano (2014) quoted a study by IBM which said through tracking
student performance either individually or based on a selected characteristic, it is possible to
determine outliers, predict outliers, prevent course attrition, identify and develop methods of
teaching, analyze current methods of teaching, and gather an evaluation of curricula (Picciano,
2014, p. 5). The ability to track student performance is an important factor when it comes to data
analytics in higher education.
What are ways that big data is being used for the benefit of student outcomes,
admissions, and administration of an institution? There are several examples of the techniques
that can be used to get the most out of the data that is collected. One technique is predictive
modeling. Predictive modeling is basically taking historical data that is collected and using that
to attempt to mathematically find a relationship between the dependent and independent
variables to predict future situations and trends (Dickey, 2012). For example, a university could
look back at all of the grades that students have received in a specific class over the years or the
number of students who have passed it. If the results from this historical data show that students

tend to generally struggle in a course, the university then knows that action needs to be taken to
improve student outcomes for that course. However, predictive modeling is a lot broader than
just focusing on student outcomes. It can also help the admissions office predict how many
students will enroll in a specific semester and how many students will transfer, among many
other things. Having this information at hand is important to a university so it can do budgeting
or housing. This just goes to show that data analytics in higher education can not only be used to
improve student outcomes but also to improve efficiency within admissions and administration.
2.2 Three Domains of Analytics
According to a study done by Adam Cooper (2012) there are three broad domains of
analytics. The three domains are the domain of the teacher and learner, the researcher, and the
domain of management and support services. Within the domains listed there are examples of
certain aspects that can be improved by implementing data analytics. The domain of the teacher
and learner gives insights to support educational aims and objectives such as course quality
enhancement, assessment efficiency and reliability, and online learning environment use. The
domain of the researcher provides insights to support the development of research proposals and
achievement of impact. Some examples of data analytics being put to use in this domain would
be for research evaluation, reputation/impact management, and bid targeting. The domain of
management and support services provides insight to support operational and strategic activity.
Examples of data analytics being put to use in this domain are energy efficiency,
funding/costing/fee-setting, and estate utilization.

2.3 Current Use of Data Analytics in Higher Education
Several universities, such as Purdue, have implemented their own techniques of data
analytics into their day to day operations. Purdue invented an analytical system called Course
Signals, which is a student success system based on predictive models (Arnold & Pistilli, 2012).
According to Arnold and Pistilli, over 24,000 students and more than 145 instructors have used
Course Signals in at least one of their courses. Course Signals uses the plethora of data that is
available through an institution about students to determine if a student is at risk. The algorithm
used to determine the level of risk looks at 4 different parts which are performance, effort, prior
academic history, and student characteristics. The performance aspect looks at the student’s
grade in the course to date. Effort is measured by evaluating students’ usage of Purdue’s learning
management system, Blackboard Vista. Prior academic history is measured by looking at high
school GPA and standardized test scores. The student characteristics that are also a part of the
algorithm include, but are not limited to, residency, age, and credits attempted (Arnold & Pistilli
2012).
Purdue named the algorithm that they created the Student Success Algorithm, or, SSA.
After factoring in all four components defined in the above paragraph, the algorithm calculates a
result and gives the student a red, yellow, or green signal. A red light is a student that is highly
likely to be unsuccessful, yellow is a student that seems to have potential to be unsuccessful, and
green means that the student is highly likely to succeed in the class. Instructors then create
something that they call an “intervention schedule,” which is where they decide the best plan for
intervening and ensuring the student is successful (Arnold & Pistilli, 2012). The study done by
Arnold and Pistilli proves that using data analytics and creating the Course Signals had a
significant impact on student outcomes. Overall, there was a ten percent point increase in A’s

and B’s in courses that implemented the Course Signals while also a six percent decrease in D’s
and F’s.
An improvement in student outcomes is not the only thing that resulted from the
implementation of Course Signals. Retention rate also saw a significant improvement. The study
by Arnold and Pistilli (2012) was done on the 2007, 2008, and 2009 cohort of students at Purdue.
The results were based on the number of Course Signal courses that students enrolled in.
Students that were enrolled in no courses that used Course Signals for the 2007 cohort had a
retention rate of 83% in year one and a retention rate of 69% by year four. Students who were
enrolled in two or more courses that used Course Signals during their cohort had a 97% percent
retention rate in year one and a 93% percent retention rate in year four. The more courses that
used Course Signals, the higher the retention rate was.
Oral Roberts University is also an example of a university that had success implementing
data analytics. Oral Roberts University did not use Course Signals, but instead used the
Brightspace Advanced Analytical solution. The goal was to improve retention rates, and they
were able to do so in just one semester; in fact, retention rates increased at the university by over
15%. According to Dr. Berchenbriter of Oral Roberts University, instructors can see much more
when using data analytics. He states that the data can show if a student struggles specifically on
tests compared to written assignments (Mathews, 2015). This allows instructors to target students
to find out why they are struggling and allows them to provide help to the students. When
instructors are able to intervene, students are more likely to be successful and will also be more
likely to stay at the university. Using data analytics also helps promote relationships between
students and teachers. When teachers are constantly in contact with students and tracking their
progress, it helps build a relationship between students and teachers.

2.4 Data Governance
There is proof that data analytics is beneficial when implemented into higher education,
but there are also risks and rules that need to be followed when collecting mass amounts of data
from students. Data governance is ensuring that all of your data is controlled and managed
properly. Stiles (2012) quoted the “data governance checklist” that was developed by the United
States Department of Education’s Privacy Technical Assistance Center in his study which breaks
data governance down into 7 main categories. The categories are decision-making authority,
standard policies and procedures, data inventories, data content management, data records
management, data quality, and data access.
Decision-making authority is the assignment of the correct levels of authority to those
who overlook the data. It also defines the scope and limitations of those that are able to access
and use the data. The standard policies and procedures category implements procedures and
policies and involves writing a plan to make sure everyone in the organization knows the
significance of the quality and security of data and that all of those involved are determined to
enforce data governance. As its name suggests, data inventories encompasses taking inventories
of large amounts of data that requires protection and classifying it by sensitivity, so upper
management knows where to focus security concerns. Data content management is the
management of the content of the data and determining the purpose of the data being collected to
justify the use of data that is sensitive. It also involves developing processes for managing data,
and assuring the federal, state, and local regulations are being followed. Data records
management is identifying the way managers and users handle data, ensuring that users have the

correct tools to follow an organization’s security policies. Data quality is making sure that data
are detailed, important, timely, and complete for their intended uses. Data quality also
encompasses constantly updating procedures for prohibiting, exposing, and amending errors and
exploitations of data. Data access is appointing different levels of data access to people in an
organization based on their roles in the organization in order to remove the potential for
illegitimate entry to data sets and to reduce the chance of a breach of data.
2.5 Implementation of Data Analytics in Higher Education
Data governance is very important when it comes to the implementation of big data
analytics. Issues with data governance are not the biggest barrier to the use of the analytics. Steve
Lavalle and his associates at MIT Sloan Management Review partnered with the IBM Institute
for Business Value and conducted a survey of approximately 3,000 executives, managers and
analysts working across more than 30 industries and 100 countries. The survey results displayed
below in figure 2.1 show the top responses for the impediments to becoming more data driven.

Figure 2.1. The Impediments to Becoming More Data Driven
(Lavelle, 2010)
Two of the top three impediments from this survey are related simply to knowledge of
the use of data analytics. In fact, a majority of the impediments listed here are related to
knowledge. So before worrying about data governance or other things that are involved in big
data analytics, it is important to inform the members of faculty exactly how it can be used and
the advantages it can provide. There are several ways that you can inform the right people about
the advantages of data analytics and how to use it. The institution could host a workshop for
those who are interested in learning how it works, similar to how SAS came to Ferrum College
in January of this year, and again during Easter break, to teach us about their program and their

code. Another way to inform those in the institution could be to simply have someone who is
knowledgeable of the subject type up a small report to send in an e-mail to those who need to be
informed of the subject when the possibility of implementation of data analytics comes into play.
It could be explained in simplified terms, so there would be a general understanding of how data
analytics can improve student outcomes and also management of the institution in general.
People seem to fear the unknown, so it is important that someone is able to give insight on data
analytics and how it can be implemented to benefit an institution.
Let’s take a look at the bigger picture to further explain why data analytics is useful in
higher education. According to Sattinger and Ehrenburg (2001), tuition at private universities has
risen annually by two to three percent more than the rate of inflation. Simply put, college is
expensive, and students who are investing in an education expect the maximum return on their
investment. The problem with that is that students are not convinced that the rising costs implies
good student outcomes. In return, students struggle to find answers to questions they have and
they either enroll anyway to get their education, or they decide not to go because it is not worth
the investment to them. This is where data analytics can come into play. Universities can use
data analytics to target specific students that they know, or have reason to believe, will be able to
succeed at their school. Once a university is able to target students that would fit in with the way
things operate at that specific school, it is able to further implement data analytics in the
classroom. If students know they’re specifically being targeted to come somewhere because
there is data to prove that they would be highly likely to succeed they are going to be more likely
to continue their education and come to college.

Chapter 3: Research Methodology
3.1 Introduction
The purpose of this methodology section is to answer a few questions that guided this
research: What are the most important student data collected by academic institutions? Which of
the data types are directly associated with student outcomes and what roles do they play in
student outcomes? How can data analytics benefit admissions or administration of a college or
university? What are the major tasks involved in implementing data analytics into higher
education? What are the challenges of implementing data analytics into higher education?
3.2 Research Design
This qualitative research was conducted through 2 separate focus groups at Ferrum
College and a personal interview. There were 6 participants in total that consisted of current
professors, administrative staff, and a data analytics expert. A focus group is a discussion in
which the researcher chooses topics and participates in an in depth discussion with participants.
Participant 1 was the participant in the personal interview. Participant 2, 3, and 4 were the first
focus group participants and consisted of 3 faculty members who would currently be considered
“super users” of our current learning management system, Brightspace. A “super user” is a user
that extensively uses Brightspace as a medium for assignments and communication with
students. The second focus group, with administration, consisted of Participant 5 and 6.
3.3 Description of Research Population and Participants
Participant 1 was a former lead programmer at a well-known higher education facility for
over ten years who worked with admissions and is currently a programmer at SAS. Three current
professors at Ferrum College are going to be Participants 2, 3, and 4 in the focus group used for
conducting research. All three of these professors have had the researcher as a student, so the

diligent use of Brightspace was confirmed. All 3 of these professors also recently attended
workshops hosted by a company named SAS here at Ferrum College. Consequently, it is clear
that these individuals have a general interest in data analytics, how it can be implemented in
higher education, and they are generally knowledgeable of the topic of data analytics.
Participants 5 and 6 were the participants for the second focus group. Participants 5 and 6 are
currently administrators at Ferrum College. Out of the 6 participants, 3 are male and 3 are
females and all range in age from 30 to 70 years old.
3.4 Research Instruments
Having an in depth conversation with Participant 2, 3, 4, 5, and 6 during our focus groups
and through a personal interview with Participant 1 was significant to the research and was the
best route for research. There is a lack of sufficient reasoning that a statistical approach would
prove beneficial for answering the research questions. A focus group allowed for a more detailed
response and gave the researcher the ability to have the participants go into greater detail about
their responses. The only limitation to a focus group was that it is only a small group of people.
This puts a limit on how much the results may be generalized. Considering that this project is
about the use of data analytics in higher education, input from these participants who have many
years of experience in higher education and also with working with data analytics provided
considerable insight.
3.5 Procedures
The researcher held the first personal interview with Participant 1, a former lead
programmer at a higher education facility, on March 1, 2016. The first focus group meeting was
on Monday, March 21, 2016 with Participants 2, 3, and 4. The second focus group was held on
March 22, 2016 with Participants 5 and 6. The participants were given as much time as they felt

necessary to answer questions during the meetings and were ensured that they would remain
completely anonymous throughout the study. Following the focus group discussion and
interviews, the researcher analyzed and reviewed the discussions. Key points that were made
during the meetings were highlighted and inserted into the next chapter of this research paper,
Data analysis and Discussion.
3.6 Data Analysis
Through repeated listening to the recording of the conversations, the researcher
transcribed the text from the focus group and interviews into a Microsoft Word document.
Transcribing the information and putting it into text made it easier to analyze the information
that was given. After transcription, the researcher read through all of the information that was
given and applied it to answer the research questions that started this project.

Chapter 4: Data Analysis and Discussion
4.1 Issues with Access to Data
A common theme discovered throughout research was issues revolving around access to
data. As much as universities want their data to be transparent and easily available, the
convention has been that the data shouldn’t be shared. Universities view the data as their data
and do not feel that people should have access to that data for many reasons. One reason that was
mentioned by every participant in this study is that the data that is collected is very politically
sensitive in nature. For example, if a university has a low retention rate or a low graduation rate,
it reflects poorly on the university and it forces those who have access to the data to try and keep
it under control. Participant 4 gave a great example of this when discussing parent panels here at
Ferrum College. Participant 4 stated that during his visit to parent panels, questions that are
frequently brought up are about the graduation rates and retention rates at Ferrum College which
are both pieces of information that are really never said aloud. According to Participant 4, “It’s is
a double-edged sword.” On one hand colleges and universities want to release the data, but on
the other hand, if the numbers are not appealing, it could easily lower students’ motivation to
attend the university. Participant 2 also gave a great example of how this data is politically
sensitive. A few years ago, some of Ferrum’s faculty and staff spoke with Roanoke Times about
internal problems. As a result of presenting this information publicly rather than working with it
internally, enrollment decreased at Ferrum College. If parents read articles in the newspaper or
online about problems at a college or university, it is going to make it harder to recruit new
students.

4.2 Lack of Understanding on How to Use Available Data
Data analytics is an extraordinary tool that can be used by higher education facilities, but
without proper use and implementation it become a waste of time and money. The most
important thing to understand is that colleges and universities must know what needs to be
measured and what data needs to be collected on students. Participant 1 stated, “A lot of people
think that if I can program it and throw technology at it, I have solved the problem. So if you
implement some sort of advanced analytics program and they feel that by simply analyzing the
data and creating algorithms and reporting on it is enough, they have lost half the battle.” With
that simplistic mindset on data analytics, nothing is ever going to get accomplished. There is a lot
of time and work required in order for data analytics to be successful. An example of an
ineffective use of data analytics at Ferrum College would be the Retention Alert System. The
Retention Alert System has been in place for roughly 3 years now and it has not been successful
according to Participant 6. The following is a description of the retention alert system from the
Ferrum College Student Handbook (2015): “Faculty and staff are encouraged to utilize the online
Portal to report concerns about a student, whether related to academics, finances, or student life.
Through the Retention Alert System, the student, as well as the academic advisor, coach or
appropriate personnel will be notified of these alerts in an effort to provide support in a proactive
manner.” The problem with the Retention Alert System according to Participant 5 is that the data
available is just too raw for it to be used successfully, but there are other possible factors that
could be contributing to the lack of success from the retention alert system. One example of a
possible factor leading to failure of the Retention Alert System is simply the lack of use of the
online Portal by faculty and staff. The researcher has been in several classes throughout his

academic career at Ferrum College and knows that there are multiple teachers at Ferrum College
who do not use the online Portal for the advantage of the Retention Alert System. If the faculty
and staff are not using the online Portal appropriately to express concerns about students, then
there is no way to accurately collect data as a whole about students. Ferrum College may be able
to use the Retention Alert System for gathering data on certain departments of faculty and staff
who diligently use the online Portal and the learning management system to be successful. As far
as collecting campus-wide data on students, it is nearly impossible to do without full cooperation
from faculty and staff.
4.3 Lack of Collaboration
Another vital key to the success of data analytics in higher education is the collaboration
between faculty and staff, administration, and admissions. Participant 4 stated that there is a
potential place for collaboration at Ferrum College but it is simply not happening. The problem
with collaboration does not lie only within the college or university, though. Participant 1 stated,
you probably need to look at peer institutions, institutions where some people have
applied to your institution and went instead, and understanding what type of practices
they are implementing. It is through that sort of collaboration you understand more about
the data.
Collaboration requires involving not only all of the different departments within a college of
university but also between colleges or universities in order to gain a further understanding of all
of the available data. At the end of the day, lack of collaboration all stems back to the issues
around sharing data.

4.4 Data Usage
Colleges and universities collect a massive amount of data on students. This data
collection occurs from the time that they apply and continues in some cases, even after
graduation. There are different ways to use the large amounts of data that are collected, though,
and that is descriptively, predictively, and prescriptively. Using data descriptively is taking the
current data available and using that data to analyze and describe the way that things are at the
university at that time. Using data predictively is using current and historic data in an attempt to
predict future trends with the intention of avoiding potential issues. Prescriptive use of data
follows up on predictive use of data because based on what has been predicted, solutions have to
be prescribed. Predictive data allows colleges and universities to foresee future trends and needs,
and the prescriptive use of that data allows these institutions to make appropriate plans to address
the future trends and needs. In order for data analytics to be successful, colleges and universities
must use all three types of these statistics.
4.5 Discussion of Findings
The research questions that were developed to guide this study are discussed in greater
detail in this section centered on the responses from the participants in the focus groups and
personal interviews that were conducted at Ferrum College. Also discussed in this section are
suggestions, final thoughts, conclusions, and closing remarks.
ResearchQuestion 1: What are the most important student data collected by academic
institutions?
ResearchQuestion 2: Which of the data types are directly associated with student outcomes and
what roles do they play in student outcomes?

ResearchQuestion 3: How can data analytics benefit admissions or administration of a college
or university?
ResearchQuestion 4: What are the major tasks involved in implementing data analytics into
higher education?
ResearchQuestion 5: What are the challenges of implementing data analytics into higher
education?
There is no definitive, most important piece of student data that is collected by
institutions. No specific student data will work across the board at every single college or
university. All of the data that a college or university collects about a student has some sort of
relevance and usefulness, but what data is most important is going to vary depending on the
university. A lot of experimentation is necessary to come to an understanding of what is most
valuable to a specific institution. The two most telling bits of data, though, are high school GPA
and SAT scores. These two pieces of student data can help a college or university predict if a
student is going to be successful.
The data types that are directly associated with student outcomes are similar to the above
question; however, they will vary from university to university. There are some basic predictors
though that an institution can look at in an attempt to predict if a student will be successful.
Those three basic predictors are high school GPA, SAT scores, and how much financial need
students have. According to Participant 1,
Most students who are capable of maintaining a high GPA in high school, they will be
able to handle college. If they score high on their SAT scores, they are probably good at
problem solving and understanding things and will probably be okay at college. If a
student doesn’t have to worry about paying for college too much they are more likely to

stay because they’re not going to have to stop coming because the office says you can’t
pay so you can’t come and have to get a job in order to come here.
Although these three pieces of data are three very basic things, they can be very telling. If a
student comes in with a low high school GPA, low SAT scores, or above average financial
needs, a college or university can directly target that student from freshman year and get him or
her involved in some sort of learning program or try to help him or her out financially by
offering other scholarships or grants. Taking a college skills class, for example, could help
improve the students’ ability to think critically, solve problems, and properly manage time to
ensure they are prepared to be successful. Being able to target those students from the day that
they enter an institution can be vital to their success and their overall outcomes.
Data analytics has the potential to benefit admissions and administration of a college or
university in many ways. The most important benefit for admissions is the ability to target
students that are a good fit for their respective schools. Admissions can use student data that is
available, specifically from students who stayed all 4, 5, or 6 years and graduated from a
particular school, and look for predictors or certain characteristics that these students shared.
This method may not be surefire, but it can help admissions target students who share these
predictors or characteristics because previously, similar students have managed to be successful
at that specific school. Administration can benefit from data analytics by analyzing the available
data and using it to determine if new programs should be implemented. An example of
administration using data analytics to implement a program at Ferrum College is the attempt to
expand the Brother for Brother Program. Participant 6 stated that the program was started
roughly 4 years ago and Ferrum College has tracked the retention of the students involved in this

program over the years, which has an upward trend. By analyzing the data and the success of this
program, Ferrum College has taken the initiative to expand this program by applying for a grant.
Although there are many benefits that can result from the implementation of data
analytics in higher education, there are also many tasks and challenges that colleges and
universities face in regard to implementation. The two biggest tasks that are faced are finding the
right data to benefit the specific school and also finding a way to involve everyone in the
organization from the president to faculty and staff to allow cross-organization collaboration.
Finding the right data involves a lot of investigating and trial and error. Participant 1 stated
You will have to do a lot of investigation to find the soft data that we may not be
collecting. It’s not a grade, whether they paid a bill, whether they have had some sort of
change in GPA, dropped a number of courses, requested transcripts, but maybe there is
other information we should be gathering about our students such as what time are they
taking their classes, are they taking classes and changing their major or are they taking
classes by professors outside of their majors?
Finding the right data involves looking at situations from different angles and really digging deep
to find out what works best for your particular college or university. Participant 1 also addressed
the task of involving everyone in the organization by saying universities have a tendency to cut
staff completely out of the picture, and they are the ones literally keeping the university going.
Cutting out faculty and staff is detrimental to the whole process of implementing data analytics
because they are the people who are interacting with these students every single day and really
know the students. Faculty and staff have better insight on the data because they will have a
general idea of what areas students struggle in the most. Given that faculty and staff are going to

have a greater insight on the data in which a college or university should be looking at, it is vital
to the success of data analytics that everyone in the institution is involved in the process.
The three biggest challenges of implementing data analytics into higher education are
time, money, and collaboration. A college or university needs time to collect data and determine
which of that data is going to be most useful to that school specifically. Furthermore, schools
must also take time to learn how to properly use evidence gained from the data to place within
the context of the university. Without collaboration, though, it is nearly impossible to determine
how the evidence can be used within the context of the university. Colleges and universities
cannot necessarily address the issue of time in a certain way, but they can address the challenges
of money and collaboration. To address the issue of money, institutions can apply for federal
grants to receive money to help pay for analytical software, institutions can fundraise from
alumni or from the board, or institutions can reevaluate the budget and allocate funds to invest in
institutional research and expanding data analytics. To address the issue of collaboration,
institutions must expand their institutional research and have collaboration with all of the other
departments on campus to determine what actions should be taken based on the results of data
analysis.
4.6 Usefulness of the Study
Upon completion of this study, the material included can be useful to individuals who are
looking for information on the implementation of data analytics in higher education in relation to
improving student outcomes. The study discussed colleges and universities that have
successfully implemented data analytics, how data analytics can be useful for improving student
outcomes, how data analytics can be used for improving admissions and administration of an
institution, and the tasks and challenges involved in the implementation of data analytics. Ferrum

College can use the results of this study to determine what actions the school should take in
regard to further implementing data analytics.
4.7 Final Remarks
The benefits that can result from implementing data analytics into higher education are
nearly endless. It is very important to realize that using data analytics is all about making things
better, improving what institutions have, and making certain productivity is not diminished.
When people are asked about data, people typically assume that something is wrong because of
that, but, data analytics can be implemented to improve the way things are done. The purpose is
not to highlight any failures an institution may have, but it can serve the purpose serve as an
effective diagnostic tool to promote growth and strengthening the institution. Students also need
to realize that it is not about highlighting their failures. Students may get offended if a teacher
approaches them about the class, but, students must realize teachers are just trying to help them
succeed. Institutions should be investing into the institutional research department and expanding
the use of data analytics. Taking these actions is not only for the benefit of the students, but for
the institution as a whole. In conclusion, this study was a chance to gain further knowledge of
data analytic and how it can be implemented to benefit higher education. Higher education and
student outcomes will always be a topic of great magnitude, and it is important to continue to
develop and improve methods that institutions use.

References
Arnold, K. E., & Pistilli, M. D. (2012). Course signals at Purdue. Proceedings of the 2nd
International Conference on Learning Analytics and Knowledge - LAK '12.
Broadbent, J. (2015). Retrieved from http://content.brightspace.com/wp-
content/uploads/Brightspace_CaseStudy_DeakinUniversity11.pdf?_ga=1.80610455.1068
102988.1454207578
Cooper, A. (2012). CETIS Analytics Series Volume 1, No 5 : What is Analytics? Definition and
Essential Characteristics. CETIS, 1(5), 1-10. Retrieved February 11, 2016, from
http://publications.cetis.ac.uk/2012/521
Dickey, D. A.(2012) Introduction to Predictive Modeling with Examples. Proceedings of 2012
SAS Global Forum, paper 337.
Ferrum College Ranking Indicators. (2010). Retrieved February 02, 2016, from
http://colleges.usnews.rankingsandreviews.com/best-colleges/ferrum-college-
3711/rankings
Ferrum College Student Handbook, 2015-2016. Retrieved May 24, 2016, from
http://www.ferrum.edu/about_ferrum/administration/offices_and_departments/student_af
fairs/student_handbook.pdf
LaValle, S., Lesser, E., Shockley, R., Hopkins, M. S., & Kruschwitz, N. (2010, December 21).
Big Data, Analytics and the Path From Insights to Value. MIT Sloan Management
Review. Retrieved February 11, 2016, from http://sloanreview.mit.edu/article/big-data-
analytics-and-the-path-from-insights-to-value/

Manyika,J., Chui, M., Bwrown, B., Bughin, J., Dobbs, R., Roxburgh, C.,&Byers, A.
H.(2011).Big data: The next frontier for innovation, competition and productivity.
McKinsey Global Institute. Retrieved 5 March, 2015, from
http://www.mckinsey.com/insights/business_technology/big_data_the_next_frontier_for_
innovation
Mathews, M. (2015). Retrieved from http://content.brightspace.com/wp-
content/uploads/Brightspace_CaseStudy_ORU1.pdf?_ga=1.183951398.1068102988.145
4207578
National Center for Higher Education Management Systems, NCHEMS. (2016) First-Year
Retention: Retention Rates – First-Time College Freshmen Returning Their Second Year.
Retrieved from http://www.higheredinfo.org/dbrowser/index.php?measure=92
National Center for Education Statistics, NCES. (2015, May). The Condition of Education –
Postsecondary Education - Completions – Institutional Retention and Graduation Rates
for Undergraduate Students. Retrieved April 26, 2016, from
http://nces.ed.gov/programs/coe/indicator_cva.asp
Picciano, A. (2014). Big Data and Learning Analytics in Blended Learning Environments:
Benefits and Concerns. IJIMAI International Journal of Interactive Multimedia and
Artificial Intelligence, 2(7), 35-43. Retrieved February 2, 2016.
Sattinger, M. & Ehrenberg, R. G., (2001). Tuition Rising: Why College Costs so Much.
Academe, 87(1), 79.

Stiles, R. J. (2012). Understanding and Managing the Risks of Analytics in Higher Education: A
Guide. Retrieved January 28, 2016.
U.S. Department of Education, Office of Educational Technology. (2012). Enhancing Teaching
and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief.
Washington, DC.

Key Terms
Big Data – extremely large data sets that may be analyzed computationally to reveal patterns,
trends, and associations, especially relating to human behavior and interactions
Business Intelligence – an umbrella term that refers to a variety of software applications used to
analyze an organization's raw data. BI as a discipline is made up of several related activities,
including data mining, online analytical processing, querying and reporting
Data Analytics – the science of examining data to draw conclusions and, when used in decision
making, to present paths or courses of action
Data Disaggregation – breaking down numeric or non-numeric information that has been
broken down into smaller units of data
Data-Driven Decision Making – using conclusion from analyzed data to help make decisions or
present courses of action
Data Governance – ensuring data and information privacy, security, quality, and auditability are
carefully controlled
Data Warehousing – collection of data in a “warehouse.” A data warehouse is a subject-
oriented, integrated, time-variant and non-volatile collection of data in support of management's
decision making process
Learning Analytics – the measurement, collection, analysis and reporting of data about learners
and their contexts, for purposes of understanding and optimizing learning and the environments
in which it occurs

Learning Management Systems – a software application for the administration, documentation,
tracking, reporting and delivery of electronic educational technology (also called e-learning)
courses or training programs

Appendix A
Focus Group Interview Questions
1. When were you first introduced to data analytics? And what are some examples of ways
you have used data analytics?
2. What is available to you on campus that you consider a source of analytical data?
3. What useful data can be gathered about students?
4. How do you think that Ferrum College can benefit from data analytics?
5. What benefit does data analytics hold to improving student outcomes, administrations, or
admissions of an institution?
6. Is there a place for collaborations between administration and faculty on data analytics?

Appendix B
Participant 1
What are the most important bits of data that can be collected about students and how was
that data gathered at your former place of employment?
There are some key measurements are freshman retention rate and the 4, 5, and 6 year
graduation rates. These are easy to calculate. Checking to see if they made it all the way through
to graduate. SAS is used to figure out to see if their major has changed, attributes have changed.
Use SAS in exploratory, or confirmatory. Data management capacity understanding following
students from space to space. In more advanced capacity, working in higher education is difficult
because you have a lot of people who apply and you can tell if those people are really serious
about taking you up on your offer of admission. Even if they accept your admission doesn’t
mean they are going to confirm and come in fall. Especially when working in an institution with
10,000 freshman. Used data mining applications to do predictive modeling to figure out based
upon the attributes we see of prior applicant and those who came to the school, can they come up
with a set of predictors to estimate whether or not someone is really likely to come to the
university. That is one application used and is really valuable in state institution because the
amount of dollars given from the government is all based up on number of students, course hours
they take, how many of these course hours are considered eligible for funding. A lot of this is
driven by the number of students coming in fall and also how to maximize the number of
students and still be able to offer the appropriate number of services to them. Also used to figure
out what students are most likely to encounter some sort of difficulty and needed support. This
was incredibly crucial in freshman year. If we have students who all of a sudden have multiple
advising sessions or they have problems with them being able to complete homework, or we

have notes from the professor saying that someone is having a difficult time. How do we try to
quantify that sort of feedback, or if we have someone who has a number in drop of credit hours.
Does that signal potential danger of not making to the next semester? Look at prior behavior to
determine what students are persisting from semester to semester, come up with predictive model
to determine based off of indicators whether or not to reach out to the students. Whether or not
the university tells you, it’s not about you making the cut and you having the right grades. They
have chosen you to be there and want you to stay and succeed and provide best support possible.
Based on predictive model identifying people who are most likely to leave the university due to
some sort of difficulties, we would try to intervene with those students to give them support.
Can you give me an example of something that would be done to intervene to help these
students?
A lot of times we would work with what we called “Student Recovery Services”
Are people hired specifically for this? Is it a separate department within the institution?
Yes. Our group was all about the data analysis. There was a provost in charge that would say I
need to figure out what is wrong with the students leaving because we need many students to
stay. So of the 5000 freshman who are here, can you break them out into groups based on the
likelihood of them leaving? And based on the services offered who should be offered additional
advising and who should be offered to move into a learning community so they are around
people who are studying the same things, which students need a more controlled academic
remediation such as tutoring services or college skills. This information was passed onto the
provost. Institutional research handles the data analysis and collaborate with other departments
on campus to figure out how to deploy an intervention.

When you are working on a project on your own you think you have control of everything you
are working on from collection of data to analysis but when you are working in an organization
such as university you may only have control of a very small portion of it, such as data analysis.
You have to use the evidence you find in this data analysis as a strong set of persuasive
recommendations to the other departments that handle student outreach and student support. You
might think your influence is large but you really only have thee data as your evidence and you
have to understand what exists in the data and be incredibly persuasive to influence people that
will pick it up and move along with it.
You said that freshman retention rate and 456 year graduation plan were the “Key
Performance Indicators,” what would you think the three most significant bits of data can
be gathered about students? What revealed the most information?
As many complicated models as we tried to make, it’s difficult to understand because a student is
there hopefully four years, so you start measuring things as they are a freshman and they may
change by junior year. So people would come up with these ideas starting this cohort were going
to put people into these learning communities and they are going to live with the people that they
study with. But by the time these people are juniors, the learning community idea has fallen by
the wayside and wasn’t implemented very well. You can’t measure people on the same scale
anymore, you have other institutional organizational things that have changed in terms of what a
major is called, how people are counting credits, have people fulfilled general education
requirements? All of these scales shift over time and we have found that the things you can really
rely on are SAT Scores, how much financial need do they have, and high school GPA. Those
sound basic and you say that to people they say of course. If they can swing it with high school
GPA or they can swing it because they were good in high school and could keep their GPA high,

they will probably be able to handle college. If they high a high SAT they are probably good at
problem solving and understanding things and will probably be okay at college. If they don’t
have to worry about paying for college too much they are more likely to stay because they’re not
going to have to stop because the office says you can’t pay so you can’t come and you have to
get a job in order to come here. People will say those are so obvious, but at the same time, you
say they are so obvious because the other things being measured are unreliable. We can maybe
make a model for a 2 year period that includes these other indicators like how many advising
sessions, are they doing entrance into a honors program, but really over the long term you have
to start looking at things that are very basic as your indicators because If you have these
measurements that are changing all the time there is no way to follow the thread of what students
are doing so you make recommendations. We have to get a handle on standardizing what we are
trying to measure so that 10 years from now we can find some common thread through all of
this.
Why does the government measure graduation on a 6 year standard when most universities
say they have a 4-year plan?
These measurements I talked about, freshman retention and 456 year graduation rate, there are a
lot of publications that request schools to publish that info such as Common Data Set. This is fed
into different reporting agencies such as College Board, Princeton review, US News and World
Report, Wintergreen Orchard House. These agencies are really important because they give PR
to your school if you end up on one of their lists for best colleges etc. There are a lot of reason
why 456 year graduation and retention rates are important. Things have become so dismal at
times with the 456 graduation rate that people can only look at 6 year graduation rate as the most
reliable method. I talked with people at institution rates where 4 year grad rate was under 50%, 5

year maybe 60% and 6 year maybe 75%. This has been a change especially in schools where
over the past 10 years where times have been tough financially. Students going to college has
become exorbitantly expensive and people income has not kept up with that cost. 4 and 5 year
rates are so low so the 6 year is what is really relied on. They are hoping this is a temporary trend
and that after coming out of the latest economic struggles in the United States that the 4 and 5
year graduation rates will increase. Also the number of students who are graduating from high
school prepared to go to college has gone down.
Could you give another example of the challenges of implementing data analytics into
higher education?
First and foremost, data standardization and availability.. Higher education is not run necessarily
like a business. In a business, records must be maintained because they have to send in their
taxes and make sure sales and payroll are accounted for and they have continued success. Higher
education is about learning, as much as we say it’s about making sure colleges and universities
are looking at their bottom line, it really turns into a different type of environment than running a
business. What comes along with that is a lot of relaxed understanding of how things should be
measured and stored, what type of longitudinal data do you need to keep? A lot of time things
are discarded because they do not think that it is important. But the biggest question is
understanding what do you need to measure? What data do you need to start collecting on your
students? A lot of things such as grades are kept but there are a lot of things we can’t really
understand how to measure easily so it’s about thinking about the university holistically and
what does it mean for a student to participate in the university? The last place I worked, they
tried to go after and capture some of this based on student engagement. How many activities are
they doing and what types of environments are they living in? That may be right for the right

campus that has an environment of student engagement and the students are coming to the
university because they want to be engaged but that doesn’t mean it will work across the board.
It means you will have to do a lot of investigation to find the soft data that we may not be
collecting. It’s not a grade, whether they paid bill, whether they have had some sort of change in
GPA or dropped a number of courses or requested transcripts, maybe there is other information
we should be gathering about our students such as what time are they taking their classes, are
they taking classes and changing their major or are they taking classes by professors outside of
their majors. There are a lot of things that you need to experiment with and understand are they
valuable to start measuring. And that is one challenge is knowing what to measure, how to
standardize it, and how to maintain it over the long term. Universities are also very afraid to
share their information with other universities, especially when it has become a challenge with
diminishing enrollments. The last university I worked out they were so against sharing any
information regarding applications or financial aid because they felt those two pieces of the data
pie were just going to be used by another university to get the students. So it is an interestingly
competitive environment. When you are talking about handing data analysis in higher education,
if you think about your university as a single entity, you are probably missing a lot. You
probably need to look at peer institutions, institutions where some people have applied to your
institution and went instead, and understanding what type of practices are they implementing
because it is through that sort of collaboration you understand more about the data. The whole
field of institutional research is probably 30 or 40 years old and it’s been kicked around by a lot
of universities. The university that I worked in, it was part of the assessment and accreditation to
make sure we were meeting standards by the accrediting bodies for the university. Became its
own thing and wasn’t really supported well. A lot of time these offices have really good people

but they have no support from the university so they end up struggling because people don’t take
what they say serious or do not think that it is important. I have worked with colleagues where
presidents of universities has said everything is data driven and we have to figure out how to
bring data analytics into making this college a success.
When would data analytics not be beneficial?
I think a lot of people think if I can program it and throw technology at it I have solved the
problem. So if you implement some sort of advanced analytics program and they feel that simply
analyzing the data and creating algorithms and reporting on it is enough, they have lost half the
battle. It is about using the evidence to make sound decisions that can better the practices of the
university. I have had a job where the president thought hiring a bunch of analysts would solve
the problems of the university. And he hired some really good people and they did very good
work but there was so much missing in terms of how they placed that within the context of the
university and it really takes collaboration between the presidents and the dean and faculty and
staff. One of the biggest problems at this university was the staff was completely cut out. So you
had faculty, deans, and president’s included in these discussions but the staff was cut out, and
they are the ones literally keeping the university going. So if people believe hiring a bunch of
analysts to run the data is enough than they are completely missing the entire picture.
With the right people, you would say that implementing data analytics is beneficial no
matter where?
You need people who can make decisions, communicate what they found, people who can
present it, either in visual or reports. You need people with domain expertise to understand what
they are doing and how this fits into not just the university but a whole group of universities who

are all trying to accomplish the same feat. It’s all about making things better and improving what
you got and making certain you do not have diminished productivity. Keeping an eye on things
and trying to come up with better answers is always going to be beneficial.

Appendix C
Focus Group 1: Participants 2, 3, and 4
When were you first introduced to data analytics?
Participant 4: I’m defining data analytics as dealing with large amounts of data and trying to
find out what it can tell us about a problem we may be looking at. In that sense, possibly, I began
perhaps when I was doing my statistics in graduate school. We uses SAS and I didn’t know
much about it then because it was my first semester, but since then, for my doctorate I used large
amounts of data and had to analyze it to find out what was happening. It was an economics PHD
so I had to deal with large amounts of data and then do some matrix analysis, etc. I would
consider that to be my first introduction to data analytics.
Participant 3: Over 40 years ago I started dealing with large data sets. Originally I was studying
biology so I had access to very large data sets regarding osprey breeding success. What size,
number of young, etc. Then I moved to lizard biology and had a data set with lots of individual
observations. When moved into business I worked with data sets, some about stock prices and
their associated accounting info and then I moved over to looking at saving and loan data. All
saving and loan in the US using data from what was then the regulatory agency for savings and
loans. I’ve been working with large data sets for decades. Originally worked in 4tran on punch
cards… then moved to a program called RATS (regression and time series). I still think this is
the strongest program for analyzing large data sets. It’s basically a very specialized programming
language designed for large sets of data. I’ve used SAS and SPSS as well. SAS more than SPSS.
Most work is done with RATS.

Participant 2: Nurse for 20 years working in hospital with large amounts of data with patients
and the information about the drugs. Grad school had large data sets to work with and had SPSS.
There wasn’t any point and click, you had to program everything in. And then, I’ve been doing
research for example I did a thing a few years ago on the athletes and if their GPA was higher or
lower than average student. Had over 4,000 entries. Done a couple things for school system on
salaries and regression and predicting. A lot of forecasting.
What is available to you, on campus, as a source of analytical data?
Participant 2: There is not much. It is hard to get data. I’ve asked several times for data from
institutional research. Used to be able to get it but you cannot now.
Participant 3: There is corporate data and we have access to databases on EDGAR.
Participant 4: There isn’t perhaps adequate systematic data about students collected. For
example, following a student from the day they enter until they graduate and the courses they
took and changes they made such as major. There is no database to pull that up and open it up
quickly.
Participant 3: There are lots of different separated data sets.
What about directly related to students? Anything you can look at to help improve
outcomes? Anything to look at to compare to other students?
Participant 3: Only have things within own classes and advisees. Can see advisees mid-term
grades and current grades but after graduation there is no access to historical data. I can’t look
across all of the students in one class and see how they are doing in all of their other classes.

Participant 2: The only data that I have had access to is data that I have collected myself over
the years.
What does Brightspace give you guys?
Participant 3: Brightspace doesn’t really give any additional information.
Participant 4: It only gives us information about the classes we are currently teaching.
What “corporate” systems outside of campus could be beneficial to look at for improving
student outcomes?
Participant 4: I assume national database will be available if we wanted to look at that but
looking specifically at Ferrum College there is nothing.
Participant 2: There’s a really nice database that came out, I can’t remember who published it,
and it was about criminology. I asked the library to purchase it but they did not. Some databases
are expensive and require yearly subscriptions. Some databases that are out there, but aren’t very
cleaned up so it takes time to really get usefulness out of them.
Participant 4: It depends on what you are looking for. You can get state databases on education
but you wouldn’t be able to relate much back to the students at Ferrum. You could make rough
comparison.
Participant 3: Institutional data that is available is aggregated so there is no way to break it
down into classes to look specifically at seniors.
Participant 4: There are lots of thing you could look at if you were given the data that could be
useful but there is no access.

If you were given access to data that is here about the students, what would you think
would be the most useful about students?
Participant 3: Personally I think you want to look more, can you predict whether a student will
dropout or can you predict if a student will graduate in 4, 5, 6 years. This would affect decisions
about financial aid and admissions. An institution wants its students to be successful but you
don’t want to admit the people that you can predict are highly likely to fail.
Participant 2: Several years ago, I got the SAT scores and compared them to students SAT
scores that graduated, so it was like a paired T. And how they scored, I did percentiles, and how
they scored on our ETS exams. That showed that the student’s had really increased a lot in their
level of at least being able to take a test. It was useful information because it said we were on
track but there is no access to that data anymore.
Participant 4: This analysis becomes important because Federal gov’t is tying Pell grants and
aid with success rates in college. Even thinks like student pre-payment of loans has become a
factor as well. Trying to predict a lot of things based on data that is available.
Participant 2: One thing that I would ask data for is looking at different grades, SAT’s, and how
students are doing since we have changed from 15 weeks to 13.5 weeks. Looking at the effect of
policy changes. Also would like to know more about e-terms.
Participant 4: Data analytics, to be feasible, the data should be easily available and easily
subjected to analysis. It is tough for small institution to organize it like that. Business use data
analytics to interconnect any and every department to provide any analysis and you can pull up
any relationships that you want to look at. It is really expensive software that is hard to get.

Participant 2: I would also like to know why students transfer. Because we have a large number
of students, there are two different databases where one is 45% and one is 50%, I’d like to know
exactly why they are transferring. You hear two or three basic reasons for transfer. One reason is,
especially at lower levels, they feel like other students aren’t serious students and it is effecting
their learning.
What do you think would improve if Ferrum College would benefit from data analytics?
Participant 4: It all depends on the findings but all the things such as retention, success
(graduation), are we choosing the right students, and what leads to success. These are things that
are of great concern to the school. Also would like to be able to find out what leads to success.
Participant 2: I think it would be nice if we could have an alumni database and the only way to
keep up is through LinkedIn or Facebook. These connections help internships, jobs, funding, etc.
Participant 4: I just heard from a student today they were getting an internship at McDonalds in
Texas. I was thinking right then that I should talk with her so I could perhaps get this as an entry
way so our students could get in. Hollins University does this all the time. They have a longtime
alumni database with lots of connections and they put this to good use. We should be doing the
same thing with our current students and forming connections and a database. Maybe just a
website so they could look up places. I think it would be good to make those connections so they
could just go to certain places and fill out paperwork.
If data analytics were to be implemented at Ferrum College, what would you say the
biggest challenges Ferrum College would face? Is there anywhere to collaborate?
Participant 2: Time and cost. None of us really have an opportunity and with something like
that you need a block of time.

Participant 4: I think we should have a really knowledgeable person in some office that
analyzes this because it is important to the school and there will be additional cost to the school
for the person. Acquiring and keeping data is expensive. I think we have an ERP system, but our
system is not connected in such a way that the data is obtained can be analyzed by any part of the
system.
Participant 3: One important thing to realize about the data is the administration views this as
their data. And they don’t feel like other people should appropriately have access to that data.
Part of it is that it is politically sensitive in nature. If a college has a low retention rate of
freshman or high rate of students failing to repay loans then that reflects adversely on the college
so to some extent the institution wants to keep that data under control. In that since it’s not so
different from businesses. For examples in business, McDonalds doesn’t let franchisees have
access to McDonald’s corporate data. I think the biggest challenge that Ferrum faces is that we
do not invest a lot into our institutional research. Right now we have one person responsible for
the data and part of that is so we can keep the data on a leash. The president decides who can see
what data. We do not invest a lot into data analytics. All of us are data people, so if we were in
that role we would double, triple, or quadruple budget for data analytics. So I think partly, a big
constraint, is knowledge of what it means to work with data and knowledge of investment
needed to make data workable. The institution has elected to not make that decision.
Participant 2: Who owns the data and who has access is a really big topic now. And, sometimes
when you have a large set of data, maybe someone might find something out that we don’t want
the public to know necessarily.

Are there any initiatives here for data analytics?
Participant 2: We just want any and all information that we can get to improve decision
making.
Participant 4: We have other systems, such as the Retention Alert System, but that is ongoing
while it is happening, current data, not trying to predict behavior.
Is there is a place for collaboration?
Participant 4: There is not much but there is a place it could happen but it is not happening.
Participant 3: Administration doesn’t release. It is politically sensitive and the fewer people
they know about it the better.
Participant 4: It some ways I think that it is prudent. I go to parent panel and question
constantly goes up like what is your retention? If information like this was widely available, even
releasing the number of students who receive financial aid, so the tuition were talking about is
paid for by a handful of people, but we don’t really say that out loud. Parent panel brings up the
question a lot about retention rates and graduation rates a lot and if it is low, it is difficult to
bring good students in. It is a double edged sword.
Participant 3: This data is available in the department of education but here nobody will
volunteer and give it to you.
Participant 2: I think part of it too is that they are afraid that we might have an agenda and that
we may use that data in a way that is not the most tasteful.
Participant 4: It is political in many ways. One time here, we went and spoke with the
newspaper about internal problems and was not the appropriate thing to do. As a result of this,

enrollment decreased. If you are a parent and see a headline in the Roanoke times that says
problems at Ferrum College, it makes it harder to recruit students. There is also the possibility of
a disgruntled employee. You have to look at it like a business because it is. Small schools like
ours have a difficult situation because of funding. Not because they aren’t doing a good job but
they are. Talk of free community college, dropout rates, and the fact that you are dealing with
students who otherwise wouldn’t get a chance. That is not talked about. Ferrum College does a
great job with all the effort from the students but there are first generation students who don’t
have that kind of support. Nobody talks about that fact either.
Participant 2: Here is an example of data I tried to get. I was asked by AAUP to get data on
how much money was allocated to athletics, academics, administration, staff, there were 5 or 6
different categories. Told at first that I could get that data and then was denied. I would like to
know how the budget is broken down. I thought that was pretty simple data.
Participant 3: I think in general business do not want to give up financial data unless they have
to. They only want to report only what is required. An example is Apple would not give
percentages of how much they invest in iPhones and laptops. This is information business do not
like to release. There are standards for what is required to be reported and that is typically the
only information that business like to release is what is required. If you release data with good
information, business will say we are doing better and if you release data with bad information,
people will not hesitate to call it out. Business have nothing to gain from releasing data that is
not required to be released.

Are there any closing remarks, any final things you would like to say about data analytics
in higher education?
Participant 3: There is data of the institution but in higher education there is also research data.
Faculty uses research data more than they do institutional data.
Participant 2: I would like to be able to get some information on the student’s and just see what
kind of impact different variables have had on the students. Just getting that data wouldn’t seem
risky to the institution.

FINALIZED SENIOR SEMINAR PAPER (1)

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (15)

Semelhante a FINALIZED SENIOR SEMINAR PAPER (1)

Semelhante a FINALIZED SENIOR SEMINAR PAPER (1) (17)

FINALIZED SENIOR SEMINAR PAPER (1)