This powerpoint is taken from a workshop for university teachers on the basics of evaluation, including its advantages and disadvantages, and how to best use evaluation as a tool for improving the student experience in higher education.
2. What makes higher education
You have a younger relative who is choosing a
university course (they are choosing between
universities, they already know the subject).
What aspects of the institution and degree course
would you tell them to investigate in order to make
this decision, and why?
NSS results? Dropout rate?
3. What is evaluation?
From Ellington, Percival and Race (1988) Handbook of Education Technology:
"Evaluation is the collection of, analysis and interpretation of information about
any aspect of a programme of education or training as part of a recognised
process of judging its effectiveness, its efficiency and any other outcomes it may
• Used to help improve teaching in an ongoing processFormative
• Used to make personnel decisions: is X a good teacher?Summative
4. Common questions
Why am I evaluating
What am I evaluating?
What are the sources of
What methods can I use?
What are the advantages
and disadvantages of
When do I want to do
5. Common evaluation methods
What are the advantages and disadvantages of each method?
6. Who carries out evaluation?
• In dept, school,
• In PGCAP or similar
• Reflective exercise
on PGCAP or CILT
• Probation mentor
• External observation
• External examiner
7. Why evaluate?
Self-improvement of teaching and students’
Wish to know what worked and what didn’t –
development of teaching and action research
Help own productivity and satisfaction
Internal department quality
Professional Bodies (e.g. NHS standards)
8. What would you do?
Pick a scenario from the handout
In your scenario: do you need
to evaluate your teaching?
If so, which evaluation
method/s would be most
Come up with an Evaluation
when, where, who and how
will you evaluate?
9. Issues with evaluation
Student evaluations mirror performance (& teacher effectiveness?)
Clayson (2009), ‘Student Evaluations of Teaching: Are they related to what students
learn?’ Journal of Marketing Education 31:1, 16-30. This meta-analysis discovers a
small relationship between positive evaluation and effective learning but that it
differs between disciplines and teaching contexts.
Gender bias in student evaluations
Driscoll and Hunt (2014), ‘What’s in a Name: Exposing Gender Bias in Student
Ratings of Teaching’ Innovative Higher Education 40, 291-303. This study examines
student ratings of teachers of online courses – students rated the same teacher
lower when they thought it was a woman than when they thought it was a man.
Interesting tool on gender bias in SET (using ratemyprofessor.com:
Statistical problems with Student Evaluation of Teaching (SET)
Stark and Freishtat (2014) find that SET overall throws up lots of statistical
problems and caution against measuring performance by SET alone.