Working with SETU

1. 1
Working with SETU
Stuart Palmer
Institute of Teaching and Learning

2. 2
A context-free SETU comment
“I found that Stuart seemed to be
complacent about this unit, his
lectures where not worth turning up
too, as it involved sitting there
listening to him talk from the unit
guide, which I did by myself at home
anyway.”

3. 3
Student evaluation of teaching (SET)
Work by Ramsden and others in the 1980s
(re-)established a link between student
perceptions of their learning environment and:
> the level of their engagement with their
studies; (hence, generally,)
> the ‘quality’ of their learning/outcomes

4. 4
Student evaluation of teaching
Reflecting the contemporary understanding of
the multi-faceted nature of quality and the
finding that it is the entirety of the university
experience that contributes to the student
‘course experience’, virtually all authors
examining the value of the SET process
recognise that SET data are only one of many
sources of information that should be called
upon when evaluating teaching

5. 5
Evaluation of teaching
Other valuable data sources include:
> objective measures of student learning (such
as unit marks)
> reflective (self) assessment of teaching
performance (what worked, what didn’t)
> peer assessment of teaching
> student focus groups

6. 6
Evaluation of teaching
> etc. …maybe Facebook?

7. 7
Interpreting SETU
USA Today has come out with a new
survey - apparently, three out of every
four people make up 75% of the
population
David Letterman

8. 8
Interpreting SETU
> a reasonable number of responses;
> a reasonable overall response rate; and
> a reasonably equal rate of responses
between campuses
…provide some confidence that the survey
sample is representative

9. 9
Consistency
> a small standard deviation on a mean item rating
estimate indicates that most respondents were in
agreement
> a large positive difference between the proportion
estimates of agreement and disagreement indicates
that substantially more students agreed
> if the written open-ended comments reveal common,
repeated themes, these are worthy of further
investigation
> etc.

10. 10
Discrepancy
> a small number of ratings that are different from the
majority can influence the reported mean estimate
> a small number of adverse comments that seem at
odds with the majority of written feedback are
probably not worth losing a lot of sleep over
> a rating in one item that is significantly different from
the others in the SETU results for the same unit
might be an indicator worth investigating further
> etc.

11. 11
Interpreting SETU – in general
SETU is only one measure of the ‘quality’ of
teaching
SETU is a pretty blunt instrument
Used in an rational manner, the quantitative
and qualitative results from SETU can be an aid
in the evaluation of teaching and learning

12. 12
Interpreting SETU – in general
The larger the number of responses, the more
reliable the data
Trends (as long as they are real) can be
informative

13. 13
Mean SETU responses 2003-2004
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Meanrating
2003
(n=51, 17%)
2004
(n=73, 42%)
2005
(n=47, 32%)
2006
(n=32, 37%)
1 The teaching staff of this unit stimulated my interest in further learning.
2 The teaching staff of this unit motivated me to do my best work.
5 This unit was well taught.
6 I had a clear idea of what was expected me in this unit.
17 I would recommend this unit to other students.
18 The use of on-line technologies in this unit enhanced my learning experience.

14. 14
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.
2 The teaching staff of this unit motivated me to do my best work.
5 This unit was well taught.
6 I had a clear idea of what was expected me in this unit.
17 I would recommend this unit to other students.
18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Meanrating
2003
(n=51, 17%)
2004
(n=73, 42%)
2005
(n=47, 32%)
2006
(n=32, 37%)

15. 15
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.
2 The teaching staff of this unit motivated me to do my best work.
5 This unit was well taught.
6 I had a clear idea of what was expected me in this unit.
17 I would recommend this unit to other students.
18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Meanrating
2003
(n=51, 17%)
2004
(n=73, 42%)
2005
(n=47, 32%)
2006
(n=32, 37%)

16. 16
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.
2 The teaching staff of this unit motivated me to do my best work.
5 This unit was well taught.
6 I had a clear idea of what was expected me in this unit.
17 I would recommend this unit to other students.
18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Meanrating
2003
(n=51, 17%)
2004
(n=73, 42%)
2005
(n=47, 32%)
2006
(n=32, 37%)

17. 17
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.
2 The teaching staff of this unit motivated me to do my best work.
5 This unit was well taught.
6 I had a clear idea of what was expected me in this unit.
17 I would recommend this unit to other students.
18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Meanrating
2003
(n=51, 17%)
2004
(n=73, 42%)
2005
(n=47, 32%)
2006
(n=32, 37%)

18. 18
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.
2 The teaching staff of this unit motivated me to do my best work.
5 This unit was well taught.
6 I had a clear idea of what was expected me in this unit.
17 I would recommend this unit to other students.
18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Meanrating
2003
(n=51, 17%)
2004
(n=73, 42%)
2005
(n=47, 32%)
2006
(n=32, 37%)

19. 19
Mean SETU responses 2003-2006
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Meanrating
2003
(n=51, 17%)
2004
(n=73, 42%)
2005
(n=47, 32%)
2006
(n=32, 37%)
1 The teaching staff of this unit stimulated my interest in further learning.
2 The teaching staff of this unit motivated me to do my best work.
5 This unit was well taught.
6 I had a clear idea of what was expected me in this unit.
17 I would recommend this unit to other students.
18 The use of on-line technologies in this unit enhanced my learning experience.

20. 20
SETU teaching comments
“I found that Stuart seemed to be
complacent about this unit, his
lectures where not worth turning up
too, as it involved sitting there
listening to him talk from the unit
guide, which I did by myself at home
anyway.”

21. 21
SETU teaching comments
> An exceptionally hardworking teacher. Always
responding promptly to enquiries. Thank you Stuart
for your efforts.
> Excellent teaching of course work
> Excellent teaching, great feedback, showed interest
in progress. Best I have come across.
> Good lecturer.
> Great course content and well delivered

22. 22
SETU teaching comments
> It was good how Stuart used videos to illustrate
things in the real world. His lectures are well
structured as all the unit material is covered in
lecturers which are always informative. It was nice to
get the reminder email about assignment1 and 3, this
shows he has compassion towards the students.
> The lectures were good and didnt drag on too much.
> The teaching support offered as part of this unit was
excellent.

23. 23
SETU teaching comments
> This unit was very well taught. I could not fault any
part of it.
> Very easy to listen to, and teaches well
> Very promptuos lecturer
> Well structured class.
> very interesting to listen to and knows how to teach.
was very helpful to me when i had questions
> very well taught, easily understood, madde the unit
enjoyable.

24. 24
SETU teaching comments
> Yes stuart palmer is dedicated lecturer who is willing
to help and puts an effort into helping students. His
lectures cover alot of content with plenty of examples.
> Good uses of informations and extra material to help
get a point across
> sometimes it feels very long to has class for 2 hours.

25. 25
2
2.5
3
3.5
4
4.5
5
W
elltaught
M
aterials
W
orkload
Requirem
ents
Feedback
Library
Recom
m
end
O
nline
O
K
O
/lenhanced
Faculty
School
Prereq
Similar
Unit

26. 26
2
2.5
3
3.5
4
4.5
5
W
elltaught
M
aterials
W
orkload
Requirem
ents
Feedback
Library
Recom
m
end
O
nline
O
K
O
/lenhanced
Faculty
School
Prereq
Similar
Unit

27. 27
2
2.5
3
3.5
4
4.5
5
W
elltaught
M
aterials
W
orkload
Requirem
ents
Feedback
Library
Recom
m
end
O
nline
O
K
O
/lenhanced
Faculty
School
Prereq
Similar
Unit

28. 28
2
2.5
3
3.5
4
4.5
5
W
elltaught
M
aterials
W
orkload
Requirem
ents
Feedback
Library
Recom
m
end
O
nline
O
K
O
/lenhanced
Faculty
School
Prereq
Similar
Unit

29. 29
2
2.5
3
3.5
4
4.5
5
W
elltaught
M
aterials
W
orkload
R
equirem
ents
Feedback
Library
R
ecom
m
end
O
nline
O
K
O
/lenhanced
Faculty
School
Prereq
Similar
Unit

30. 30
2
2.5
3
3.5
4
4.5
5
W
elltaught
M
aterials
W
orkload
R
equirem
ents
Feedback
Library
R
ecom
m
end
O
nline
O
K
O
/lenhanced
Faculty
School
Prereq
Similar
Unit

31. 31
2
2.5
3
3.5
4
4.5
5
W
elltaught
M
aterials
W
orkload
R
equirem
ents
Feedback
Library
R
ecom
m
end
O
nline
O
K
O
/lenhanced
Faculty
School
Prereq
Similar
Unit

32. 32
1.5
2
2.5
3
3.5
4
4.5
2005
2006
2007

33. 33
1.5
2
2.5
3
3.5
4
4.5
2005
2006
2007

34. 34
1.5
2
2.5
3
3.5
4
4.5
2005
2006
2007

35. 35
Using SETU
The circumstances, context and influences on a
particular unit are individual in nature
Standard, boilerplate responses probably aren’t
that useful
SETU might offer some insights of value

36. 36
Online PD materials
www.deakin.edu.au/itl/pd/tl-modules/scholarly