Presented workshop at Distance Teaching and Learning conference on 8.12.14 with Dylan Barth and Nicole Weber Visit DTL2014.wikispaces.com for more info

1. Evaluating the Effectiveness of Faculty
Development Programs for Blended
and Online Programs
August 12th, 2014
Distance Teaching and Learning Conference
Tanya Joosten, @tjoosten, Dylan Barth, @dylanbarth, Nicole Weber, @nwebs
University of Wisconsin – Milwaukee
tjoosten@uwm.edu | djbarth@uwm.edu | nicolea5@uwm.edu

2. Overview
Introduction to UWM
(3) Breakout activities, questions to consider in
planning to determine faculty development
effectiveness
Sharing of UWM’s pedagogical model, faculty
development, evaluation strategies, and evaluation
tools as well as other institutions
Conclusions

3. University of Wisconsin –Milwaukee
Tanya Joosten, tjoosten@uwm.edu, @tjoosten
Dylan Barth, djbarth@uwm.edu , @dylanbarth
Nicole Weber, nicolea5@uwm.edu, @nwebs
Introductions

4. Tech
Enhanced
Blended
Online
Traditional
Self-paced
MOOCs
Flex
Faculty development
programs and pedagogical
consultation
Technology training and
support
Evaluation and research
About us

5. About
UWM Online

6. What is
blended and online?

7. A Backwards
Design Approach
to Evaluation

8. Evaluation planning
Input
• Medium (F2F,
Tech)
• Instructor
demographics
• Discipline
• Course level
• Instructor
past
experiences
• Social
information
Process
• Defining good
blended and
online course
• Defining
pedagogical
model
• Designing
faculty
development
program
Output
• Course level
• Program level
• Institutional
level

9. F2F is not the gold
standard
Focus on the
process…the
pedagogy
Inputs

10. Define a good
blended and
online course
Define
pedagogical
model
Design a faculty
development
program
Process

11. What are we
doing today?

12. Discuss: Steps to evaluating effectiveness
1.) Break into discussion groups using Adobe Connect.
You will be able to use audio or text chat.
2.) Access the Google Docs to document your activities at:
DTL2014.wikispaces.com
3.) Discuss, share, and brainstorm responses to the
following series of questions on the 3 steps to planning
evaluation.
Don’t forget to identify the member of your group and
document your group’s response in the Google Doc.

13. Step 1: Defining
effective

14. 1. What is a “good” blended or online
course? What pedagogical model
facilitates a “good” blended or online
course?

15. Good course
Higher retention
Better grades on assessment
• Quizzes and exams
• Projects and papers
• Overall
Satisfied students
• Future enrollments
Learning
Engaging
Enrollments

16. Pedagogical model
Content
• Text
• Images
• Audio
• Video
Interactivity
• Discussions
• Groups
• Feedback
Assessment
• Written and oral examination
• Discursive
• Portfolio

17. Content

18. Interactivity

19. Assessment
Cognitive
Affective
Psychomotor
or Behavioral
Individual Group
Projects
Discussions
Writing
Quizzes

20. Step 2:
Facilitating
success

21. 2. What elements and format should be
considered in designing and developing a faculty
development program?
What opportunities and experiences should be
available to help instructors learn effective
practices in design, delivery, and teaching
blended and online courses?

22. UWM’s
faculty
development

23. • Ten questions
• Designing learning modules
• Online vs. F2F - Integration
• Decision rubric for
content choices
• Learning objects
Content
• Progressive/summative
• Before, during, and after
• Self evaluation
• Peer evaluation
• Student evaluation
Course Evaluation
• Rubrics
• CATs
• Templates
• Traditional formats
Assessment
• Synchronous/asynchronous
• Establishing voice
• Discussion forums
• Small groups
Interactivity
• Managing expectations
• Time management
• Technology support
Helping Your Students
• Staying organized
• Managing workload
• Avoiding course and a half
Course Management
Course Redesign
Transitioning to
online and
blended teaching
Pedagogy and more

24. Experience

25. F2F 1
3 hrs
Online 1
F2F 2
3 hrs
Online 2
F2F 3
3 hrs
Showcase
3 hrs
Post-
Program
Blended

26. Backwards design

27. Active learning

28. Practice-based

29. Participant
Self-
Reflection
Progressive
Participant
Evaluation
Facilitator
Self-
Reflection
Summative
Participant
Evaluation
Program evaluation

30. Step 3:
Documenting
success

31. 3. How will you know when faculty are
providing quality blended and online
courses?
What tools or services could be provided
for evaluating the effectiveness of blended
and online courses?
How will quality be communicated to
others?

32. Did it work?
CC Flickr katherine.a

33. Course evaluation

34. Course evaluation

35. Student Evaluation Data
Course evaluation

36. Course evaluation

37. Who is your audience?
What variables will be
examined?
How will the data be collected?
analyzed?
How (and where) will the data
be presented?
Who will be involved in the
evaluation process?
When will the evaluation be
completed?
Evaluation plan

38. Audience

39. Input Process Output
Students Interactivity Students/Faculty
Demographics Communication Learning
Age Engagement Satisfaction
Gender Social Presence Performance
Ethnicity Content
Student Status Assessment Course/Institutional
Data
Full time/Part time Retention
Employment Status Drop/Withdrawal Rate
Zip Code Grade
Course/Instructor
Evaluation
Course
Discipline
Course Level
Instructor
Mode of Delivery
Variables

40. Collecting

41. Analyzing

42. Dissemination
CC Flickr bengray

43. Human Resources
CC Flickr Vandy CFT

44. Timeline
CC Flickr zamboni.andrea

45. Challenges
CC Flickr EverExplore

46. Questions

47. Thank you!
Tanya Joosten, tjoosten@uwm.edu, @tjoosten
Dylan Barth, djbarth@uwm.edu, @dylanbarth
Nicole Weber, nicolea5@uwm.edu, @nwebs