1. Program Review Process: An Interdisciplinary Approach to Ensure Quality and Accountability Higher Learning Commission Conference | Chicago, IL | April 2009
2. Overview • Introductions • APUS Mission and Demographics • Goals of the Program Review Process • Program Review Process, Tools, and Resources • Use of Continuous Improvement in Business Program • Use of Fact Books to Drive Decisions • Conclusion and Dialogue
3.
4.
5. Our Mission To educate the nation's military and public service communities by providing respected, relevant, affordable, and student-focused online programs which prepare them for service and leadership in a diverse, global society.
6. Who We Are • Founded in 1991 as American Military University to provide affordable and convenient distance learning to service members • In 2002, expanded reach to national security, public safety and other professionals seeking affordable education and online convenience, establishing American Public University System with member institutions: American Military University and American Public University • We are a 100% online for-profit school that serves 49,000+ students who study online from 100 countries.
7.
8. % of Students by Degree Program % of Students by School Student Body Profile Graduate 24% Undergraduate 76%
10. • Course books • Electronic resources • Learning strategies Library and Learning Resources • Student learning outcomes • Instructional strategies • Evaluation procedures • Academic rigor Curriculum Assessment • Expert reviewer report • Industry Advisory Council report External Reviewer Feedback Paper Free Process Electronic Repository of Information
11. • Analysis of faculty credentials and expertise to ensure breadth and diversity Faculty • Student demographic information • Enrollment History • Growth trends Students • Curricular Mapping • Assessment measures • Fact book Learning Outcomes Assessment Paper Free Process Electronic Repository of Information
12. • Program benchmarking with similar programs and institutions Program Benchmarking • Evaluation of findings • Program recommendations • Three year proposed strategic plan Program Directory Summary • Dean’s observations • Meeting minutes Program Review Findings Paper Free Process Electronic Repository of Information
Harbert 2 min Our curriculum aligns with our mission – educating those who serve. Specialty programs to match professional requirements as well as traditional educational options for students in that community who may wish to careers in other areas of service.
Harbert - 2 min A diverse student body of 31,000 civilian and military students The school of education is a new school – excepting that, you can see that the students are fairly evenly distributed
The ongoing program review process at APUS consists of 6 distinct phases: 1 - Data Collection: Each PD is responsible for this phase with assistance from their faculty and administrative staff. PD reviews different aspects of the program and prepares data and information for others to review (will outline this information in the next few slides). 2 - External review: The program is reviewed by an evaluator from outside of the APUS system and community. 3 - Analysis: The School Dean and PD review all relevant data. After they complete an analysis of the program, they prepare a report of the findings. After all of this information is compiled, it becomes available to individuals within the APUS community to read and review. 4 - Program Review Meeting: After folks have had a chance to review the material from the previous 3 phases, there is a formal 2 hour review meeting led by our Academic Dean to engage stakeholders from across the University. Stakeholders include – PD, school dean, enrollment management, student services, transfer credit folks, registrar’s office, faculty development, library resources, learning outcomes assessment, marketing, and the Provost. These meetings provide an opportunity for stakeholders to openly and honestly discuss the data that is presented by the PD program director. At this time, all parties are involved in the discussion. 5 - Three Year Plan: Each PD is required to develop a 3 year plan of program development as part of the review. Following the review meeting, this ensures that the PD is held accountable. Given that programs are on a 3 year cycle of review this document becomes a source of analysis for the subsequent reviews also. 6 - Follow up: To monitor progress, the 3 year plan is followed up by the school Dean to ensure that the plan is on target. The follow up takes place during the annual strategic planning cycle so each program can prepare for the coming year.
The analysis of data in the program review is a key part of the process, enabling folks to evaluate all aspects of the program during each phase. The primary tool used to house the data is an electronic storage repository that has been developed specifically for this purpose. This is a paper free repository that allows members from across the university to gain access to program information on a shared network drive. There are 9 different dimensions of the program that are examined in the review: Library and Learning Resources: How frequently is the library used by students in the program? What electronic resources are used? Are the text books relevant and up-to-date? Curriculum Assessment: What are the program level objectives? Are they current and relevant? Are the course level objectives aligned with the program and institutional level objectives? What instructional strategies and evaluation strategies are used? Are they varied and do they accommodate different learning styles? To accomplish this review all course syllabi are reviewed as part of this process. External Review Feedback: There is a review of the program by an evaluator from outside of the APUS system. This is typically a recognized academic leader in the program area who teaches or is an administrator at another University. The reviewer writes up a report that includes an assessment of the program and recommendations for change. Some programs have industry advisory councils, a group if individuals who meet a few times a year to provide guidance to the development of the program.
Faculty: Who are the faculty in the program area? What are their credentials? What is the student/faculty ratio? Students: Who are the students and from where do they come? Enrollment history & program growth are examined. What draws them to this program and why? Are students seeking specific program content that is not covered? What is unique about this program from a student’s perspective and how does this uniqueness serve our mission? Learning Outcomes Assessment : PD create curricular maps to ensure that course objectives aligned with program and institutional objectives. Also, the OOP creates fact books for each of the programs. Fact books contain information such as student and faculty demographic data, course completion & withdrawal rates, grade distributions for courses, results from direct and indirect measures of assessment. They are typically 15-20 page booklets that display all data available for a particular program.
Benchmarking: How does the program compare with similar programs at other accredited academic institutions? The courses offered, # of credit hours, cost of the program, student population, and other related data is analyzed in a detailed spreadsheet. PD Summary: The PD prepares a report that identifies the strengths and challenges of the program. This report includes an evaluation of findings, program recommendations, and a 3 year proposed strategic plan for continuous improvement. Review Findings : This information is compiled after the program review meeting and includes Dean’s observations and meeting minutes. Chad will now talk about how the program review process has been used for continuous improvement in the Business Administration program.
Introduction – Courses included in the Bachelor of Business Administration program are spread across three other departments including management, marketing, and math. Although these core and major courses are owned by different department chairs the courses must be included in BBA Learning Outcomes Assessment because they reflect the students overall program experience. The program data will stay intact.