This document provides an overview of the institutional effectiveness process used at a university. It discusses identifying outcomes and standards to measure, developing valid measurement tools like rubrics and surveys, and implementing consistent data collection. Key parts of the process include analyzing results, disseminating findings, utilizing the data to suggest necessary changes, and documenting the process. The university uses oversight groups and an annual review to evaluate effectiveness and inform strategic planning and quality improvement initiatives.

1. An Overview of the Institutional EffectivenessProcess Kelly McMichael, Ed.D.

2. Institutional Effectiveness “To establish and sustain a data-driven and documented system of evaluation and assessment that will ensure the University and its branches perform in a manner that is always improving its educational and operational quality.”

3. Institutional Effectiveness “The effort to make certain the institution makes progress on its goals, accreditation and strategic plan.”

4. Institutional Effectiveness

5. Institutional Effectiveness “Anything worth doing is worth measuring.” “You get what you measure.” “You expect what you inspect.”

6. IE Assessment Process Identification (Outcomes and standards are collectively developed and criteria quantified) “What’s important to measure?” 7. Documentation (Minutes and records are kept through meetings and archived using the IE Report template for future reference) “How and where do we keep data and changes?” 2. Instrumentation (Rubrics, tests, records, surveys, tracking programs developed with reliability & validity) “Do we have valid tools to use in measuring?” 6. Utilization (Necessary changes are suggested and being made) “What do we do with the results? What needs to change?” 5. Dissemination (Results are sent to key repositories) “Who needs the data and results?” 3. Implementation (Consistently and comprehensively performed) “How and when will we measure?” 4. Analyzation (Data is computed, summarized, and easily understood) “What does the data mean?”

7. Instruments in Use We need to ensure reliability (consistency of measurements) and validity (capturing sound intended data through questions that support outcomes and goals). CAPS course database Portfolio Review rubrics IDEA course surveys Student Satisfaction Inventory Annual Employee Survey Employee training evaluations Accuplacer Test WritePlacer Test ETS Proficiency Profile Alumni surveys Capstone course rubrics Internship surveys Committee minutes Help Desk Log College Success (FS101) surveys

8. Portfolio Reviews One particularly valuable and direct measure of student learning occurs when academic departments assess student learning outcomes at the 3rd, 6th, 9th and 12th quarters of student performance in the program. At these milestones key student learning outcomes are evaluated using specially developed programmatic rubrics. The rubrics are applied by a panel of faculty evaluators and the University uses their findings to assist students in making improvements in presentation and content mastery. Repeated sub-performance in multiple reviews by multiple students may indicate needed course corrections in program curriculum and instructional methodology.

9. Examples of Data-Administration

10. Examples of Data-Academic

11. The Nichols’ Template

12. The Nichols’ Template

13. Oversight Groups The University has established (appointment by the University’s President) three domain-specific oversight groups to conduct reviews of the Institutional Effectiveness Reports. These reviews occur annually prior to kick-off of the Strategic Planning and Budgeting Processes. The oversight groups are charged with the following key tasks: (1) evaluate whether the institution successfully achieved its desired outcomes from the previous institutional effectiveness and planning cycle, (2) identify key areas requiring improvement that were identified in the institutional effectiveness reports’ through analysis, and (3) develop strategies and recommendations for quality improvement initiatives for the next strategic planning and budgeting cycle.

15. The Educational Support Services Oversight Group reviews measures of assessment data and other key information concerning the effectiveness of student services.