The Development and Usability Evaluation of a Standards-Based Grading Tool for Faculty

1. The Development and Usability Evaluation of a Standards-Based Grading Tool for Faculty Alaa Sadik, Sultan Qaboos University (OMAN)

2. Introduction Grades are the common method for reporting student performance across universities. Today, universities tend to use standards-based assessment approach to assess student performance.

3. Introduction Sultan Qaboos University(SQU): a cross-cultural organization, brings together hundreds of instructors from around the world. the largest academic community, consists of 9 colleges, awide variance with regard to grading practices.

4. Problem of the Study Many instructors believe that: Preparing, scoring, grading, and reporting student academic performance are each extremely exhausting and time-consuming tasks. It is difficult to perform standards-based assessment according to the university quality assurance policy and requirements.

5. The Need Therefore, the need was emerged to: Use/develop a standards-based assessment tool that allows for more accurate and relevant grading and reporting.

6. Research Questions How effective (useful) is the assessment tool in documenting, grading and reporting student performance? How efficient (easy to use) is the assessment tool as perceived by users? What is the overall satisfaction of users toward the use of the assessment tool? What individual difference variables influence instructors’ perceived usability of the assessment tool?

7. Purpose of the Study The main purpose of this study was to design, develop and evaluate a standards-based assessment tool for instructors at SQU.

8. Method Development of the assessment tool A review of the literature A series of interviews and focus groups were conducted, to determine the gap between the existing grading practices of faculty and those that are needed for the assessment tool.

9. Method The assessment tool should Support the University grading scale and generate students’ grades automatically; Enable qualitative assessment of student performance; Attach content and performance standards to grade sheets; Have a built-in e-mail function for communicating grades with students; Facilitate total point and percent weight of scores and assignments; and Generate course statistics for distributions, correlations and variances.

10. Method

11. Method

12. Method

13. Method

14. Method Sample 116 faculty members from 4 colleges at SQU.

15. Method Procedures Participants were asked to perform a series of specific tasks in RealGrade, each of which had several subtasks.

16. Method Instruments Grading tool usability questionnaire To solicit participants’ perceptions toward the effectiveness, efficiency and acceptance of the tool. Instructor interviews To gain a thorough understanding of the use of RealGrade and to provide rich detail and insights into instructors’ experiences.

17. Results: Questionnaire Effectiveness Participants strongly agreed or agreed (m=3.93) that RealGrade : Is effective in facilitating the process of standards-based assessment, Has the tools needed to assess student performance and communicate grades with students, and Increases their productivity.

18. Results: Questionnaire Efficiency Participants indicated that (m=3.34): Learning to use RealGrade takes a short time, The user interface menus and dialogue boxes are favorable, Weighting, assessing, and managing student assignments based on performance standards are very easy tasks.

19. Results: Questionnaire Satisfaction The majority of participants (m=4.19) reported that they liked and felt comfortable with RealGrade as a tool to assess student performance.

20. Results: Questionnaire

21. Results: Questionnaire Participants with greater degrees of computer experience had higher perceptions of effectiveness, efficiency, and satisfaction with RealGrade. Participants who had less than five years of teaching experience found RealGrade more effective, efficient and satisfactory than those who had more than five or ten years of experience.

22. Results: Interview

23. Discussion and Conclusions Differences among instructors are vital to the eventual acceptance and implementation of the standards-based assessment tool. Further research is needed to develop report card to help instructors and students understand the standards-based assessment information included and to make it more comprehensible.

24. Thank You More information www.alaasadik.net/realgrade Contact alaasadik@gmail.com