1. faculty of arts ict in education
| 1
IDEM DITO:
Towards one system for digital testing
at the University of Groningen
Symposium: A Broader Experience, Bremen, 22 March 2012
Sake Jager, University of Groningen

2. faculty of arts ict in education
| 2
About the project
› IDEM DITO:
 DIgitaal TOetsen = Digital Testing
 Different faculties: one system
› Period: April 2011-May 2012
› University funding: ICT strategy fund
› Participants:
 Faculty of Medical Sciences
 Faculty of Arts
 Faculty of Economics and Business
 Educational Support and Innovation
› Aims to examine:
 Which / what kind of testing system works for our organisation?
 Who are going to use it?
 How do we manage it?
 What is the cost? (Which return?)

3. faculty of arts ict in education
| 3
Institutional context
› University testing policy › Internationalization
 More and better testing  More RUG students
 Improve student turn outs abroad
More visiting students
› Professionalisation Exam 
Erasmus Mundus
boards 
collaborations
 Better preparation for supervision
and standards setting tasks › Central test facility
288 computers
› Professional development 
Multifunctional use
faculty

 Flexible testing
 Didactic qualifications configuration
 Personal portfolio  Specific applications for
testing?

4. faculty of arts ict in education
| 4
Situation at start project
› Blackboard 9.1 learning environment:
 Management and support: Educational Support and Innovation Centre
 Decentralized support Faculties
 Link to student registration (Progress) and scheduling (Ocasys)
› Tests in Blackboard (with Respondus)
 Diagnostic/formative: as part of learning
 Summative: exams and (language) tests
› Didactic benefits:
 Multimedia: Arts, Culture and Media programme
 More frequent tests in relation to mandatory study advice
› Logistic / organisational issues:
 Lack of test facilities: multiple takes of same test
 Limitations Blackboard: re-sits, item banks, version management, test
statistics.

5. faculty of arts ict in education
|5
| 12-1-2013
Potential benefits of digital tests
› Reduction of correction time, paper savings
(tests, answer sheets), easier distribution and
workflow
› Compiling item banks and re-use items
› Better legibility compared to handwritten work
› Immediate access to distribution of scores,
difficulty items, and other statistical data
› Multiple question types; pictures, graphics,
photos, sound, video, animations; concurrent use
of additional program (e.g. SPSS)
› Immediate results (scores) for students, feedback

6. faculty of arts ict in education
| 6
Example case
› Law teacher
› 200 international students
› Student laptops and word for exams
› “Handing in” of assignments on USB sticks
› Legibility no longer an issue
› “This is much more efficient than pen-and-
paper”
› But: Even here, further efficiency gains are
possible!!

7. faculty of arts ict in education
| 7
Plan of action
› Specification of evaluation criteria 
› Desk research (15) 
› Presentation testing applications (8) 
› Pilots with teachers, students, test specialists,
technicians, four packages (QMP, LPlus,
Compass / Teletoets, Quayn) 
› Advice on organization and management 

8. faculty of arts applied linguistics
|8
| 12-1-2013

9. faculty of arts applied linguistics
|
Pilots completed
Pilot Faculty Date Objective
Essay-exam FEB 2 December 2011 Open questions / essay
Teacher pilot 1 FMS 20 December 2011 Teacher evaluation of useability
History Arts 20 December 2011 Multiple question types (mc, ordering, open)
Feedback
Arts, Culture, Arts 10, 31 January; 6 Blackboard
Media February 2012
Pilot Master FMS 17 January 2012 Student comparison of (two) packages
students Medicine
Teacher pilot 2 FMS 25 January 2012 Teacher evaluation of useability

10. faculty of arts applied linguistics
|
Pilots in progress
Pilot Faculty Objective
Language Centre English Arts Multiple question types (mc, cloze, essay)
Correction open questions
Audio
Test results and analysis ESI Comparison with legacy application for analysis
Export of test data
Language test Russian Arts Multipe question types (mc, ordering, open)
Non-Western fonts (teacher, student)
Connectivity Blackboard ESI Examine possibilities (together with suppliers)
and SAS packages
Arts, Culture, Media Arts Test taking and correction open questions
essay test

11. faculty of arts ict in education
| 11
Some preliminary results
› QMP:
 Pro: No failures, import, question bank, support
 Con: stand-alone authoring, text editor, expertise required
› Quayn:
 Pro: Wysiwyg interface, easy to use, web-based
 Con: Flash required, under development
› Compass / Teletoets:
 Pro: Interface, text editor, version management, web-based
 Con: Test matrix, language support
› LPLus:
 Pro: Versatile, many options
 Con: Architecture (databases, not fully web-based)

12. faculty of arts ict in education
| 12
Emerging scenario of use
› Test requirements covered by combination of Blackboard
and dedicated testing program
› Registration as now (Progress)
› Separate Blackboard server, specific BB course for each
test, temporarily available
› Test in Blackboard or dedicated program (link in BB course,
no separate registration)
› Consolidation of grades in Progress
› Tests prepared in Word, Blackboard or test program
› Gradual increase in use of dedicated testing application

13. faculty of arts ict in education
| 13
Additional actions
› Teacher training
 Test design
 Test analysis
 Digital test construction
› Exam scheduling
 Identify courses for digital testing
 Prioritise these
› Management plan
 Authentication and identification
 Usage policy test facility
 PC configuration
 Maintenance and support

14. faculty of arts applied linguistics
ict in education
| 14
Thank you for your attention
Contact:
S. Jager
http://www.rug.nl/let/idemdito
http://www.rug.nl/staff/s.jager
s.jager@rug.nl