More Related Content
Similar to Tips for Creating and Delivering Effective Mobile Assessments (20)
More from Kenji Lamb (20)
Tips for Creating and Delivering Effective Mobile Assessments
- 1. Tips for Creating and Delivering
Effective Mobile Assessments
Ivan Forward
ivan.forward@questionmark.co.uk
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 2. What will we cover today?
Tips for mobile and embedded assessment
Example applications of mobile assessment
Embedded assessment
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 3. Growth in Internet Connected Mobile
Devices: 41% from 2009 to 2010
Source: ComScore
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 3
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 4. Tips for Creating and Delivering
Effective Mobile Assessments
Mobile delivery enables new possibilities for
observational assessment and mobile exam rooms
Surveys and course evaluations: Sometimes less is
more
Embedded assessments can add value and provide
useful metrics on social/informal learning
Think small: Screens, input devices and bandwidth
To Flash, or not to Flash? Be aware of what
technologies your participants’ devices support
Test and test again: Try before you deliver!
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 4
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 5. Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 5 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 6. Inexpensive, very portable Internet-
connected mobile devices
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 7. What are the potential applications?
Deliver to the
Observational Mobile exam
device in your
assessments rooms
hand
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 8. Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 8 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 9. Observational assessment vs.
‘traditional’ quiz, test or exam
Traditional assessment
• Participant is presented with
questions for direct response
Observational assessment
• Observer is presented with
questions to ‘rate’ participant
according to pre-determined
criteria
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 9
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 10. What are Observational Assessments?
Observational assessments provide a mechanism
for an ‘observer’ to assess a participant
For example, a participant might be observed and
rated on…
Performance of a specific task or procedure
Knowledge acquisition and application
Demonstration of skills/abilities that ‘traditional’
assessment may be less reliable in measuring
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 10
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 11. Typical workflow for “assessor”
Assessor logs in
Selects assessment
Selects participant to be observed
Answers/submits responses to questions
Results/feedback reported to stakeholders
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 11
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 12. Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 12 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 13. Mobile Exam Rooms
Use mobile devices
Greater flexibility in where you deliver
assessments
Assessments can be conducted ‘on location’
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 13
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 14. “Securing” an iPad…
Consider if a standard browser
presents security concerns
For example, Questionmark provides
an app that prevents candidates
from easily getting to other URLs
Consider if access to “home
button” (which allows task-
switching) presents security
concerns
Hardware solutions are available to
enable use of iPad in a “kiosk” mode
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 14
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 15. Consider the “stakes” and if
monitoring/proctoring is appropriate
g
Levels of Monitoring
High
Medium Stakes
Stakes Exams
Low Exams
Stakes Tests
Diagnostic
Tests
Formative Quizzes
Course Evaluations Stakes of Assessment
Risk of cheating; content security
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 16. Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 16 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 17. Quizzes on mobile devices
Low stakes and openly available
Often appropriate for mobile devices
Enables retrieval practice to reduce
forgetting
Provides the participant with corrective
feedback for revision and learning
Encourages self-directed and independent
learners
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 18. When is feedback useful?
Dealing with difficult questions
Helps participant understand and learn from the
experience
Correcting misconceptions
Provide guidance when there is evidence of a
fundamental misunderstanding of a topic
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 18
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 19. Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 19 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 20. Surveys and course evaluations
Boost response rate by encouraging participants
to access via mobile delivery – “on the spot”
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 21. True for most surveys;
Especially true for mobile surveys
Consider your objectives:
Keep your surveys brief to help increase
response rate
Put your most important questions at the
start of the survey
Include only one concept per question
Group related questions together
Avoid relying on open-ended
questions (e.g. ‘essay’)
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 21
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 22. Authoring the Survey –
questions focused on:
- Demographics
- Instructor
- Course Materials
- Facilities
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 22
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 23. Course Evaluation Analytics
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 23
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 24. Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 25. Screen Furniture
Ensure your form buttons and other navigation
controls (a/k/a “screen furniture”) work with low
screen resolutions
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 25
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 26. Input
Provide short URLs when hyperlinking not possible
e.g. writing the URL on the board
Consider using a QR Code linked to an assessment
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 26
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 27. Images
Optimise size and compression of larger image files to
minimise bandwidth requirements
Detail on images should be kept to a minimum to
ensure readability on small screens
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 28. Be aware of what technologies your participants’ devices
support
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 28 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 29. Be cautious in use of Flash
Helps minimise bandwidth requirements
Strengthens cross-device compatibility
Flash will not work on Apple iPhones, iPads or iPod
touch devices
Flash is compatible with some mobile platforms such as
Android phones (and tablets), Windows Mobile,
Blackberry (later this year)
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 29
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 30. Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 30 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 31. Test your content!
Test your assessment content…
With different browsers
On as many different devices as practical
Emulators can help understand how content will appear
on the ‘small screen’ -- but don’t always give a true
“user experience”
Setting up a mobile exam room?
Test delivery environments when possible – particularly
for signal/wifi strength
Many free online bandwidth tests are available
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 32. The same technology that can accommodate small-screen
mobile devices can also help make it easier to embed
iFrame delivered assessments in your learning content
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 32 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 33. Technologies used to support
social/informal learning initiatives
Portals
Social Networking
Wikis Forums
Blogs
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 33
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 34. Embed in a Wiki
Embedded in a
Wiki
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 34
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 35. Embedding in Blogs, Portals, Wiki’s
For Learners For Instructors
Focus the learners’ attention Easily use assessments
Practice memory recall throughout the learning
Identify what was learned process
Provide feedback to assist the Provide seamless user
learner to: experience
Correct misconceptions Centralised management
Instill confidence of assessment content
Point to relevant material and results
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 35
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 36. Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 36 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 37. Blended Delivery
g
Levels of Monitoring
High
Medium Stakes
Stakes Exams
Low Exams
Stakes Tests
Diagnostic
Tests
Formative Quizzes
Course Evaluations
Stakes of Assessment
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 38. Access to your Assessments
Mobile delivery makes it easier for
participants to access
assessments:
Use mobile Apps
Place links in content
(PDFs, HTML Web Pages, etc)
Embed within a Portal
For employees, partners and
customers
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 39. Autosensing Device and Browser
Questionmark Packages the
Assessment for Device/Browser/iframe
Sends
Senses
Device/Browser
HTML
Auto-Size
senses pixel size of window
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 39
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 40. Benefits
Author once Deliver to any
device or context
Schedule once
Single results set
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 40
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 41. Apps for Delivery
Can sometimes offer more control,
convenience than built-in browser
Apps available for
Android
Apple iOS
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 42. Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 42 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 43. Tips for Creating and Delivering
Effective Mobile Assessments
Mobile delivery enables new possibilities for
observational assessment and mobile exam rooms
Surveys and course evaluations: Sometimes less is
more
Embedded assessments can add value and provide
useful metrics on social/informal learning
Think small: Screens, input devices and bandwidth
To Flash, or not to Flash? Be aware of what
technologies your participants’ devices support
Test and test again: Try before you deliver!
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Slide 43
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
- 44. Tips for Creating and Delivering
Effective Mobile Assessments
Ivan Forward
ivan.forward@questionmark.co.uk
Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.