Assessment outcomes from the TENCompetence project
1. Assessment in TENCompetence: Sharing Assessment Process Information Yongwu Miao Educational Technology Expertise Centre Open University of The Netherlands CETIS Assessment SIG meeting Glasgow, Sep. 26, 2007
2.
3.
4.
5.
6. learning design (or assessment design) unit of learning (or unit of assessment) role activity environment property component method condition expression action play act role-part learning support service learning object staff learner internal change-property notification show/hide assessee assessor assessment- specific service item activity-description resource QTI_type external assessment An extended IMS LD model assessment_item outcome item_body assessment_test interaction response response_processing A simplified IMS QTI model associate choice . . . . . .
7.
8.
9. A peer assessment example A Standard-based and Service-oriented Approach to Support Assessment Process design assignment writing article1 reviewing article2 improving article1 writing article2 reviewing article1 improving article2 final assessment2 final assessment1 learner1 learner2 tutor article2 feedback1 improved a2 paper1 article1 feedback2 improved a1 paper2 tutor’s feedback1 tutor’s feedback2 assessment form1 assessment form2
10.
11.
12. Conceptual Model role activity assessment design artefact rule certifier designer assessee assessor stage rating responding instructing editing portfolio constructing item performing input tool output assignment portfolio assessment test assessment item evidence assessment result expression action portfolio editor concept mapping QTI editor feedback simulator LSA QTI player giving feedback react on feedback preparation evaluation evi. collection processing teaching-learning add/remove role member trigger activity alloca. activity to role send email change property …… …… evi. provider consultant decision maker …… …… distribute artifacts present information ……
13. Assessment process model Programmed instruction model Peer assessment model <…> 360 degree feedback model Domain-specific models Assessment process specification Unit of assessment mapping Assessment Process Specification IMS LD + IMS QTI + Assessment-specific services
14.
15.
16.
17. File Edit View Tool Template Help Peer assessment Here shows the guidance to make design decisions State bar user’s decisions Specify role structure, artifacts, and services Show and edit peer assessment model artifact role service lane stage
18. File Edit View Tool Template Help Peer assessment How many stages in the peer assessment process? Please make a decision user’s decisions artifact role service lane stage Default (three stages) With design assessment stage With additional feedback stage More complicated setting
19. File Edit View Tool Template Help Peer assessment How many roles will be involved? Five stages have been defined artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 Default (two persons) With the involvement of tutor More complicated setting
20. File Edit View Tool Template Help Peer assessment Would you like to define role structure? Default role setting and with the involvement of a tutor artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R
21. File Edit View Tool Template Help Peer assessment Would you like to define a typical activity for each stage? Please make a decision artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R Default setting More complicated setting
22. File Edit View Tool Template Help Peer assessment Would you like to define a typical activity for each stage? Nine activities are defined and roles are assigned artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R learner1 learner1 learner1 learner 2 learner 2 learner 2 learner1 learner2 tutor
23. File Edit View Tool Template Help Peer assessment Which stages will the tutor be involved? Please make a decision artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R learner1 learner1 learner1 learner 2 learner 2 learner 2 learner1 learner2 Give feedback 1 Give feedback 2 More complicated setting tutor
24. File Edit View Tool Template Help Peer assessment Which stages will the tutor be involved? The tutor is assigned to work in the stage of “give feedback 2” artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor
25. File Edit View Tool Template Help Peer assessment Would you like to create artifacts ? Please make a decision artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R Default setting More complicated setting learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor
26. File Edit View Tool Template Help Peer assessment Would you like to create typical artifacts ? artifacts are created artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1.1 A feedback1.2 A outcome1 A outcome2 A assignment1 A assignment2 A improved outcome1 A improved outcome2 A feedback2.1 A feedback2.2 A
27. File Edit View Tool Template Help Peer assessment Would you like to change the titles of artifacts? artifacts are created finished artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1.1 A feedback1.2 A outcome1 A outcome2 A assignment1 A assignment2 A improved outcome1 A improved outcome2 A feedback2.1 A feedback2.2 A
28. File Edit View Tool Template Help Peer assessment Would you like to change the titles of artifacts? The title of an artifact has been changed finished artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A
29. File Edit View Tool Template Help Peer assessment Would you like to define artifacts? An artifact has been created finished artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A learner 2 learner 2 Review Form How do you think about this article? accept without modification accept with minor/major modification reject Comments: assessment form QTI assessment editor assessment form (QTI item) A
30. File Edit View Tool Template Help Peer assessment Would you like to define artifacts? An artifact has been defined finished artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A assessment form A
31. File Edit View Tool Template Help Peer assessment How will the artifacts be distributed? Please make a decision artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A assessment form A Reciprocally Mutually Randomly Default setting
32. File Edit View Tool Template Help Peer assessment How will the artifacts be distributed? An artifacts flow has been defined artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A assessment form A
33. File Edit View Tool Template Help Peer assessment How will the artifacts be distributed? An artifact flow has been defined artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A assessment form A