Ims and qti assessment
•Transferir como PPT, PDF•
2 gostaram•4,105 visualizações
Recomendados
Recomendados
Mais conteúdo relacionado
Destaque
Semelhante a Ims and qti assessment
Semelhante a Ims and qti assessment (20)
Ims and qti assessment
- 1. IMS Specifications and Standards: QTI and Assessment Dr Roger Rist ICBL Heriot-Watt University
- 2. Outline of Talk • IMS and Standards • What is interoperability? • The QTI specification for Interoperability • What next?
- 3. Who Needs Standards? • Providers • Vendors • Authors • Mentors • Learners
- 4. IMS Global Learning Consortium • Apple, Sun, IBM • Cisco, Oracle, SCT, Saba • Blackboard, WebCT, PeopleSoft • Pearson Education, Thomson Learning • US DoD, Industry Canada • JISC, U. California, U. Michigan • and others • Plus 2000 in developer network
- 5. What Does IMS Do? • Develops and promotes the use of specifications for information resources, enterprise system components, and scenarios of activity that enable distributed learning
- 6. Why Do We Need Interoperability? • Interoperability solves interchange problems between proprietary question formats and between different users. • The IMS specifications allow the import and export of data that may differ from its original formats.
- 7. How Do IMS Specifications Foster Interoperability ? • Interoperability! The goal is to provide a standard format to allow interoperability between many different systems used to create distributed learning. Code & Digitized IMS specifications and Compatible items, Delivery systems, information data modeling templates learning objects, authoring systems, content libraries reporting systems, certification, meta data, assessments
- 8. Specification, Implementation, and Standardization Accepted standards Official Sanction ion vis Re IEEE Mature products Trials Initial specs at ion alu Ev Emerging nd ta Consensus Tes Early implementations
- 9. Start with: Consortium Members Standards Domain Specific Bodies Content Consortia Providers Government Researchers Agencies Distributed Learning Test-beds Organizations Commercial Developers
- 10. What benefits come from Specifications? • They make digital content more independent from services and hardware used to deliver it • They provide more uniform, more precise access to networked learning resources and services • They help extend the life time of capital purchases and organizational changes • They facilitate the integration and maintenance of system components and data resources
- 11. IMS Development Process Functional Needs, Technical Means, Practical Constraints Community Review Products, of Interest Board Services, Base Practices Review and Draft feedback Working cycle Group Public Draft Tests and Experiments Final Developers, Release Users
- 12. IMS Interface Specifications Harmonization, Consolidation, Consistency Meta-data Digital Libraries Enterprise Systems Packaging Learner Enterprise Environ- Mgmt ments Groups Profiles QTI Learner/Group Management Information Services Assessment Services
- 13. IMS Specifications • Metadata • Enterprise • Content Packaging • Question and Test Interoperability • ePortfolio • Content Management • Digital Repositories • Learner Information Package • Learning Design • Accessibility for All
- 14. IMS Specification Adoption • Microsoft, Blackboard, MindLever, Eduprise, SmartForce, NETg, … • IMS Centres (Europe, Australia, Singapore, Canada, Catalonia-Spain) • DoD SCORM • IEEE LOM • JISC projects - eLearning programme • UK Further Education MLE Program • DoEd LAAP: Indiana University project • DoEd NCAM/IMS Accessibility project
- 15. Specification Documents • Information Model – Lists the data elements that comprise the specification • XML Binding Document – Shows how to convert the data in the information model into an XML record • Best Practice Guide – A good place to start as a support document to explain more about the information model – Examples of XML content packaging – Glossary
- 16. The Information Model • This document is really the core of the specification. It dictates: – which data elements are expected to appear in a manifest XML file – where those elements may appear in the data structure – a few other constraints on the data elements themselves such as whether they contain a boolean (true or false) value, a string value of a particular length, and so on.
- 17. The XML Binding • A binding is a well-defined way of writing down information in a formal language such as XML. • A binding is a technical realization of the information model that defines a content package. • There may be several bindings of the same information model.
- 18. The Best Practice Guide • This document is an informative guide to implementing the specification. • It provides help with some questions about how the specification should be used in practice. • It contains guidelines, examples, and implementation issues that users may find helpful when attempting their own implementations of the specification.
- 19. What is XML anyway? • Extensible Markup Language is used to structurally describe data • Specification produced and maintained by the World Wide Web Consortium
- 20. XML Meets Internet Demands • XML has no predefined tags. XML allows data to pass easily between different applications. • XML is a series of your own tags to delineate: – Elements – Element contents – Element attributes • XML allows all devices, platforms, and devices to share data.
- 21. XML Advantages Browser - human readable XSL Style Sheet XSL Printed material XML Style Data Source Sheet XSL Style Populate web page content Sheet Data transformation EAI process XML eXtensible Mark Up Language XSL eXtensible Style Language Proprietary application EAI Enterprise Application Integration Business Application
- 22. What Is Not Covered by IMS Specs? • User interfaces • Pedagogical paradigms • Policies to constrain innovation, access, interoperability, or re-use
- 23. Question and Test Interoperability • What is it? – Defines a standard format that allows interoperability for questions and tests between different computer systems – Formats for constructing and exchanging assessment information – Export from one system and import into another
- 24. QTI: Why do we need it? • Allows interchange of questions between computer systems • Allows exchange of questions and tests between different users • Allows publishers to provide questions and tests in a format everyone can use • Allows questions to be built up for the long term
- 25. Where is the QTI Info on the IMS Web-site? Click here for the specs The URL is http://www.imsproject.org
- 26. IMS QTI page Web-site Click here to download the specs
- 27. QTI Format • Expressed in XML – Tags define the logical structure of the data – Format for holding Question and Test data • Does NOT define how Q &Ts are delivered NOR how results are analysed • Does NOT define how called from a Learning Management System NOR how Q & Ts interact with content
- 28. QTI Terms • Items • Assessments • Participants • Sections
- 29. Question Types supported 1 • Multiple Choice • Multiple Response • True/ False • Image hot spot • Fill in the blank • Short answer text • Numeric entry
- 30. Question Types supported continued • Slider • Drag and Drop • Order Objects • Essay Text • Match Item • Connect the points • and others ... • Extensions possible
- 31. Scoring and Feedback • Optional not mandatory • Response processing – can assign scores, feedback or other actions depending on the response made • Results handling will be added in future versions
- 32. Future Versions • Will describe tests (i.e. assessments and sections) in more detail • Will describe a standard format for holding results • May include proprietary extensions • Compatible with existing specification
- 33. Status of IMS QTI • Version 2.1 available in the public domain • This v2.1 Public Draft Version 2 specification is a draft and liable to change before the completion of the Final specification.
- 34. IMS QTI Spec Details • Specification of an assessment in a tag language (XML) – <test_title> Chemistry Assessment 2 </test_title> • Further details of IMS – http://www.imsproject.org
- 35. Question Mark Tool • First QTI tool - QTI XML Viewer • Converts between QML and IMS QTI XML. • Available from Question Mark’s website • http://www.questionmark.com
- 36. CETIS • UK Centre for Educational Technology Interoperability Standards • Funded by JISC • Allows access to IMS groups for ALL UK HEIs • Web site: http://jisc.cetis.ac.uk • Uptake supported by Working Groups
- 37. Conclusion • Interoperability : a Good Thing • XML the basis for IMS standards • IMS QTI tool is available • Further information about IMS and QTI from JISC-CETIS
Notas do Editor
- As the 21 st Century dawns, there is an emerging trend towards non-affiliated extranet and resource consolidation where even competitors share standards for the same underlying platforms, structures, and systems, in order to maximize the collective effectiveness of their supply chains. Large buyers and sellers will lose the ability to dictate the form and process for interbusiness transactions. Partners and vendors will contribute and interact as equal participants in trading interactions.
- We meet to develop and promote the specifications that provide the infrastructure for distributed learning. That involves an infrastructure that can support the learning activities themselves. This consists of a variety of computing and software resources as well as the enterprise computing facilities, the systems, and services that go into conducting learning. This applies to both an educational or a life-long training setting.
- Why does the world need QTI? Here are two real-world problems that QTI can solve: QTI allows interchange of questions between computer systems. The problem: At present, if you computerize your questions, they are stored in a proprietary format, for use with one particular computer system. When technology changes, or if you want to change your computer system, it's usually difficult. You either have to re-type your questions into the new system, or engage in some complicated conversion procedure. Most people who have been involved in computer assessment for some years will have found this problem. The solution: With the QTI specification, it should be possible to export from one system into the QTI format, and import directly into the other. QTI allows exchange of questions and tests between different users. 2. The problem: At present, if you have created questions or tests for your personal use, or for use within your department or organization, it is difficult to pass them onto others, as there is no standard format to describe them in. This makes sharing, exchange and re-use of material much harder than it could be. The solution: With the QTI specification, there is a standard format, and it should be possible to easily communicate questions and tests between people.
- Q: Why does the world need IMS specifications? A: In using the word Interoperability , the goal here is to have systems inter-operate with one another. Interoperability is really about exchanging some XML files. Computer software that supports IMS specifications will allow export into and import from this XML format, so for example, that if you computerize questions or tests on one system, then the material will also be usable on another system. IMS is not building a software product. It is defining technical specifications that developers and creators of products and services should incorporate so that they work together (in techno-speak, "interoperate"). These specifications are the primary IMS deliverable. So, with our specs, we expect that if someone has produced an excellent authoring system, we can use an excellent delivery system that is compatible with that excellent authoring system. So, we kept asking, “How do we get that interoperability?” That was our goal.
- The evolution of standards is very much an evolutionary process. IMS is in the position of being rather close to the emerging consensus that gives rise to early applications and early projects on the research side. And to early products and early offerings of services on the commercial side. But what actually happens over time is that the interaction of innovation and the marketplace causes the standards to evolve until they finally reach a fairly mature form and are more or less universally adopted. So, you can think of those of us in IMS as believing in the future and believing that we’re at a point where we can actually begin to realize it by meeting together and creating a common environment for developing products and services. It definitely is working. The second message of this slide is: there is more work to do. Our standards should be expected to evolve over time. We’re not trying to capture the ultimate result, because we don’t know what it is yet. Our specific role is to work on the infrastructure and the interfaces in a broad system context that promotes integration and interoperability amongst a lot of existing resources. Those resources could be content, or assessment services, or the kind of enterprise computing systems that support any kind of an institutional offering or a product offering.
- 6 IMS doesn’t work in isolation from other consortia and other sources of input. Our membership itself has representatives from all of these constituencies that are shown in this slide. We have working relationships, sometimes at the level of formal memoranda of understanding, sometimes at the level of common membership in our groups, with other standards bodies, like IEEE. The IEEE learning objects management group and the IMS group together produced our meta-data spec. We work with the AAICC (Aviation Industry/CBT Committee), especially now at the ADL co-lab to provide a migration path from where AICC has already gone. We are dividing the labor into pieces we each can do to foster and improve online learning. We have a number of university participants in IMS who are generating innovative new ways of doing distributed learning. We inter-operate with the advanced distributed learning (ADL) co-laboratory of the Department of Defense as a test bed for some of our specs. Some other agencies that are IMS members: Dept. of Education and Dept. of Energy. Obviously, a number of commercial companies are members. There are 35 Consortium members. So, the IMS activities are not taking place in isolation, and they are consciously integrative and collaborative. It makes the work hard. There is a lot of table pounding and fist shaking, and cajoling each other to keep working together, but that’s the only way to do it. Standards and specification work involve working hard to sense what the consensus is, and then working hard to express it in a form that everyone can get behind and use. But it’s very much worth the effort. You see the results of that today.
- 2 At the top level, the reason why we all believe that specifications are important is that the they support the interoperability and reuse amongst the people and the pieces that go into providing learning environments for people. The reason why the consortium exists is that the commercial companies and the users and the providers of learning all believe that a rising tide will lift everyone’s boat. And that, if we work together to build the infrastructure, then a much more exciting and a more educated, profitable world will follow.
- The IMS development process is one that is fairly well proven, now that we’ve created 4 final specifications using it. The IMS releases various specifications documents as the process of developing them proceeds. Step 1: Broadly speaking, we gather requirements from as public an inquiry as we can as to what are the real functional requirements and the essential features that the specification should cover. We gather those requirements from all the constituencies that might possibly be involved as stakeholders in the result. Step 3: Create the draft documents and the final documents as they are reviewed and worked on by many members of work groups, developers, and users. Step 4: Perform test, experience, and tool bashes to try the specs out in real life. Step 5: Introduce the specification and support its early use in order to help it to be adopted. That, for us, is a step not only in delivering the spec, but also in gathering the feedback necessary for the next cycle of development and refinement for the specification. None of that, of course, happens in isolation.
- IMS has developed and released the following final specifications: question and test specification meta-data specification enterprise computing specification content packaging specification Work is going on right now on the following new specifications: profile specification content management specification. We anticipate a number of future specs: a groups specification that describes classes or groups of learners Instructional design Accessibility for those who are disabled or use wireless communications The point of this slide graphically is that we’re working on the infrastructure. We don’t necessarily create or specify the interfaces among pieces. We believe that all the specifications have to be there and work together to provide the full system context, to deliver the potential of distributed learning.
- IMS has delivered these final specifications to be adopted by others. The QTI specification documents are on the IMS website at www.imsproject.org/guestion/index.html The most recent final QTI specification is v2.1
- it’s one thing to put a tick on a brochure to say, “Yes, we support IMS standards,” and it’s another to make a commitment to participate in the working groups and to adopt the specifications in the work place or study centers. So we appreciate all those who support the use of IMS specs in the real world.
- The working groups produce 3 documents for each specification. The three CP specs documents are each about 100 pages, available in HTML, zipped files, and PDF readme format from the IMS Website at http:// www.imsproject.org/content/packaging/index.html to access these documents. The information model explains and defines the actual data elements that we use within the standards, within the specification. If you want to understand the foundations of the spec, the information model is good to read. It is the conceptual model of the body of CP information. The XML binding document presents code that shows how to convert data in the information model into an XML record. However, there is no reason that the same data cannot be bound to different technologies than XML. For example, the data of the information model could be directly structured into the tables of a database. It is is only to promote interoperability that IMS provides an XML binding. The best practice guide is the one that we always recommend to people to start reading first. Use it to clarify terms, see examples, find more CP resources. It gets into example XML code, and you can usually equate a real content example with real XML. You can also find the Glossaries here. Use it to f ind out what this version of the specification enables you to do now.
- The IMS Content Packaging specification clearly separates the information model and the binding into two documents. The difference between an information model and a binding may be revealed in differences in naming or the letter case (upper, lower or mixed). The IMS specification reduces this to practice in a technical language, in this case, W3C's XML 1.0 that is a “textualized serialization of structured information”. [In other words, this phrase says that you can take something apart, send it through a narrow pipe, and put it back together again at the other end.]
- The binding document serves as an informative guide for translating the details of the information model into an actual XML document that will be valid against example control documents (schemas, DTDs, etc.). The XML binding is NOT the specification itself, and it is important not to confuse it in this manner. There may be other languages or formats which can be bound to the content packaging information model. Since the IMS has chosen XML as its primary data representation language, that is the binding the IMS works out and provides for specification users. Use the XML binding to create a manifest file that must be included in the zipped content package interchange file so that it can be transported over the Internet or another network. Only the manifest file needs to be in XML. A learning Management System "unwraps" or unzips the content package and organizes the content according to the manifest. A DTD, XDR or XML-Schema control file may guide the creation and validation of the manifest file.
- IMS uses XML (eXtensible Markup Language) for its standard format. This is an ASCII text format, which uses markup tags to give meaning to different parts of the text. The XML specification is produced and maintained by the worldwide web consortium, the W3, that brought us such things as HTML. More information on XML is at www.w3.org/xml/. Although this description may not satisfy the technical purists, one way of thinking about XML is that it's similar to HTML. However, that instead of using tags to describe layout and appearance as you do in HTML, with XML you use tags to define the logical structure of the data. Except with XML, there are no predefined tags; you must define your own. XML is, is a series of tags that delineate the elements so we can say, “Here is the question, the question, the question, question, the end of the question.” “ Here is the content, content, content, content, end of content. So, that’s an element. Stop” It tags the content. We can say what is actually in the content, and then finally we can put element attributes on it. For more information about XML, see the last segment of this QTI learning presentation for a list of URLs and a short summary of each. It is “Useful XML Sites and Links”.
- Recently, technologies that have been around for many years have been adapted to meet the needs of the Internet. XML’s parent is SGML (Standard Generalized Markup Language). Because of its flexibility, XML is becoming the lingua franca of the Internet because it allows all devices, platforms, and applications to share data. IMS decided to provide a DTD and an XML binding for each specification. It is possible that we could bind with other market languages, but we chose to start with XML. All IMS specifications are developed as a generic Information Model, which is then realised in XML. This realisation is presented as the XML Binding. You can bind this data model to a specific representation, and you can also do it in ISO’s abstract data model.
- Because of its structural description of data, XML allows that data to pass easily between different applications. Some applications have a proprietary data format, and others use open standards. Applications can share data even in the rendering or interpretation of the data is different. XML can act as a bridge between all these different data formats to ensure that data is understood across the entire enterprise. Data from one source can quickly be changed to be used in different formats once the data modeling and DTDs have been created.
- It’s important to mention some of the things we’re NOT doing. IMS is specifically not working on the specifications for the user interface itself. That’s an area where we expect some differentiation in the field and some competition as to what kind of interface and what kind of features it will have. Other consortia are working in that area at a level of intensity and at a worldwide contribution that IMS could never match. The Worldwide Web Consortium, for example, is developing basic standards for some of these features, and we’re following those standards. We also don’t have the intention of implementing any particular pedagogical paradigm. The goal is to provide the infrastructure that would support a broad range of pedagogical styles and scenarios of interaction. There will obviously need to be policies for these issues: How the systems are deployed and used Systems that either support or constrain the rate of innovation Systems that allow or don’t allow access Systems that allow or don’t allow interoperability Condition reuse on the observance of property rights or ownership, or not on property rights or ownership. That’s not our job to decide these things. Other groups are working in those areas too, and we’ll follow their lead, or we’ll create infrastructures that will allow many options.
- To get the QTI information on the IMS website, begin on the first page. Click on Specifications on the top right hand corner menu bar. From the drop down list of specs, select Question and Test . This will bring up another page devoted to this specification.
- The QTI page gives you explanations about QTI and allows you to download all of the IMS spec and it provides you with useful meta-data. This page allows IMS to inform you of changes and updates. The URL for this page is: www.imsproject.org/question/index.html Highlight the documents you desire to read: the question information packets, the bindings, the best practices, or certification information. This is where you will find the extra information located that we call the Resource Kit. You can select whether you will use zip files or PDF readme files for each document. Also, the new QTI final document, Version 1.01, was recently approved by the technical board, and you can find out more about it from the news section of the IMS homepage.
- It’s taken us about a year to get to version 1.0 and version 1.01 releases that are now available. It’s going to take another 9-12 months to complete version two. And we’re already thinking about what may go into version two, version three, etc.