Dr Enrico (Ric) Canale, Associate Director for Flexible Teaching & Learning, Curriculum,
Teaching & Learning Centre La Trobe University
La Trobe recently completed an evaluation of Blackboard Learn 9.1 and Moodle 1.9.7 that took into account a number of different LMS perspectives. Ric will cover the different evaluation perspectives in terms of the main issues that are common to all institutions and summarise the specific findings as they apply in the La Trobe context by way of example. The La Trobe LMS evaluation included:
System functions and features, assessed as software capabilities, Usability or ease of use, Accessibility, assessed by Vision Australia, Content migration, assessed in the interest of minimising staff time on migration of existing LMS subjects, Comparative costs, including in-house and externally hosted options for both Blackboard and Moodle, Technical fit with La Trobe IT infrastructure and future directions, Vendor responsiveness and support levels, including evidence of their capacity to support the Design for Learning Project (or replace DFL with your institution’s major T&L strategy). These and other considerations supporting La Trobe’s rationale for adopting Moodle in 2011 will be discussed.
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
LMS evaluation and rationale for moving to Moodle in 2011
1. A Recent LMS Evaluation The Task The Evaluation The Rationale for Moodle 25 June 2010 Slide 1 of 5
2. The Task Evaluate Blackboard Learn and Moodle to determine which has the best fit with La Trobe’s LMS needs. Requirement to meet Semester 1, 2011 deadline for new LMS. 25 June 2010 Slide 2 of 5
3. The Evaluation Vendor briefing documents Vendor presentations Vendor meetings Vendor written submissions Staff usability testing Student usability testing Accessibility testing Functional criteria Content migration Vendor responsiveness 25 June 2010 Slide 3 of 5
4. Key results of the Evaluation Blackboard 9.1 marginally ahead on functionality, ease of use (staff), and accessibility, with high scores for both LMS. Wider differences observed in terms of cost structures, flexibility and vendor responsiveness. 25 June 2010 Slide 4 of 5
The evaluation was highly contextualised to LTU’s needs. The task was not about determining which was the better LMS (that’s a religious war, right?). It was about determining which would be better suited to a particular junction of purpose, time and place. The impact of this was that evaluation of a range of perspectives was needed, not limited to LMS functionality and ease of use.The implementation deadline was a second very significant constraint that had a big impact on the design of the evaluation. We had very little time to conduct the evaluation and reach a clear, evidence-based recommendation.
The following elements comprised the evaluation:Vendor Briefing.Vendors were each sent a vendor briefing pack which contained:A covering letter,The functional criteria,The Technical criteria and Total Cost of Ownership Questions, Questions for Vendors document, (open ended questions for vendors to respond to),Usability Testing requirements,The Curriculum Review and Renewal at La Trobe University White Paper,The LMS subject life-cycle diagram.Vendor PresentationsVendors presented their LMS products to staff on 22 February and a straw poll was conducted after the presentations. Vendor meetingsThe Project Board and team members held separate meetings with representatives from each vendor to clarify the University's needs with respect to LMS and for the vendors to clarify options in their proposals.Vendor written responsesThe vendors responded to the technical criteria, total cost of ownership questions, including in-house and externally hosted options for both Blackboard and Moodle and ‘Questions for Vendors’ documents by March 5. Staff Usability testingVolunteer staff were given tasks to perform in one of the LMS products and rated the product for ease of use. Student Usability TestingUsabilityOne was commissioned to undertake a fully online assessment of ease of use based on analysis of set tasks performed by first year students.Accessibility testingVision Australia was commissioned to undertake desktop research and typical use-case scenario testing to assess strengths and weaknesses of the two LMS products in respect of accessibility and to advise on implementation for high levels of accessibility. Functional criteriaA comparison of Blackboard and Moodle, based on functional criteria was conducted by the LMS Evaluation Officer.Content migrationVendors were provided with a sample LMS subject (in Blackboard CE6 format) for conversion to their LMS format. In response to a staff concern that conversion to Moodle resulted in the loss of deeply nested structures, NetSpot provided two additional versions that effectively handled highly structured LMS subjects as alternatives to the standard conversion.Vendor responsivenessThroughout the evaluation phase, the efficiency and effectiveness of vendor responses to our requests were monitored.
An interesting observation is that if you limit yourself to the products alone you get one answer. If you bundle the products up with their respective cost structures, technical environments and vendor relationships, we get a different result. I think this deserves further explanation and in particular I’d like to take the next few minutes to consider to what extent our observations are contextual and peculiar to La Trobe and to what extent they are structural and inherent in the two different business models represented by Blackboard and Moodle/NetSpot.
