1. Story development Evaluation Plan
Introduction
This evaluation plan describes the scenario of the target activity to be evaluated and the
basis of the evaluation.
Background
We are designing a course module on Agile Software Development as described in
http://www.slideshare.net/itanagimenes/olds-moocweek3artefacts It is a distance
learning course for a large international audience. It includes mostly professional
participants but younger learners should also be considered. This module contains the
following activities: Initialize; Propose a system; Develop user stories; Set project builds;
Program & Test; and Iterate.
Purposes
One important decision we have to make is if we should adopt a synchronous
communication mechanism to support the activity Develop user stories. As the module
is in the context of distance learning, it might have people from around the globe with
different time zone. Moreover, this people work and have restricted time available.
However, we have the feeling that synchronous interaction may improve the learning
and also give them a sense of belonging making a better pathway to work in group.
Therefore, the purpose of the evaluation is finding evidences to support our decision of
adopting or not a synchronous communication mechanism to support the activity
Develop user stories.
Audiences
Experts, Instructors and Learners of distance learning courses in computer science taken
from a sample of potential countries.
Decisions:
If the evaluation leads to the adoption of a synchronous interaction mechanism we will
have to search for a mechanism and evaluating it regarding scale (number of users,
licenses, support required and cost). If the decision is for not adopting a synchronous
interaction mechanism we will have to look for strategies to engage the students in the
tasks so that they keep motivated and interact with their partners.
Questions
Experts: Do you believe, based on previous experience, that a typical learner of this
course would be motivated to use a synchronous mechanism, like Google hangout to
support the activity?

2. Instructors: Do you think instructors can motivate and support learners to use the
synchronous mechanisms to improve the interaction in the activity?
Learners: Do you think it is difficult to engage in synchronous activities, due to time or
technological problems?
Methods
First we will undertake a systematic review on the use of synchronous mechanisms and
tools in distance learning. The objective is to find out existing work in similar context
that has dealt with this issue. Second we will apply the questionnaire to a sample of
people from different countries, than we will apply statistical methods to analyze the
results. The questionnaire will be applied online.
Sample
We will aim to get a sample from 2 countries in each continent big enough to apply
statistical methods.
Instrumentation:
Online questionaires
Email to contact potential participants
Excel to analyze data
Word for report analysis
Limitations
Threats of validity can be cultural differences. Not to have enough people to answer the
questionnaire. Time to get the answer is limited due to the deadlines to launch the
module.
Logistics
The course team leader will lead the process support by Mrs. Bee.
Time Line
Systematic review: 2 months
Questionnaire preparation: 1 month
Questionnaire application: 1 month
Analysis and report: 2 months.
Budget
Salary of the team leader + Mrs. Bees, 12 hours a week for 6 months.