by Ahmet YUSUF

1. Evaluation
Language Curriculum Design
chapter 8
By AHMAD YOUSSEF

2. THE AIM ??
 The aim of the evaluation in the curriculum
design process is to decide if the course is
successful and where it needs to be improved.

3. What is an Evaluation?
 evaluation looks at all aspects of curriculum design to
see if the course is the best possible .
 Evaluation requires looking both at the results of the course,
and the planning and running of the course .

4. evaluation answer questions like the
following:
 • Is the teaching of the course is high standard?
 • Is the course preparing the learners properly for their use of
English at the end of the course (e.g. to pass the TOEFL test, to study
in an English-medium university, to work as a tour guide)?
 • Are the learners satisfied with the course?
 • Is the course cost effective?

5. Steps in an Evaluation 9
 1 Find who the evaluation is for and what kind of
information they need.

2 Find what the results of the evaluation will be used for –
to improve the course, to decide whether to keep or get
rid of the course.

3 Decide if the evaluation is necessary or if the needed
information is already available.
4 Find how much time and money are available to do the
evaluation.

6. 5 Decide what kinds of information will be
gathered.
 • Amount of learning
• Quality of learning
• Quality of teaching
• Quality of curriculum design
• Quality of course administration
• Quality of support services – library, language lab, etc.
• Teacher satisfaction
• Learner satisfaction
• Sponsor satisfaction
• Later success of graduates of the course
• Financial profitability of the course.

7.  6 Try to gain the support of the people involved in the evaluation.

7 Decide how to gather the information and who will be involved in
the gathering of information.

8 Decide how to present the findings.

9 Decide if a follow-up evaluation is planned to check the
implementation *application * of the findings.

8. Purpose and Audience of the Evaluation
 Kiely and Rea-Dickens (2006: 225–271) make a useful
three-way scope distinction:
(1) teacher-led evaluations,
(2) management-led evaluations.
(3) large-scale evaluations

9.  A course evaluation can be an expensive and time-
consuming procedure. Like if a university wants to
evaluate a course : outside evaluators/fees/hotels/
tickets .
 Because of this investment of time and money, it is
important that an evaluation is well focused and well
motivated.

10. The first critical step is to find out who the evaluation is for
and what kind of information they value. This step is very
important because :
 Firstly, it helps determine the degree of confidentiality of the
evaluation.
 Secondly, it helps determine what kind of information should be
gathered and what kind of information should not be gathered.
 Thirdly, knowing who the evaluation is for is useful in determining
whether the data to be gathered will be provided willingly or
reluctantly.

11. In this stage , the evaluator should be able to tell
the person commissioning the evaluation:
(a) whether the evaluation is worth doing
(b) whether the evaluation is possible
(c) how long it might take
(d) how much it might cost
(e) whether the evaluator is willing to do it
(f) what kind of evidence the evaluation will gather.
If all the people are in agreement, then the evaluation can
continue.

12. The Types of the Evaluation

13. NOTE
In evaluation a teacher’s quality cannot be reliably or
validly assessed from watching one or two lessons. The
lessons may be especially well prepared for the
evaluation, or the teacher could be nervous and having
a bad day.

14. Have you been observed before ?

15. THREE FACTORS OF EVALUATION
Cognitive factors
Involve learning and teaching and the
gaining of knowledge,
Typical questions would be:
Affective factors
involve feelings of satisfaction and
attitudes
Typical questions would be:
Resource factors
involve costs, profit, learning
resources such as books, classrooms,
visual aids, tape recorders,computers
Typical questions would be:
“How much has been taught ?” “Are the learners pleased with the
course?”
“Is the library adequate for the needs
of the learners?”,,
“How much has been learned?”, “Do the staff work well together?”, “Are the classrooms large enough?”
, “Has the course improved learners’ “Do the teachers feel the course is
effective?”
“Does the course make a financial
profit?”

16. Gaining Support for the Evaluation
 A course evaluation looks for strengths and weaknesses,
but it is naturally the weaknesses that cause concern.
Because :
 They will criticize your weaknesses
 These are the points that you need to improve

17. THE QUESTION IS
HOW CAN YOU GAIN SUPPORT FOR THE EVALUATION ?
How can you get honest data and information ?
How can you proceed the evaluation effectively ?

18.  it is necessary for those involved in the evaluation, particularly those
who are sources of information, to feel that the evaluation is
worthwhile and not personally threatening to their “face”
and their job security. This will require meeting with those involved
and involving them in the planning and carrying out of the
evaluation.
 So Gaining the agreement and cooperation of the staff
a prerequisite (condition ) to doing the evaluation.

19. The assumptions behind an evaluation
usually are that(before ):
 (1) this course is worth improving,
 (2) the people running and teaching the course are
capable of improving it,
 (3) the people involved in the course have the freedom
and flexibility to make changes to the course,
 (4) the improvements will make it a better course for all
concerned.

20. Gathering the Information

1
Interviews
2
Self-report scales
3
Observation and checklists

21. Interviews ?
• the interviewer has a procedure
and a set of questions to follow
and generally keeps to these
structured
• No prepared questions
unstructured

22. The table shows what information you want to get and the tools for that

23. Self-report scales
??????????????????????????????
 A self-report scale is a type of survey or questionnaire
 ( it could be for mostly learners and teachers of the course )
 self-report is any method which involves asking the participants
about their feelings, attitudes, beliefs and so on.

24. Self-report scales are very efficient when
1
• (1) You survey a large number of people,
2
• (2) there is a large number of pieces of information
to gather,
3
• (3) there are very clear focuses for the evaluation
4
• 4) there is a need to summarize the data to get a general picture,
and compare it with previous evaluations or other courses.

25. There are several dangers of self-report
scales:
 1. They tend to result in average results , This is usually avoided by also
showing how many people responded with 5 (excellent), how many responded
with 4 (very good) and so on.
 2.Self-report scales involve pre-determined questions and types of answers.
 3. Self-report scales are often used for student evaluation of teaching and
giving them short period of time to answer . This can be partly avoided by
giving them time and encouraging learners to reflect on the whole course and
by allowing them to discuss in pairs or small groups before responding
individually.

27. Observation and checklists
 a course can be evaluated to some degree through
observation and analysis. These include analyzing the
course book, observing learning in lessons, observing
teaching, and observing the performance of learners after
the course.
 The Observation has two types structured
unstructured

28. the observer or
analyzer has a
checklist of
features to look for
and evaluate.
the structured
observation with a
checklist makes sure
that everything that
was thought to be
important to look at is
looked at.
structured
the observer or
analyzer tries to see
what is there without
too many
preconceptions.
The unstructured
observation may pick
up features that may
not have been
included in any
checklist.
unstructured

29. A practical checklist should not be too long,
easy to use, and easy to interpret its results.
should
not be
too long,
easy to
use
easy to
interpret
its results

30. Have you been observer before ?

31. The disadvantages of
checklists
 (1) they may “blind” the
observer from seeing other
important features that are
not on the list,
 (2) they tend to become out
of date as theory changes
(consider the course book
evaluation form designed by
Tucker (1968)), and
 (3) many checklists are based
on the assumption that
summing the parts is equal to
the whole.
The advantages of checklists
 (1) they ensure that there is
a systematic coverage of
what is important,
 (2) they allow comparison
between different courses,
lessons, teachers etc., and
 (3) they can act as a basis
for the improvement of a
course through formative
evaluation.

32. Formative Evaluation as a Part of a Course
 An important question in an evaluation is : who will be
involved in the evaluation ?
 The involvement of curriculum designers and teachers in
an evaluation can be an important part of professional
development.
 The evaluation is a part and it doesn’t end but it
continuous .

33. The formative evaluation can be planned as a part
of curriculum design in the following ways:
 1 Parts of the curriculum design can be negotiated between the
teacher and the learners This may include negotiation of
classroom activities, some of the goals of the course, and some
assessment procedures. This negotiation is a kind of evaluation
with immediate effects on the course.
 2 The course can include periodic and systematic observation
of classes by teacher peers.
 3 The staff hold regular meetings to discuss the progress of the
course.

34.  4 Teachers are required to periodically fill self-evaluation
forms that they discuss with a colleague.

5 Learners periodically fill course evaluation forms.

6 Some class time is set aside for learner discussion of the
course and
providing feedback for teachers.

7 Occasionally an outside evaluator is invited to evaluate
aspects of the course.

35. The Results of an Evaluation
 When an evaluation has been done, the results need to be presented. This
presentation involves ethical issues, particularly those of confidentiality (privacy
)and consideration for the feelings of others.
 Because The results of an evaluation may also be threatening to the individuals
concerned, especially if weaknesses are revealed.
 The results of evaluations of teaching are usually only available to the teacher
concerned and perhaps to the head of the department. They are not seen by
colleagues.
 # In the comparison of the evaluation there are two ways
norm-referenced and criterion-referenced?

36.  In such evaluations particular comments by students may be reported, but the
names of the students who made these comments are not reported.
 Most evaluations involve a written report, or in some cases two written
reports
 A report of an evaluation needs to indicate the quality of the course and it must be
made clear what the standard for the measure of quality is.
 The written report will usually be accompanied by an oral report. This oral report
has two purposes,
(1) to make sure the written report is clearly understood, and
(2) (2) to say things that could not be put tactfully in writing.

37.  Is the report the end of the evaluation ?
 The report needs to be considered and then
acted on. The evaluation procedure may involve some
later follow-up to monitor the effects and implementation
* application* of the evaluation for the planning of future
courses.

38. Summary of the chapter

1 Discover the purpose and type of the evaluation.
2 Assess the time and money needed.
3 Decide what kinds of information to gather.
4 Gain the support of the people involved.
5 Gather the information.
6 Present the findings.
7 Apply what has been learned from the evaluation.
8 Do a follow-up evaluation.