1. ESP Materials Design And Evaluation<br />Natalia Ladik, Department of International Relations, Belarus State University<br />Far from being at the end, we are at a turning point in the history of both the English language and the world; that the world in which English has become so important is in a state of major transition ñ in terms of economic relations, of structures of government, of technological progress, and in the way social identities are constructed and maintained. ÖIf we are in the middle of the story, so many different endings ñ or at least next chapters ñ seem possible. (David Graddol 1999:5) <br />Introduction <br />Nearly all teachers of ESP find themselves involved in materials writing sooner or later, because textbooks are seldom written with a particular group in mind. ESP materials focus on aspects of English specific to the subject area. While producing our own materials, we should check that ESP materials meet target needs and that the language taught matches the language that the students will use. Besides we should put emphasis on the development of specific skills and strategies for operating in the ESP context. Project work approach fits comfortably within TESP since it responds to the learnersí specific purposes, the students operate in the target language, it is activity and research based, multi-skilled (develops linguistic & research skills), involves learners in both individual and group work, uses authentic material, is set up in collaboration with subject teachers. It provides an opportunity for real world and classroom experience to overlap, gives learners a feeling of achievement. Besides project can also encourage positive classroom behaviours such as co-operation, enjoyment, motivation and interest. It can be an elegant culmination of the ESP course. Here is described a project which have been carried out by the students of International Relations of the Belarusian State University. <br />Introduction of parameters of the project <br />Level Intermediate to Advanced Age Junior students Time 20 hours over 8 weeks <br />Objectives <br />- to define some qualifications desirable in members of the Foreign Service/Ministry of Foreign Affairs; - to compare the work in the Foreign Office/MFA with the work abroad; - to learn about the diplomatic etiquette on public occasions; - to read, analyse and write a letter of credence, a letter of recall; - to encourage an awareness of language problems in diplomatic intercourse; - to establish a working rapport with university lecturers or acquire contacts with diplomatists; - to collect printed and visual materials from the media in various languages in addition to English; - to conduct interviews and organise meetings with professionals; - to exercise note-taking; - to provide an opportunity for audio or video recording; - to design questionnaires; - to make oral presentation and reporting or to produce a video display; <br />Skills <br />All four skills (speaking, listening, reading, writing) and a whole range of sub-skills are integrated towards one objective. <br />Speaking skills: discussion, negotiation, information seeking, conversational formulas, conducting interviews or surveys, oral presentation and reporting Listening skills: listening to video and audio materials for detail, for specific information or gist, transferring and summarising information, outside speakers questions and answers, listening to each groupís presentation Reading skills: skimming, scanning, sequencing, briefing/making summaries, inferencing, critical reading, finding additional reference materials Writing skills: note-taking, formal letters of invitation, appointments, recording information from interviews, questionnaire design, using sources & organising, written presentation of the project Interview techniques: active listening (combining both hearing and processing), summarising, and asking open questions; sub-skills ñ interrupting, repeating, eliciting, probing, concluding Presentation skills: Voice: speed, loudness, variation, pausing, clarity of intonation and pronunciation; Body language: appropriateness of gestures and movements, firm stance, smile; Visuals: introduction to, relevance of, pin-pointing reason for, few words in and not reading them aloud; Structure: introduction, purpose, logical points, time, what is to come, summary, recommendations, powerful end; Language: no major misunderstandings, as appropriate and varied as reasonable; Overall Impact: impression of confidence, message come across, listeners included and not bored (Dudley-Evans & St John, 1998:269). <br />Equipment: audio recording equipment (optional); video camera and film (optional); printed or visual materials <br />Teacherís preparation <br />1. Select, modify and provide paper-based materials (magazines, books, text-books, newspapers, brochures etc.), for the students to read, discuss, collate and refer to. Note down relevant film, video or audio programmes, which may be useful. Later you can use learner-generated material. 2. Invite a guest speaker or another group of students for debate or a discussion, a seminar, or a workshop session (if you have access to video or audio-recording equipment you may record some parts for further classroom discussion of specific language items). 3. Make use of role play, information-gap, simulation exercises in the classroom during the preparatory stages of the project. <br />Note: The studentsí motivation and interest determine the success of a project. On the day you actually start work on the project, it is essential to try to capture the studentsí imagination immediately. This might be with something like a video, a meeting with a professional, an interesting story which provokes the desire to respond by talking, writing, acting out, or in some other way. That is why the key words, especially at the initial stages of the project, will be ëto suggestí, ënegotiateí, ëencourageí, ëinspireí, ëinvolveí, ëstimulateí, and ëget positive reactioní. <br />Procedure <br />The structure of a project can be shown in the following sequence (Shalom 1996: 98): Project production drafting outline discussion topic selection <br />Stage 1 <br />The starting point has to be the choice of topics. Find out what questions about diplomatists and diplomatic practice in general correspond to the interest of the learners. In collaboration with them discuss what issues they would like to explore in more depth. In my experience the chosen topics of interest were: · The qualifications desirable in diplomatists (members of the Foreign Service) · Diplomatic ethics and etiquette on public occasions · The duties of a diplomat abroad · Language problems in diplomatic intercourse <br />Stage 2 <br />During the next four weeks the topics are discussed. Here I will describe only some approaches which have been tried and have proved to be successful. · Ask your students, as a homework task, to define the notions diplomacy, diplomat or diplomatist consulting some dictionaries and making use of quotations (some of them can be disputable), e.g <br />Diplomat ñ is a government official, usually in an embassy, who negotiates with another country on behalf of his or her country. BBC English Dictionary <br />Diplomacy - is the management of international relations by negotiation; the method by which these relations are adjusted and managed by ambassadors and envoys; the business or art of the diplomatist. Oxford English Dictionary Diplomacy is to do and to say the nastiest thing in the nicest way Isaac Goldberg <br />The worst kind of diplomatists are missionaries, fanatics & layers; the best kind are the reasonable & human sceptics. Thus it is not religion which has been the main formative influence in diplomatic theory; it is common sense. Sir Harold Nicolson (British diplomatist, author, critic, & journalist; 1886-1968) <br />Diplomacy ñ is the conduct of relations between nation-states through their accredited officials for the purpose of advancing the interests of the appointing state. Peter A.Toma, Robert F.Gorman <br />Amateur diplomatists are prone to prove unreliable. It is not merely that their lack of knowledge & experience may be of disadvantage to their governments, it is that the amateur diplomatist is apt out of vanity and owning to shortness of his tenure to seek for rapid successes;Öthat he has not acquired the humane and tolerant disbelief which is the product of a long diplomatic career and is often assailed by convictions, sympathies, even impulses; that he may cause offence when he wishes only to inspire geniality; and that in his reports and despatches he may seek rather to display his own acumen and literary brilliance than to provide his government with a careful and sensible balance-sheet of facts. Sir Harold Nicolson <br />· Encourage a brain-storming session on the questions ñ what personal and professional qualities and attainments a contemporary diplomatist needs for a job; and what qualities the would-be candidate for the diplomatic career shouldnít possess. Donít push your students to give the answers immediately. They have to think it over. Either you or one of your students can write the suggested qualities and ideas on the blackboard. You can read or hang some quotations on the blackboard to stimulate a discussion, e.g. <br />He [the ambassador] must regard himself as an economist, a commercial traveller, an advertising agent for his country; he wields the weapon of culture for political ends; he promotes scientific and technical exchanges and administers development aid. He cannot wholly detach himself from the technicalities and personal inconveniences which accompany the battle for intelligence. He must concern himself with the relations not only governments, but also of politicians, scientists, musicians, dancers, actors, authors, footballers, trade unions and even women and youth, these two new technical professions in the modern world. But he continues to have a basic political job to negotiate with the other government and to keep his own government informed about anything in the country to which he is accredited which affects his countryís interests. Lord Trevelyan, Diplomatic Channels <br />Ambassador is an honest man sent to lie abroad for the good of his country. Sir Henry Wotton, 1604 <br />In a word, the ambassador is out of the serious play. He is rarely viewed as the best channel of communication with the head of government, he is not charged with the critical negotiating tasks, he is subordinated to visiting Washington officials & negotiating experts (visitors would be amazed if he had the temerity to offer substantive views), & - to make the message unmistakably clear ñ his expected duties are those of the hosteler. Ambassador J.Robert Schaetzel <br />The good British diplomatist is tolerant and fair; he acquires a fine balance between imagination and reason, between idealism and realism; he is reliable and scrupulously precise; he possesses dignity without self-importance, demeanour without mannerism, poise without stolidity; he can display resolution as well as flexibility, and can combine gentleness with courage; he never boasts; he knows that impatience is as dangerous as ill-temper and that intellectual brilliance is not a diplomatic quality; he knows above all that it is his duty to interpret policy of his government with loyalty and common sense and that the foundation of good diplomacy is the same as the foundation of good business ñ namely credit, confidence, consideration and compromise. Sir Harold Nicolson <br />· Bring to the class some materials, handouts on the topic in question for the students to skim, take notes and exchange with another group of students. Information can be set up in the form of a table, e.g. <br />Attributes needed in a diplomat: accuracy, courage , dignity, warm heart, accuracy courage, dignity, warm heart, logic, charisma, good manners, industry,good temper, prudence, initiative personality, intelligence, tact, loyalty, charm, modesty, discernment, patience, hospitality, reliability, retentive memory, self-confidence, versatility and adaptability, sense of humour integrity,truthfulness, knowledge, unruffled calm, linguistic proficiency; <br />Attributes a diplomatist should refrain from displaying:coldness, superfluous gesticulation, domineering manners , air of superiority, snobbishness, poker face,rush decisions, starchiness and pomposity, cynicism, emotional effusiveness, high gloss, lack of linguistic proficiency, heartiness and bluff bonhomie, deception, disloyalty, oppressive display of intellectual powers. <br />Note: Your students may object and attribute some qualities, considered in some sources as negative, to positive and vice versa, e.g. they can claim that a certain capacity for deception is needed in a diplomatist and a poker face isnít always out of place in diplomacy. Donít worry if your students sometimes object and criticise. Provide a willing ear or encourage a debate. At least open discussion generates spontaneous language practice. <br /> Recommendations <br /> While discussing ëDiplomatic Ethics and Etiquette on Public Occasionsí you can conduct a class survey to find out whether your students know the rules of precedence amongst Ambassadors and foreign Ministers, the etiquette which should be followed when members of the Diplomatic Corps are entertained, introduced etc., i.e. diplomatic protocol. You can speak with your students about the image and appearance of an ideal diplomat/woman president. Language of gestures, poses and facial expressions can be a productive topic for discussion. The initial stage of the project on the topic ëThe Duties of a Diplomat Abroadí may benefit from a visit to an international organisation, or a diplomatist coming in to talk to the students. You can initiate a discussion about diplomatic spouses who interrupt or sacrifice their own careers by repeatedly being sent abroad. Tell your students about the Diplomatic Family Service Association, part funded by and housed in the Foreign Office, which has been fighting for some kind of spouse compensation package or allowance based on loss of earnings and pension rights. As a practical example read, analyse and teach your students to write a letter of credence, a letter of recall. Many aspects of linguistic difficulties encountered in diplomatic intercourse can be discussed when you come to the question of ëLanguage problems in diplomatic intercourseí. Here are some problems confronted in diplomatic life: a great number of official languages in use; differences in the ëlanguage conventioní between nations supposedly speaking the same language (BrE, AmE and other major varieties Can, Aus, NZ, SA English); different connotations; difficulty and technicality of subjects. Humour, quotations, references and proverbs are some of the most difficult things to convey in another languages. You may practise with the students two types of interpretation, the so-called consecutive and simultaneous interpretation; encourage a discussion of the necessary qualifications and qualities of a good interpreter. You can read with your students about language learning in the Foreign Office and revise nationality words. <br />Stage 3 <br />· After the content and scope of the project have been negotiated and students have divided into teams, they move out of the classroom to conduct and record interviews, to gather printed and visual information and then transform it into spoken and written format. A healthy rivalry can take place. · An individual or a team can approach their task from a different angel. In my experience one female student who was working on the topic ìDiplomatic Ethics and Etiquette on Public Occasionsî made a wonderful performance about the image of a business woman pursuing diplomatic career. Her idea was so fresh and new that everyone listened to her presentation with a genuine interest. · At this stage you should act as a monitor, who keeps track of what the students are doing and helps them to deal with difficulties. Make sure that everyone is involved. · Draw up a detailed (but flexible) timetable. <br />Follow-up <br />1. Your students may prefer to present the results of their work to each other in the classroom. But offer them a challenge to invite university authorities and inform other teachers about the presentation, or encourage them to organise a debate or forum with their fellow students. If applicable, you may record the studentsí presentations for analysis, etc. It will be a valuable material for your further work. 2. Alternatively, if you have a fairly ambitious group, together with the students you may decide to contact the media and try to gain publicity through newspapers, magazines, radio, and television. 3. Written practice can also be based on the studentsí research and can be tackled in a variety of ways - from the formal report, to seminar or broadsheet/poster presentation, booklet, chart, newspaper article or memorandum to students of international relations. 4. It is very important to evaluate the studentsí work. You can design a certificate for successful performance of the project. Your students would appreciate if you and your invitees add to the formal certificate your personal observations, words of praise and comments about the work and presentation of each team and individual. <br />Evaluation <br />1. First of all I can report that all materials used on the project were relevant to, and of interest to, the students. 2. Topic and material selection was determined by a process of continuing negotiation between teacher and learners. 3. Communicative approach was used in the classroom. 4. Various teaching-learning techniques were applied: whole group, small-group work, pair and individual work. 5. Medium: text, video/audio sources, speech. 6. Materials used in the project work contained many exercise, activity or tasks in the various skills: information/opinion gap, information transfer, ranking, problem solving, role plays, language games, simulation exercises, etc. 7. Skills and sub-skills were integrated: reading, listening, writing, speaking, interview techniques, presentation skills. In general the project came as a pleasant surprise for specific-purpose learners, namely international relations students. They realised that the teacher was interested in their particular needs and was prepared to go to some trouble to explore the complicated subject area of international relations. <br />References <br />1. Bilan V Official and Diplomatic Writing, Minsk,1999 2. Dudley-Evans T., St John M. J., Developments in English for Specific Purposes, Cambridge: Cambridge University Press, 1998 3. Dickie J., Inside the Foreign Office, Chapmans Publishers, 1992 4. Fried-Booth, D.L., Project work, Oxford: Oxford University Press, 1986 5. Graddol D., ëWhat will English look like in 2050?í in IATEFL, Issues 149/6-7, 1999 6. Hill D., Projects. English Teaching professional, 13/10, 1999 7. Hutchinson T., Waters A., English for Specific Purposes, Cambridge: Cambridge University Press, 1987 8. Shalom C., ëPoster Presentation on the Pre-sessional EAP Courseí in Evaluation and Course Design in EAP, Volume 6/1, 1996 9. Toma P. A., Gorman R. F., International Relations: Understanding Global Issues, Brooks/Cole Publishing Company, Pacific Grove, California, 1991 10. West R., ESP Module Units 0,1,2,3,9, School of Education, The University of Manchester, 1992 <br />Introduction<br />Evaluation is a part of the control function of management. Its primary purpose is to determine whether the plan has been implemented in order to achieve the present objectives. In the classroom, the accomplishment of these objectives is not only as perceived by the teacher but most importantly by the students. In the educative process, the students have the unique role of being the product and the consumer rolled into one. Hence, the quality and effectiveness of instruction should be centered on students’ learning and the satisfaction of their personal goals. The complexities involved in the learning process can be reflected in a good evaluation instrument.<br /> <br />Evaluation of teaching performance can be both formative and summative. Teaching performance evaluation for formative use serve as a monitoring device so that management will be appraised with what is going on in the classroom. The information gathered here will be used to redirect the teaching- learning process toward the accomplishment of goals. <br /> <br />On the other hand, if the evaluation is used for a summative purpose, evaluation result is considered as an input information in the administrator's decision making process as regards the individual faculty. <br /> <br />Our instrument for Faculty Evaluation by the students, inspite of its imperfections, provides feedback data/information as to how evaluation results are used by the administrators towards improvement of instruction.<br /> <br />Apparently, evaluation results are used for summative purposes, i.e., for granting cash incentives and FSAS points. For faculty with T-status, the rating is one of the bases for the renewal of the contract, among others. <br /> <br />Time constraints do not allow content analysis of the instrument. Hence, at this juncture, the issue on reliability and validity of the instruments is not determined. This does not mean, however, that this will be presumed. Further study will be pursued in the future and appropriate recommendations will be made on this regard. Back to top<br />Faculty Evaluation Instrument<br />The Silliman University Instruments for Faculty Evaluation are of two kinds and used by four groups. The first is the Student Instrument for Evaluating Teachers. This instrument is used by students in evaluating the teacher's classroom performance with the following parameters: Openness/Interpersonal Relation, Objectives/Content of the Syllabus, Teaching Techniques, Teaching Devices/Learning Materials, Mastery of Subject Matter. <br /> <br />The second instrument is used for self-evaluation of the teacher by his/her peers (3 peers), and evaluation of the teacher by his/her superior. This instrument assesses the teacher on two areas : Teaching Competencies and Teacher's Personality and Interpersonal Relation.<br />Back to top<br /> <br />Purposes and Objectives<br />As mentioned earlier, an evaluation rating is used as a monitoring device that provides data/information whether the objectives of the organization have been achieved with reference to desired performance of personnel. Hence, the primary objective of teacher evaluation is to obtain data/information of the teaching performance of the faculty in the classroom. This, in essence, should provide feedback on the teacher's effectiveness in facilitating learning and his/her efficiency in managing time and other resources to achieve the course objectives. <br /> <br />Towards this end, each evaluation component should attain its corresponding objective: <br /> Student evaluation - To asses the teacher's effectiveness in facilitating learning. Peer evaluation - To asses the teacher's capability to relate with <br /> colleagues and his/her contribution towards the creation of a wholesome working enviroment within the <br /> unit. Self evaluation - To asses oneself of the effort put to the job relative to <br /> his/her best. Superior evaluation - To asses the teacher's effectiveness in teaching and in performing funtions inherent to the job outside the <br /> classroom.<br />Back to top<br /> <br />Assumptions<br />To lend credence to the evaluation, users of the data (quantitative analysis result) adopt the following assumptions: <br />- T he instruments used are valid and reliable. - The evaluators, i.e., students, self, peer, and superior are candid, sincere, and truthful in making their ratings. <br />Unless the reader of this report anchor his/her view on these assumptions, whatever information that maybe derived from this analysis will be null and void. <br />Back to top<br />Evaluation Components<br />Faculty evaluation has 4 parameters: Student Rating, Self-Rating, Peer-Rating, and Superior Rating. An evaluation set is composed of student ratings from two classes of the teacher evaluated, one self-rating, three peer ratings, and two superior ratings. <br /> <br />STUDENT RATING <br />Student rating refers to the students’ evaluation rating of the faculty. Two of the classes handled by the faculty are used in these ratings. One class is selected by the faculty while the other is randomly selected by the evaluation coordinator or the designated staff of the Office of Instruction. <br />SELF RATING <br />Self-rating refers to the rating of the faculty of one's own effort in teaching. One’s self-rating is required for each set of evaluation. Only full-time faculty members are required to submit to this rating. <br /> <br />PEER RATING <br />Peer rating refers to the rating of the faculty by his /her colleagues in the discipline taught. Peer ratings are for full time faculty only. As practiced, three colleagues are asked to rate a faculty. However, in cases where this is not possible due to the size of the department/unit, a maximum combination will be used. <br /> <br />SUPERIOR RATING <br />Superior rating refers to the rating of the faculty by the immediate supervisor. The faculty may be rated by the Department Chairperson and /or the Dean/Director of the College/School. If more than one superior will rate the faculty, the average of the ratings will be taken. <br />Back to top<br />Evaluation Procedure<br />- Evaluation is conducted every semester. <br />- Student evaluation of the faculty starts two weeks before the midterm examination <br /> and ends one week before the final examination week. This covers a period of <br /> about nine to ten weeks. <br />- The forms for Self, Peer, and Superior ratings are distributed to each faculty <br /> member and administrators concerned at the start of evaluation period. <br /> Accomplished forms should be returned to the Office of Instruction before the first <br /> day of the final examination week of every semester. <br />- Regular faculty submits to one evaluation annually. <br />- Temporary faculty submits to one evaluation every semester <br />- Evaluation of Visiting Professors is conducted upon request <br />- Part-time faculty and Graduate Teaching Fellows submit to student ratings and <br /> superior rating every semester <br />Back to top<br />Evaluation Instrument<br />There are only two types of instruments used in evaluating of the faculty performance: the student evaluation instrument and one for the other three parameters. <br /> <br />STUDENT INSTRUMENT FOR RATING TEACHERS <br />This instrument is composed of five sections: Openness/ Interpersonal Relation, Objectives/Content of the Syllabus, Teaching Techniques Devices/learning Materials, and Mastery of the Subject Matter. Each section has equal weight in the computation of faculty rating by students. <br />Self, Peer, and Superior evaluation use the same Instrument. This instrument has only two sections : Teaching Competencies and Teacher's Personality and Interpersonal Relation. <br />Back to top<br /> <br />Computation of Overall Rating<br />The annual overall rating of regular faculty is the average of one evaluation set. <br />The annual overall rating of a faculty on temporary status is the average of two evaluation sets. <br />Overall ratings for part-time and Graduate Teaching Fellows are not computed. Their student and superior ratings are used for administrative and teaching enhancement purposes. <br />Back to top<br />Why is evaluation necessary?<br />Training evaluation can help to define the training objectives more sharply, get rid of unnecessary training content, ensure that training methods meet the requirements of trainees, relate them to their training needs and reduce training costs. Evaluation is part of the whole process of transformation that is education and training. It can ask whether the aims of the curriculum and the learning objectives have been achieved, what learning has taken place, and how. It can also ask what difference this has made to the learners and to their lives, their work and to their relationships to others.<br /> <br />Evaluation may be either summative or formative. Summative evaluation concentrates partly on whether aims and objectives have been achieved and is usually done soon after the end of the programme, since the information required needs to be obtained while it is still fresh in the minds of those involved. Impact evaluation is carried out later, in order to allow enough time to pass so that the longer-term rather than the immediate effects can emerge. In particular, it will often need some time for the views of those involved to become clear and be put into perspective. <br /> <br />Formative evaluation is the on-going process of assessing and re-assessing the progress being made throughout the course, the direction in which the course is heading, and the speed at which the aims and objectives are being achieved. It can also be called ‘monitoring’. Although formative evaluation will be undertaken throughout the course by the teacher-trainers, often in association with the learners, provision should be made in the schedule of work for more systematic opportunities for review and assessment. Regular orientation is needed throughout the course to gauge how far the course has reached and how much further there is to go. By highlighting the areas that are successful and identifying those which need revision, alterations to the course can be made. The aim of formative evaluation is to provide the basis for course improvement, to determine the need for modification and ultimately to lay the foundations for future planning. It is a continuing process of critical reflection on experience leading to action. <br /> <br />