HỘI THẢO KHOA HỌC CÁC TRƯỜNG THPT CHUYÊN KHU VỰC DUYÊN HẢI & ĐỒNG BẰNG BẮC BỘ LẦN THỨ VIII CHUYÊN ĐỀ BỒI DƯỠNG HỌC SINH GIỎI Assessment and Testing in EFL Class at High School Level Người thực hiện: Nguyễn THỊ CẨM TÚ Đơn vị: Trường THPT Chuyên Nguyễn Bỉnh Khiêm - tỉnh Quảng Nam I INTRODUCTION he classroom test is one of the most important aspects of the teaching learning process, and designing the classroom test is one of the most demanding responsibilities facing school instructors Unfortunately, most of teachers of English at Vietnam’s upper-secondary school have had little, if any, preparation in the craft of wring tests; consequently, the process is not only difficult, it is also frustrating and often ineffective Writing test questions will always be demanding, even for experienced instructors, but it will be less frustrating for those who know the techniques for writing specific types of items and have some guidelines for general test constructions The objective test is only of many ways in which students can be evaluated Tests can be formal or informal, oral or written; and no one form of testing is necessarily better or worse than another It is useful to consider multiple-choice items in some detail, as they are undoubtedly one of the most widely used types of items in objective tests However, since the multiple-choice item is one of the most difficult and time -consuming types of items to construct, numerous poor multiple-choice tests now abound In the scope of this paper, we will examine some general principles of constructing multiple-choice items and try to find out some common problems in designing multiple-choice questions, then we will give some suggestions for designing multiple-choice questions II THE GENERAL PRINCIPLES OF CONSTRUCTING MULTIPLECHOICE ITEMS A multiple-choice item is composed of two parts: a stem that identifies the question or problem, and a set of alternatives or possible answers that contain a key that is the best answer to the question, and a number of distractors that are plausible but incorrect answers to the question Students respond to multiplechoice items by indicating the alternative that they believe best answers or completes the stem The task of a distractor is to distract the majority of poor students (i.e those who not know the answer) from the correct option According to J B Heaton (1988) the general principles should be observed when multiple-choice items are constructed: Each multiple choice item should have only one answer This answer must be absolutely correct unless the instruction specifies choosing the best option (as in some vocabulary tests) Although it may seem an easy matter, it is sometimes extremely difficult to construct an item having only one correct answer Only one feature at a time should be tested: it is usually less confusing for the testees and it helps to reinforce a particular teaching point Each option should be grammatical correct when placed in the stem, except of course in the case of specific grammar test items For example, stems ending with the determiner a, followed by options in the form of nouns or noun phrases, sometimes traps the unwary test constructor All multiple-choice items should be at a level appropriate to the proficiency level of the testees The context, itself, should be a lower level than the actual problem which the item is testing: a grammar test item should not contain other grammatical features as difficult as the area being tested, and a vocabulary item should not contain more difficult semantic features in the stem than the area being tested Multiple-choice items should be as brief and as clear as possible (though it is desirable to provide short contexts for grammar items) In many tests, items are arranged in rough order of increasing difficulty It is generally considered important to have one or two simple items to ‘lead in’ the testees, especially if they are not too familiar with the kind of test being administered Nevertheless, areas of language which are trivial and not worth testing should be excluded from the test III SOME COMMON PROBLEMS IN DESIGNING MULTIPLECHOICE QUESTIONS (MCQs) Common problems in designing the stem a There appears to have grammatical clues in the options e.g capital letters, a or an, etc Example: He was badly injured in an yesterday A traffic jam B accident C crash D collision The article an in the above question helps the testee easily know the key (B) b The stem does not have sufficient information and it contains words or phrases which repeated in each option Example: Choose the best option A, B, C or D to complete the following sentence by circling the letter you choose He is A the man whom I talk about B the man which I talk about C the man that I talk about D the man that I talk c The stem is not clear and this makes the testees confused Example: What is not mentioned in the text as one of her excuses? NOT must be written in capital letter so that the testee can pay attention to the requirement of question d The stem contains unnecessary information Example: All of the following sentences, which contain relative clauses, are correct EXCEPT for A The suitcase which my father bought was upstairs B The suitcase my father bought was upstairs C The suitcase bought by my father was upstairs D The suitcase buying my father was upstairs “which contain relative clauses” is an unnecessary information, because the testees can understand the stem without it In short, the primary purpose of the stem is to present the problem clearly and concisely The testee should be able to obtain from the stem a very general idea of the problem and the answer required At the same time the stem should not contain extraneous information or irrelevant clues, thereby confusing the problem being tested Unless students understand the problem being tested, there is no way of knowing whether or not they could have handled the problem correctly Although the stem should be short, it should convey enough information to indicate the basis on which the correct option should be selected Common problems in designing the lead-in The lead-in does not present the problem clearly, sufficiently and concisely Example: Choose the best answer Choose the letter of the best answer Choose the correct answer Circle the best answer These lead-in parts not provide the criteria for the testees to answer the questions Let’s consider the following lead-in from the TOEFL sample test: Questions 1-15 are incomplete sentences Beneath each sentence you will see four words or phrases, marked A, B, C, and D Choose the one word or phrase that best completes the sentence Then, on your answer sheet, find the number of the question and fill in the space that corresponds to the letter of the answer you have chosen Fill in the space so that the letter inside the oval cannot be seen This lead-in part is long but sufficient The testees will know how to the test The lead-in is not necessarily long as in the example above but the sufficient criteria are prior Common problems in designing the options (key and distractors) 3.1 The item has more than one correct option (double key) Example: Choose from the prepositions marked A, B, C and D the correct one to fill in the blank by circling the corresponding letter A, B, C or D weekends, John often goes fishing with his friend George A on B in C at D by 3.2 The item has no correct option (no key) Example: Choose one word whose underlined part is pronounced differently from the others by circling the corresponding letter A, B, C or D A celebration B certain C century D excellent 3.3 The distractors or incorrect options are reasonably unattractive and implausible Example: He is a very honest man; he never lies A speaks B say C tells D utters Choice B in the above item is much below the level being tested and will be eliminated by testees immediately: their chances of selecting the correct option will then be one in three 3.4 The options are unparallel Example: Every atlas has its on legend The word legend in this sentence probably means: A mythical story B famous person C A part that explains meanings of symbols D special color Choice C attracts the testees’ attention because it is longer and its structure is completely different from the others Thus, the testees tend to choose this option for their answer (in this case, it is the choice C that is the correct answer) IV CONCLUSION - SUGGESTIONS The following suggestions for designing MCQs are organized into three sections: 1) General Strategies, 2) Designing Stems, and 3) Designing Alternatives General Strategies - Write questions throughout the term Multiple-choice question exams are challenging and time-consuming to create The teacher will find it easier if he/she writes a few questions each week, perhaps after a lecture when the course material is still fresh in his/her mind - Instruct students to select the “best answer” rather than the “correct answer” By doing this, the teacher acknowledges the fact that the distractors may have an element of truth to them and discourage arguments from students who may argue that their answer is correct as well - Use familiar language The question should use the same terminology that was used in the course Avoid using unfamiliar expressions or foreign language terms, unless measuring knowledge of such language is one of the goals of the question Students are likely to dismiss distractors with unfamiliar terms as incorrect - Avoid giving verbal association clues from the stem in the key If the key uses words that are very similar to words found in the stem, students are more likely to pick it as the correct answer - Avoid trick questions Questions should be designed so that students who know the material can find the correct answer Questions designed to lead students to an incorrect answer, through misleading phrasing or by emphasizing an otherwise unimportant detail of the solution, violate this principle - Avoid negative wording Students often fail to observe negative wording and it can confuse them As a result, students who are familiar with the material often make mistakes on negatively worded questions In general, avoid having any negatives in the stem or the options In the rare cases where the teacher uses negatives be sure to emphasize the key words by putting them in upper case and bolding or underlining them Designing Stems - Express the full problem in the stem When creating the item, the teacher wonders if the students would be able to answer the question without looking at the options This makes the purpose of the question clear - Put all relevant material in the stem Do not repeat in each of the alternatives information that can be included in the stem This makes options easier to read and understand, and makes it easier for students to answer the question quickly - Eliminate excessive wording and irrelevant information from the stem Irrelevant information in the stem confuses students and leads them to waste time Designing Alternatives - Limit the number of alternatives Use between three and five alternatives per question Research shows that three-choice items are about as effective as four or five-choice items, mainly because it is difficult to come up with plausible distractors - Make sure there is only one best answer Avoid having two or more options that are correct, but where one is “more” correct than the others The distractors should be incorrect answers to the question posed in the stem - Make the distractors appealing and plausible If the distractors are farfetched, students will too easily locate the correct answer, even if they have little knowledge When testing for recognition of key terms and ideas keep the distractors similar in length and type of language as the correct solution When testing conceptual understanding, distractors should represent common mistakes made by students It is unlikely that students would choose options a, b, c or d, even if they didn’t know the answer A better question would have plausible links between the stem and the distractors: - Make the choices grammatically consistent with the stem Read the stem and each of the choices aloud to make sure that they are grammatically correct - Place the choices in some meaningful order When possible, place the choices in numerical, chronological or conceptual order A better structured question is easier to read and respond to - Randomly distribute the correct response The exam should have roughly the same number of correct answers that are a's, b's, c's, and d's (assuming there are four choices per question) - Avoid using “all of the above” If “all of the above” is an option and students know two of the options are correct, the answer must be “all of the above” If they know one is incorrect, the answer must not be “all of the above” A student may also read the first option, determine that it is correct, and be misled into choosing it without reading all of the options - Avoid using “none of the above” The option “none of the above” does not test whether the student knows the correct answer, but only that he/she knows the distractors aren’t correct - Refrain from using words such as always, never, all, or none Most students know that few things are universally true or false, so distractors with these words in them can often be easily dismissed - Avoid overlapping choices Make the alternatives mutually exclusive It should never be the case that if one of the distractors is true, another distractor must be true as well - Avoid questions of the form “Which of the following statements is correct?” There is no clear question being asked, and the choices are often heterogeneous Such questions are better presented in the form of True/False questions TỔ NGOẠI NGỮ THPT CHUYÊN NGUYỄN BỈNH KHIÊM QUẢNG NAM REFERENCES Do, T M (2005) Designing an Objective English Test at UpperSecondary Schools, VNU, Journal of Science, Foreign Language, T.XXI, No 4AP ELIN 823 Module (2003) Assessment in the Language Curriculum Victoria, University of Wellington Harris, M and McCann, P (1994) Assessment Macmillan Publishers Limited Heaton J B (1988) Writing English Language Test Longman Group UK Limited Group Nguyen, Q T (2005) Chuẩn đánh giá dạy học ngoại ngữ, cited in Kỷ yếu Hội thảo Quốc gia “Chuẩn đánh giá công cụ đo đào tạo cử nhân ngoại ngữ - Lý luận thực tiễn” Danang, Vietnam 6.http://www.adm.uwaterloo.ca/infotrac/tips/designingmultiplechoiceques tions.pdf http://www.lts.rmit.edu.au/renewal/assess/faq_mcq.doc http://www.uwe.ac.uk/elearning/mcq.pdf http://edu.georgianc.on.ca/teaching/pub/pedagoggles/MCQs_2_4.pdf 10 ... without looking at the options This makes the purpose of the question clear - Put all relevant material in the stem Do not repeat in each of the alternatives information that can be included in the... to read and understand, and makes it easier for students to answer the question quickly - Eliminate excessive wording and irrelevant information from the stem Irrelevant information in the stem...I INTRODUCTION he classroom test is one of the most important aspects of the teaching learning process, and designing the classroom test is one of the most demanding responsibilities facing school