VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POSTGRADUATE STUDIES NGUYỄN THỊ OANH A STUDY ON THE CONSTRUCTION OF A MARKING SCHEME FOR END-OF-SEMESTER ENGLISH ORAL TESTS FOR 10th GRADE STUDENTS AT CAM GIANG HIGH SCHOOL IN HAI DUONG Nghiên cứu xây dựng bảng đánh giá cho kiểm tra nói cuối kỳ môn Tiếng Anh cho học sinh lớp 10 trường THPT Cẩm Giàng, Hải Dương M.A MINOR PROGRAMME THESIS Field: English Teaching Methodology Code: 60140111 Hanoi – 2017 VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POSTGRADUATE STUDIES NGUYỄN THỊ OANH A STUDY ON THE CONSTRUCTION OF A MARKING SCHEME FOR END-OF-SEMESTER ENGLISH ORAL TESTS FOR 10TH GRADE STUDENTS AT CAM GIANG HIGH SCHOOL IN HAI DUONG Nghiên cứu xây dựng bảng đánh giá cho kiểm tra nói cuối kỳ môn Tiếng Anh cho học sinh lớp 10 trường THPT Cẩm Giàng, Hải Dương M.A MINOR PROGRAMME THESIS Field: English Teaching Methodology Code: 60140111 Supervisor: Nguyễn Thị Ngọc Quỳnh, Ph.D Hanoi - 2017 DECLARATION I hereby state that I, Nguyen Thi Oanh, declare the thesis entitled “A study on the construction of a marking scheme for the end-of-semester English oral tests for 10th grade students at Cam Giang High School in Hai Duong” is my own research for the Degree of Master of Arts at the Faculty of Post-Graduate Studies, University of Languages and International Studies- Vietnam National University, Hanoi This thesis is the result of my own research and efforts and it has not been submitted for any degree at any other university or institution Hanoi, 2017 Nguyen Thi Oanh i ACKNOWLEDGEMENTS First and foremost, I owe my deepest gratitude to my supervisor, Dr Nguyen Thi Ngoc Quynh for her encouragement, support, and guidance and for giving me constructive feedback throughout this thesis She has guided me in searching for relevant theory to my thesis and has also assisted in collecting data Consequently, I have learnt a lot about the assessment of English oral tests and construction of a marking scheme for oral tests Secondly, I would like to thank Ms Bui Thien Sao, an expert of the Center for Language Testing and Assessment of the University of Languages and International Studies for her invaluable assistance during the research time Thirdly, this thesis would not have been possible without the enthusiastic participation of six English teachers and 150 students at Cam Giang High School where the research was carried out Lastly, I would like to thank my family whose love and support help me complete this thesis ii ABSTRACT This thesis was conducted at Cam Giang High School The participants consisted of 150 tenth-grade students and six English teachers at Cam Giang High School Two experts in the Center for Language Testing and Assessment of the University of Languages and International Studies also participated in this study The research aims at constructing a marking scheme for the end-of-semester English oral test of tenth-grade students Constructing a marking scheme for oral tests is a complex process In this paper, a combination of three methods: intuitive, qualitative and quantitative are employed by the researcher The research started with writing the draft of marking scheme Then, the researcher obtained the judgments from experts and other teachers Next, the marking scheme was piloted with 150 tenth-grade students Based on analyzing the students‟ scores of the oral tests, the researcher examined how well the marking scheme works The findings revealed that the marking scheme can be used by the teachers effectively although there is still a need for further investigation to improve and validate the current marking scheme The majority of the teachers took a positive attitude toward the marking scheme They believed in the efficacy of marking scheme in spite of some difficulties at the beginning and recommended the continuation of using the marking scheme for the next school year Pedagogical implications and suggestions for future studies are drawn out based on the research findings iii TABLE OF CONTENTS DECLARATION i ACKNOWLEDGEMENTS ii ABSTRACT iii TABLE OF CONTENTS iv LIST OF ABBREVIATIONS vi LIST OF TABLES vii PART A: INTRODUCTION .1 Rationale Aims and objectives of the study Research question Scope of the study Significance of the study Method of the study Design of the study PART B: DEVELOPMENT CHAPTER 1: LITERATURE REVIEW 1.1 Communicative competence 1.1.1 Communicative competence in the CEFR 1.2 What is speaking? 1.2.1 Assessing speaking 10 1.3 Marking scheme 10 1.3.1 What is a marking scheme? 10 1.3.2 Approach to construct a marking scheme 12 1.3.3 Steps to construct a marking scheme .13 1.3.4 Types of marking schemes 15 1.3.5 Structure of a marking scheme 17 1.3.6 Available speaking marking schemes .18 1.3.7 Previous studies 20 iv CHAPTER 2: METHODOLOGY 22 2.1 Setting of the study 22 2.2 Participants 23 2.3 Description of the end-of- semester oral test 24 2.4 Research design 24 2.4.1 Rationale for using a multiple-method approach .24 2.4.2 Research procedure 25 2.5 Data collection instruments 28 2.5.1 The interview with the teachers 28 2.5.2 Sample oral test 29 2.5.3 Data collection procedure 30 2.6 Data analysis method 30 2.6.1 Descriptive technique .30 2.6.2 Statistical technique 30 2.6.3 Data analysis procedure 31 CHAPTER 3: RESULTS AND DISCUSSION 33 3.1 The finding from interviews with teachers and experts 33 3.2 The finding from scores of students‟ oral tests 37 PART C: CONCLUSION 43 Summary of the study 43 Pedagogical implications 44 Limitation 45 Suggestions for further studies 45 REFERENCES 46 APPENDIX I APPENDIX XII APPENDIX XIII APPENDIX 4A XVI APPENDIX 4B XIX APPENDIX XXV APPENDIX 6: XXVII v LIST OF ABBREVIATIONS CEFR: Common European Framework of Reference MOET: Ministry of Education and Training vi LIST OF TABLES Table 1: Pearson Correlation on Grammar 38 Table 2: Pearson Correlation on vocabulary 39 Table 3: Pearson Correlation on pronunciation 39 Table 4: Pearson Correlation on fluency and coherence 40 Table 5: Pearson Correlation on Sum 40 vii PART A: INTRODUCTION This part is offered to introduce the rationale of the study, the problem to be addressed in the study, the aims and objectives of the study, and the research questions to be answered It will also present the scope of the study, significance of the study, an overview of the employed methods and the design of the study Rationale English is the most popular official language in the world and the primary language of global trade and commerce It is an international means of interaction and communication in almost all countries Proficiency in English is seen as a desirable goal for a lot of people in the world In many countries including Vietnam, English is taught as a compulsory subject at school and it is included in many exams Of all four skills: reading, speaking, listening and writing; speaking is generally thought to be the most important due to the fact that a lot of learners have spent years studying English but they still cannot speak it fluently In order to speak a foreign language, the learners must master the sound system of the language, use appropriate vocabulary and be able to put words together intelligibly with minimal hesitation Moreover, they also need to understand what is being said to them and respond appropriately to maintain good–natured relation to achieve communication goal (Luoma, 2004) Although students‟ speaking skills are often practiced and developed, it is not extensively assessed Comparing to other skills, speaking is the most difficult language skill to assess the reliability The student‟ speaking ability is usually judged during a face-to-face interaction, in real time between the teacher and the student Besides, the factors such as the nature of the interaction, the kind of tasks, the questions asked, the topic raised and the opportunity given students to speak in English will all have an impact on the student‟s performance (Luoma, 2004) On assessing speaking skills, the teacher has to take on the role as an interviewer and assessor at once, which puts him or her under a lot of pressure This makes teachers hesitant to assess speaking and focus on assessing other skills instead (Rychtarik, 2014) However, “if you want to encourage oral ability, then test learnt utterance with with frequent handle short starts and much pausing to search exchanges, despite reformulation pausing for expressions, to very articulate noticeable very obvious less hesitation, familiar words and frequent to are false repair starts communication -Can’t link words -Link or groups words of groups of words words with simple frequently words with very with basic connectors connectors like “and” or -Link groups of -Can use the most basic connectors like “and”, or “and” or “then” “then” like occurring “but”” connectors to link and “because” simple sentences in order to tell a story or something describe as simple list of points XVIII a APPENDIX 4B VERSION OF THE MARKING SCHEME Level performance of Poor Bad Average Good Excellent ( 0pt) (1pt) (2pts) (3pts) (4pts) Criteria Grammar use -Attempt to use a -Use -Student is not -Can’t present or he/she completed doesn‟t few answer sentences; any questions only simple grammatical structures, simple sentence phases in a learnt but repertoire a -Use correctly a learnt few learnt simple use a few words structures and correctly but and sometimes makes patterns mistakes learnt simple structures Student frequently Nevertheless, makes mistakes large number of may this sometimes make doesn’t affect the mistakes but the intelligibility mistakes are not systematic Vocabulary -Student is not -Only produces a -Use learnt simple -Use learnt simple -Use appropriate present or he/she few doesn‟t isolated words answer words any questions phrases and vocabulary and phrases of certain convey in a topics XIX and to vocabulary to talk personal about the information and situations familiar and and perform learnt repertoire words but the basic topics isolated phrases are linked communicative words and phrases together to create needs are not linked complete meaning together to create complete meaning Pronunciation -Student is not -Pronunciation is -Pronunciation present or he/she often not doesn‟t accurate answer unintelligible any questions clear is - Pronunciation is - Pronunciation is and partly intelligible -Can generally articulate intelligible -Can articulate a -The word stress is simple words and - Can articulate very limited not clear phrases, however, correctly of -The intonation is mispronunciations repertoire learnt words and not correct (raise are frequent simple with words phrases phrases and lower pitch at -The word stress is sometimes limited the accuracy end and sentence mispronounce correct) most of the words XX of are the not clear simple and though make some errors with not -The intonation is individual sounds not correct - Have some difficulty with difficult words -Sometimes make mistakes about word stress but it doesn‟t affect communication -The intonation is relatively correct Fluency coherence and -Student is not -Don’t -Pause present or he/she understand doesn‟t to before most words pause at times answer answer any questions lengthily -Speak slowly and -Speak slowly but the and relatively fluently, frequently know how to keep teacher’s unable to convey speech going and questions basic message self –correct the -Little errors communication possible, only use isolated words and phrases XXI -Can’t use connectors basic -Sometime like basic “and” or “then” use -Use simple -Can use the most connectors connectors to link frequently to link coordinated coordinated sentences “and” occurring like sentences or talk “and”, like connectors to link opposite simple sentences about time process sentences like “but” in order to tell a like “then” and cause and effect story or describe like “because” something (If the student’s answers are not correct or not relate to the topic, subtract from to in each criterion The teacher can give odd marks 0.5 or 1.5 The student’ score = (Grammar x 2+ Vocabulary x 3+ Pronunciation x + Fluency and Coherence x 3) / 20 XXII Mức độ Kém (1điểm) Trung bình (2điểm) Khá (3 điểm) Giỏi (4 điểm) Học sinh không tới dự thi không trả lời câu hỏi thi Học sinh không sử dụng câu hoàn chỉnh, trả lời dạng từ/ cụm từ đơn lẻ học Học sinh sử dụng số cấu trúc ngữ pháp mẫu câu đơn giản vốn ngữ pháp học thường xuyên mắc lỗi Từ vựng Học sinh không tới dự thi không trả lời câu hỏi thi Học sinh không tới dự thi không trả lời câu hỏi thi Sử dụng từ, cụm từ đơn giản thuộc chủ đề cụ thể từ, cụm từ có tính liên kết để tạo thành ý Âm phát không rõ ràng, không xác Trọng âm từ không rõ rệt Ngữ điệu không phù hợp ( lên xuống giọng cuối câu không hợp Học sinh sử dụng xác số cấu trúc đơn giản mắc số lỗi không mắc lỗi cách hệ thống Sử dụng từ, cụm từ để nói tình chủ đề quen thuộc Phát âm Chỉ sử dụng từ, cụm từ đơn lẻ học từ cụm từ tính liên kết để tạo thành ý thống Phát âm thường không hiểu Chỉ phát âm số lượng nhỏ từ, cụm từ học với độ xác thấp phát âm sai phần lớn từ Học sinh sử dụng xác số cấu trúc đơn giản mắc lỗi Tuy nhiên việc mắc lỗi không ảnh hưởng đến hiểu Sử dụng từ cụm từ đơn giản học để truyền đạt thông tin cá nhân thực nhu cầu giao tiếp Phát âm gần hiểu Phát âm từ, nhóm từ đơn giản nhien thường xuyên phát âm sai Trọng âm từ Tiêu chí Ngữ pháp Yếu (0 điểm) XXIII Âm phát hiểu Phát âm rõ ràng, tương đối xác từ, nhóm từ đơn giản phát âm sai lý) Độ lưu loát Học sinh kết nối ý không tới dự thi không trả lời câu hỏi thi -Học sinh không hiểu để trả lời câu hỏi giáo viên -Ít có khả giao tiếp, sử dụng số từ cụm từ đơn lẻ -Không thể sử dụng từ nối đơn giản “ and” “ then” -Ngập ngừng lâu trước hầu hết từ thường xuyên khó khăn để diễn đạt ý -Thi thoảng sử dụng từ nối đơn giản để nối ý đẳng lập “and” tiến trình thời gian “then” XXIV không rõ số ràng âm Ngữ điệu Gặp khó không phù khăn hợp phát âm từ khó Đôi mắc lỗi trọng âm không ảnh hưởng đến giao tiếp -Ngữ điệu tương đối phù hợp - Phản ứng -Nói chưa nhanh câu đơn giản nói tương đối ngập trôi chảy, ngừng nhiều biết cách lần trì nói tự sửa lỗi -Sử dụng từ nối đơn giản để nối ý đẳng lập “and” ý đối lập “ but” , nguyên nhân kết “ because” -Có thể sử dụng từ nối hay dùng để liên kết câu đơn giản để kể câu chuyện hay miêu tả vật APPENDIX Interview questions Section 1: Criteria Question 1: Do the performance criteria match with the objective of the course and students‟ proficiency? Question 2: Are the number of criteria enough and suitable to assess the students‟ oral tests? Question 3: Are the performance criteria clear and easy to score? Question 4: Do you want to add any criteria other than those in the marking scheme? Section 2: Levels of performance Question 5: Is each level on the marking scheme objective? Question 6: Does the highest level represent exemplary performance as required to the tenth-grade students? Section 3: Descriptors Question 7: Are the descriptors clear and understandable? Question 8: Do the descriptors explain clearly each level of performance for each criterion and describe in detail the characteristic of levels of performance? Question 9: Is there a distinction among the descriptors and is there a progress in higher levels? Section 4: General look Question 10: Does the marking scheme have these elements: criteria, levels of performance, descriptors and score? Question 11: Is the marking scheme manageable and practical to use by the teachers at school? Question 12: Do you have any further general comments or proposal to the completion of the marking scheme? XXV Câu hỏi vấn Tiêu chí Các tiêu chí có phù hợp với mục tiêu chương trình học lực học sinh không? Số lượng tiêu chí có đủ phù hợp để đánh giá nói học sinh không? Các tiêu chí có rõ ràng chấm không? Thầy cô có bổ sung thêm tiêu chí khác không? Mức độ đánh giá làm học sinh Mỗi mức độ bảng đánh giá có khách quan không? Mức độ cao hoàn thành thi tốt theo yêu cầu chương trình học học sinh lớp 10 không? Mô tả Phần mô tả cho mức điểm tiêu chí có rõ ràng dễ hiểu không? Mô tả có giải thích rõ cấp độ thực cho tiêu chí mô tả chi tiết đặc điểm mức độ học sinh hoàn thành thi? Mô tả phân biệt cấp độ học sinh thể tốt mức điểm cao không? Nhận xét chung 10 Nhìn chung, bảng đánh giá có đủ yếu tố bảng đánh giá theo kiểu phân tích: tiêu chí, mức độ hoàn thành nhiệm vụ, điểm số mô tả chưa? 11 Bảng đánh giá sử dụng thực tế cho giáo viên học sinh trường không? 12 Thầy cô có đề xuất việc hoàn thiện bảng đánh giá không? XXVI APPENDIX 6: Translation of Transcriptions of the interviews Expert 1: Q1 Yes The criteria match the objective of the course and students‟ proficiency Q2 Yes The number of criteria are enough and suitable to assess the student‟s oral tests Q3 Four criteria: Grammar, Vocabulary, Pronunciation, Fluency and Coherence are clear and easy to score They didn‟t make examiners misunderstand among criteria Q4 No I don‟t want to add any criteria other than those in the marking scheme Q5 Yes Each level on the marking scheme is objective and there is clear difference among levels Q6 Yes The highest level represents exemplary performance as required to the tenth-grade students The students after finishing grade ten are able to use the simple learnt vocabulary and grammar structures to talk about familiar topic and situation The student may make some errors with pronunciation but they didn‟t affect the communication Q7 There are some confusing words and the words used are academic It is better to use simple words and easy to understand for school teachers Q8 The descriptors should be explained clearer in each level of performance and different characteristic of each level of performance should be highlighted to make easier for examiners Q9 There was a distinction between level and in spite of no distinction between two levels and in Grammar, Vocabulary and Pronunciation For example, in Grammar criterion the descriptors for level and are the same “a few” similar to “some” There was no difference and better at level In Vocabulary, “basic communication needs” was the same as “routine everyday transactions” In pronunciation, both level and level used “mostly intelligible”, they didn‟t have difference between these two levels Q10 Yes The marking scheme had enough four elements of an analytic marking scheme including criteria, levels of performance, descriptors and scores XXVII Q11 Yes Q12 No Expert 2: Q1 Yes The criteria of this marking scheme met the objective of the course and students‟ proficiency Q2 Yes The number of criteria are enough and suitable to assess the student‟s oral tests Q3 Fluency and Coherence Criteria sometime make it difficult for examiners to score because many students pause very long but they still use the connectors While many students speak fluency but they use a few connectors and used simple sentences Therefore, it should be made clearer about assessing when combining two criteria Q4 No Q5 Yes Each level on the marking scheme is objective and there is clear difference among levels Q6 Yes Q7 There are some confusing words How to differentiate “sufficient vocabulary” it should be cleared and have a concrete reference that all teacher familiar with” Q8 In term of pronunciation skill, the intonation and the word stress should be added Moreover, in case the students pronounce the simple words correctly but mispronounced difficult words, which mark the student will get May be it should be clearer in the descriptors Q9 There is no distinction between two levels and in Grammar, Vocabulary and Pronunciation Q10 Yes Q11 Yes Q12 No Teacher 1: Q1 Yes The criteria matched the objective of the course and students proficiency XXVIII Q2 Yes The number of criteria are enough and suitable to assess the student‟s oral tests Two main criteria including accuracy and fluency were included in the marking scheme Q3 All the criteria are easy to score and clear Q4 No I don‟t want to add any criteria other than those in the marking scheme Q5.Yes All the teachers are familiar with assessing students at levels poor, bad, average, good and excellent The marking scheme can be used by any teachers to assess any tenth-grade student Q6 Yes Q7 The words are confusing and academic For example, “a limited control”, “a limited repertoire of learnt words” and “sufficient” Q8 Yes Q9 There is no distinction between two levels and in Grammar, Vocabulary and Pronunciation Q10 Yes Q11 Yes The marking scheme was manageable and practical to use for assessing the end-of-semester oral test of tenth grade students Q12 No Teacher 2: Q1 Yes The criteria match the objective of the course and students‟ proficiency The English text book is divided into five parts and the speaking part requires students to use the learnt vocabulary and grammar structures in each unit Q2 Yes The number of criteria are enough and suitable to assess the student‟s oral tests Q3 All the criteria are easy to score and clear Q4 Yes I want to add “Topic development” criterion Q5 Yes Each level on the marking scheme is objective and there is clear difference among levels Q6 Yes However, a few excellent students at school can perform better levels in XXIX the marking scheme and beyond these levels Q7 It is quite difficult for examiners to understand Q8 The descriptors explained each level of performance for each criterion and described in detail the characteristic of levels of performance Q9 Two levels and in Grammar are the same There is no difference Q10 Yes Q11 The marking scheme with four clear criteria and levels of performance from poor to excellent can help classify and assess students‟ proficiency Q12 No Teacher 3: Q1 Yes The criteria match the objective of the course and students proficiency Q2 Yes The number of criteria are enough and suitable to assess the student‟s oral tests Q3 All the criteria are easy to score and clear Q4 No Q5 Yes Each level on the marking scheme was objective and there was clear difference among levels Q6 Yes Q7 The marking scheme is quite difficult to understand because there are some confusing words Q8 Yes Q9 There is no distinction between two levels and in Grammar, Vocabulary and Pronunciation Q10 Yes Q11 Yes Q12 No Teacher 4: Q1 Yes The criteria match the objective of the course and students proficiency Q2 Yes The number of criteria are enough and suitable to assess the student‟s oral XXX tests Two main criteria including accuracy and fluency were included in the marking scheme Q3 All the criteria are easy to score and clear They don‟t make the examiner misunderstanding among criteria Q4 I want to add “content” criterion Q5 Yes Each level on the marking scheme is objective and there is clear difference among levels Q6 Yes Q7 No Q8 Yes Q9 Two levels and in Grammar are the same There is no difference Q10 Yes Q11 Yes Q12 No Teacher 5: Q1 Yes The criteria of this marking scheme meet the objective of the course and students‟ proficiency Q2 Yes The number of criteria are enough and suitable to assess the student‟s oral tests Q3 All the criteria are easy to score and clear Q4 No Q5 Yes Q6 Yes Each level on the marking scheme is objective and there is clear difference among levels The highest level in the marking scheme presents the best performance in levels but it still allows students to make mistakes but the mistakes are unsystematic Q7 The descriptors are not clear and understandable Q8 Yes Q9 Two levels and in Grammar are the same There is no difference XXXI Q10 Yes Q11 Yes Q12 No Teacher 6: Q1 Yes The criteria of this marking scheme meet the objective of the course and students‟ proficiency Q2 Yes The number of criteria are enough and suitable to assess the student‟s oral tests Q3 All the criteria are easy to score and clear Q4 No Q5 Yes Each level on the marking scheme is objective and there is clear difference among levels Q6 Yes Q7 The words used should be simpler and easier to understand Q8 Yes Q9 There was no difference and better at level in Vocabulary and Grammar Q10 Yes Q11 Yes Q12 No XXXII ... thesis entitled A study on the construction of a marking scheme for the end -of- semester English oral tests for 10th grade students at Cam Giang High School in Hai Duong is my own research for. .. aim of the study to construct a marking scheme for end -of- semester English oral tests for 10th grade students at Cam Giang High School in Hai Duong, the study is designed to use a combination of. .. speaking, assessing speaking, definitions of marking scheme, approaches to construct a marking scheme, types of marking scheme and the structure of a marking scheme A review of the available marking