Washback effects of highstakes English tests on Vietnamese EFL high school teachers’ teaching

Washback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teachingWashback effects of highstakes English tests on Vietnamese EFL high school teachers’ teaching

HUE, 2024

MINISTRY OF EDUCATION AND TRAINING

HUE UNIVERSITY

UNIVERSITY OF FOREIGN LANGUAGES AND INTERNATIONAL STUDIES

TONG THI LAN CHI

WASHBACK EFFECTS OF HIGH-STAKES ENGLISH TESTS ONVIETNAMESE EFL HIGH SCHOOL TEACHERS’ TEACHING

DOCTOR OF PHILOSOPHY THESIS IN THEORY AND METHODOLOGYOF ENGLISH LANGUAGE TEACHING

HUE, 2024

MINISTRY OF EDUCATION ANDTRAINING HUE UNIVERSITY

UNIVERSITY OF FOREIGN LANGUAGES AND INTERNATIONAL STUDIES

TONG THI LAN CHI

WASHBACK EFFECTS OF HIGH-STAKES ENGLISH TESTS ONVIETNAMESE EFL HIGH SCHOOL TEACHERS’ TEACHING

DOCTOR OF PHILOSOPHY THESIS IN THEORY AND METHODOLOGYOF ENGLISH LANGUAGE TEACHING

CODE: 9140111

SUPERVISOR: Assoc Prof Dr PHAM THI HONG NHUNG

I certify that the present dissertation submitted entitled “Washback effects ofhigh-stakes English tests on Vietnamese EFL high school teachers’ teaching” for

the Degree of Doctor of Philosophy in theory and methodology of English language teaching, is the result of my own research and that, to the best of my knowledge and belief, contains no material which has been accepted for the award of any other degree in any institute, college, or university, and previously published or written by another person, except where due reference is made in the text of the dissertation.

Tong Thi Lan Chi

This PhD journey has profoundly transformed me Returning to academic study after a decade since earning my MA degree in English linguistics from Hogskolan Dalarna (Sweden) and immersing myself in the realm of English language teaching theory and methodology has been both challenging and immensely rewarding While I relished acquiring valuable knowledge, personal hardships often clouded my path, tempting me to give up on several occasions However, overcoming these struggles has made reaching the completion of this thesis a deeply emotional experience I am immensely grateful to the numerous individuals who have taught me invaluable lessons of resilience and supported me along this journey Though it is impossible to name them all, I extend my heartfelt appreciation to those whose encouragement and support made this journey possible.

First and foremost, I extend my heartfelt appreciation to my supervisor, Associate Professor Doctor Pham Thi Hong Nhung, for her unwavering guidance and invaluable support throughout this challenging journey I am deeply grateful to her for not only sharing her expertise and wisdom but also for her empathy and encouragement Without her guidance, this work would not have reached its final form.

Also, I would like to extend my gratitude to the lecturers and professors at the University of Foreign Languages and International Studies (Hue University) for their patient and supportive guidance throughout my learning journey Their knowledge and insightful comments during lessons and seminars have been invaluable to me.

Special thanks are also due to the high school teachers in Buon Ma Thuot City, whose names I refrain from mentioning here due to ethical considerations Without their willingness and patience, I would not have been able to gather such valuable data for this study.

Finally, I want to express my heartfelt appreciation to my colleagues and friends for their unwavering encouragement in my educational pursuits I am especially grateful to my two compassionate and supportive children, who have been my anchors during some of the stormiest moments of my life.

The study investigates the washback effects of high-stakes English tests, specifically within the framework of the National High School Graduation Exam (NHSGE), on the teaching practices of Vietnamese EFL high school teachers Employing a mixed-method research design grounded in washback theory, the research explores three key dimensions of EFL teaching influenced by these tests: content coverage, instructional methods, and in-class assessment techniques Data collection methods, including questionnaires, semi-structured interviews, and teaching artifacts, provided comprehensive quantitative and qualitative insights from fifty-eight EFL teachers in Buon Ma Thuot city (Dak Lak province, Vietnam) Firstly, the findings reveal significant negative washback effects, indicating that teachers closely align their instructional content and practices with NHSGE specifications This alignment, driven by adherence to program distribution plans and the provision of test-oriented materials, highlights the pervasive influence of high-stakes testing on instructional decision-making Secondly, teachers predominantly employ direct instruction and activities like grammar review and language games, reflecting a tendency to align teaching practices with NHSGE formats Furthermore, assessment tasks are tailored to mirror NHSGE content, emphasizing vocabulary, grammar, and pronunciation skills through limited test types such as multiple-choice quizzes and mini written tests These findings underscore the adverse washback effects of the NHSGE English tests on EFL teaching practices, emphasizing the need for a nuanced approach to curriculum development and pedagogical training to mitigate these impacts The study's insights are crucial for policymakers, curriculum developers, teacher trainers, and test designers, advocating for the implementation of contextually relevant strategies to mitigate the negative impacts of high stakes testing and promote effective EFL teaching practices.

LIST OF ABBREVIATIONS

DOET Department of Education and Training

MOET Ministry of Education and Training NHSGE National high school graduation exam

1.1 Background of the study 1

1.2 Rationale for the study 3

1.3 Research questions 5

1.4 Scope of the study 6

1.5 Significance of the study 7

1.6 Organization of the study 8

CHAPTER 2 LITERATURE REVIEW 10

2.1 Washback effects: definitions, dimensions, factors, and models 10

2.1.1 Definitions of washback effects 10

2.1.2 Washback dimensions 14

2.1.3 Factors generating test washback 17

2.1.4 Washback models 22

2.2 EFL teaching: definition and aspects of teaching 28

2.2.1 Definitions of EFL teaching 28

2.2.2 Common EFL teaching principle, methods, strategies, and aspects of

2.3 Empirical research on washback of high-stakes tests 33

2.3.1 Empirical insights into teacher factors contributing to washback effects 34

2.3.2 Empirical insights into aspects of EFL teaching under the washback ofhigh stakestests37 2.4 Conceptual framework for washback effects of the NHSGE English high-stakes tests on Vietnamese EFL teachers’ teaching 44

2.4.1 Review of the test factors of the NHSGE English tests 44

2.4.2 Conceptual framework for analyzing the washback effects of theNHSGEEnglishtests on teaching

3.4.2 Follow-up semi-structured interviews 68

3.4.3 EFL teaching artifacts 71

3.5 Data gathering and analysis procedures 72

3.5.1 Data gathering procedures 73

3.5.2 Data analysis 74

3.6 Validity and reliability 76

3.7 Ethical concerns 77

3.8 Chapter summary 79

CHAPTER 4 FINDINGS AND DISCUSSION 80

4.1.The teacher factors that are affected by the NHSGE English tests andrelated contextual factors 80

4.1.1 The teachers’ perceptions of contextual factors 81

4.1.2 The teachers’ perception of the NHSGE English test factors 92

4.1.3 The teachers’ perspective of effective teaching and test preparation 95

4.2 Washback effects of the NHSGE English tests on EFL teaching aspects asperceived by the teachers 99

4.2.1 Washback effects on what teachers teach 100

4.2.2 Washback effects on how teachers teach 109

4.2.3 Washback effects on how teachers conduct in-class assessment 115

4.3 Chapter summary 125

CHAPTER 5 CONCLUSION AND IMPLICATIONS 127

5.1 Summary of key findings 127

5.1.1 Teacher factors contributing to the washback effects of the NHGSE

LIST OF AUTHOR’S WORK 136

Appendix 1 The questionnaire 152

Appendix 2 Hoang’s et al (2020) English 12 book map 160

Appendix 3 The official interview protocol 160

Appendix 4 A sample interview transcript 163

Appendix 5 The English 12 Program Distribution Plan Used in School X 168

Appendix 6 A sample lesson plan 170

LIST OF GRIDS

Grid 2.1 Contextual Factors Generating Test Washback (Shih, 2009) 19

Grid 2.2 Hughes’s (1993) Washback Model 24

Grid 2.3 Bailey’s (1996) Washback Model 25

Grid 2.4 Washback Model by Shih (2009) 27

Grid 2.5 Conceptual Framework for Washback Effects of the NHSGE English Tests on EFL Teaching

51Grid 3.1 Research Design of the Current Study 58

Grid 3.2 Stages For Data Gathering and Analysis 72

LIST OF TABLES

Table 2.1 Test Factors That Generate Washback (Shih, 2009) 20

Table 2.2 Teacher Factors That Generate Washback (Shih, 2009) 21

Table 2.3 Alderson and Wall’s (1993) Washback Hypothesis 23

Table 2.4 The Format of the English Tests in the NHSGE from 2015- 2022 45

Table 2.5 Writing Task Sample in the 2015- 2016 and 2017-2022 Versions of the NHSGE

46Table 2.6 Test Tasks in the 2017-2022 NHSGE English Tests (MOET, 2017-2022) 48Table 2.7 The Content of the 2017 NHSGE English Tests (MOET, 2017) 50

Table 3.1 Demographic Information of the Participants (n=58) 61

Table 3.2 Descriptions of the Questionnaire 66

Table 3.3 Predetermined Interview Themes and Tentative Questions 69

Table 4.1 Summary of the Teachers’ Responses on Perception of Contextual Factors 81Table 4.2 Content Prescription for Teaching and Learning English 12 (MOET, 2018) 83

Table 4.3 A Part of the English 12 Program Distribution Plan Used in School X 87

Table 4.4 Summary of the Teachers’ Overall Evaluation of their 12th Graders’

Table 4.6 Summary of Teachers’ Viewpoint on Effective Teaching and Test

95

Table 4.7 Summary of the Participants’ Responses on “What to Teach” 100Table 4.8 Summary of the Teachers’ Responses on “How Teachers Teach” 109

Table 4.9 Summary of Teachers’ Responses on “How Teachers Conduct In-Class

Assessment Tasks”

115

Table 4.10 A Mini Test Used in School Y 121Table 4.11 A Quiz used in School Z 124

CHAPTER 1 INTRODUCTION

This chapter serves as an introduction to the current thesis, providing an overview of the study's background and rationale to elucidate the motivations behind its undertaking It articulates the study's purpose and research questions, offering a thorough overview of the study's scope, contribution, and organizational structure.

1.1 Background of the study

In Vietnam, the educational system operates under a centralized structure, where national-level educational policies are formulated by the Ministry of Education and Training (MOET) The Ministry is tasked with setting standards, compiling, editing, and determining official textbooks used across general education nationwide Moreover, the Ministry exercises supervision on teaching content and methodologies for each subject within the educational program This supervision is facilitated through the issuance of a secondary educational program (referred to as "chương trình giáo dục phổ thông" in Vietnamese) for each subject, with the current program established in 2018 (MOET, 2018) These programs outline educational objectives, content (themes, topics), teaching methods, and assessment criteria to be adhered to by schools nationwide.

Within the general education program, particular emphasis has been placed on foreign languages among the various subjects that students are required to learn Ministry regulations stipulate five foreign languages—English, French, Chinese, Russian, and Japanese—are officially introduced to Vietnamese students, beginning from grade 3 (or grade 1 under specific conditions) Additionally, in 2021, two more foreign languages, Korean and German, were incorporated into the options for students, allowing for a more diverse range of choices This inclusion was experimentally implemented in accordance with Decision no 712/ BGD&ĐT, issued on February 9, 2021, by the Ministry of Education and Training.

Despite the Ministry's endeavors to introduce a variety of foreign languages at the general education level, English remains the predominant foreign language taught and learned in Vietnamese schools Several factors contribute to the widespread interest in English, notably its role as a language for global integration and business transactions However, it is apparent that the English proficiency of Vietnamese individuals, particularly the younger demographic, falls short of the expectations and requirements of the labor market, in contrast to neighboring countries such as Thailand or the Philippines As outlined by Nguyen (2013), the experts involved in Project 2020 identified that 98% of Vietnamese students, having undergone seven years of English education (from grade 6 to grade 12, aged 11–18), were unable to utilize the language for basic communication These findings served as the impetus for the issuance of the Vietnamese Prime Minister’s Decision No 1400/QĐ-TTg, titled "Teaching and Learning Foreign Languages in the National Education System, Period 2008 to 2020" (commonly referred to as the National Foreign Language Project 2020 by the public).

The objectives of the National Foreign Language Project were as follows:

by the year 2015, to achieve significant progress in professional skills and language proficiency among human resources, particularly in prioritized sectors; by 2020, to ensure that the majority of Vietnamese students graduating from secondary, vocational schools, colleges, and universities can confidently use a foreign language in their daily communication, studies, and work within an integrated, multicultural, and multilingual environment This aim is intended to establish foreign language proficiency as a comparative advantage for the development of the Vietnamese people in the pursuit of the country's industrialization and modernization goals.

(Vietnamese Prime Minister’s Office, 2008).

Although the Project did not meet its ambitious goals within the expected period as admitted by 2016 Minister of Education and Training, i.e., Mr Phung Xuan Nha (Nguyen, 2017), it has brought about changes in teaching foreign languages in general and English in particular, to students of all the three levels in general education Significant positive transformations have occurred in the teaching and assessment of English for Vietnamese learners, particularly in secondary education The secondary educational program, guided by Decree no.32/TT-BGD ĐT, underscores the Ministry's emphasis on student-centered teaching methodologies and the cultivation of communicative skills Under this framework, students undergo a series of assessments throughout the academic year, including ongoing, mid-term, and term-end English tests, culminating in a final accumulative mark for the subject Additionally, since 2015, Vietnamese grade 12 students have been mandated to sit for the National high school graduation exam (NHSGE), alongside tests in Literature, Mathematics, Foreign languages and either Social Science or Natural Science Notably, these high-stakes assessments, except for the Literature test, adhere to standardized testing principles, ensuring uniformity in question administration and scoring methodologies (Great School Partnership [GSP], 2015) The implementation of these NHSGE tests has attracted considerable attention from various stakeholders, particularly given that the test results are primarily utilized to make critical decisions regarding students' graduation from secondary education, their admission to universities in Vietnam.

1.2 Rationale for the study

In the domain of language education, testing and assessment play multifaceted roles beyond mere evaluation, shaping teaching methodologies, learning outcomes, decision-making processes, and employment prospects (Bachman and Palmer, 1996; Shohamy, 1998) Particularly, high-stakes tests wield significant influence in educational settings worldwide, including Vietnam, where they are pivotal in shaping critical decisions concerning students and other stakeholders The concept of washback

effects, delineating the influences—be it positive or negative—of tests on various aspects of teaching and learning (Brown, 1997; Cheng, 1997; McKinley & Thompson, 2018; Nguyen, 2017; Pizarro, 2010), has emerged as a central theme in language testing discourse Extensive research worldwide has explored the washback effects of various high-stakes English tests on stakeholders like teachers and students However, there is a notable absence of research focusing on these effects within the Vietnamese context, especially in secondary education While scholars such as Dinh (2020), Nguyen (2017), and Nguyen and Gu (2020) have examined these effects in higher education, there is a scarcity of research in secondary education settings One possible reason for this absence may stem from the centralized supervision of the MOET on teaching and learning for secondary education The Ministry's focus on curriculum implementation and adherence to standardized testing procedures may prioritize administrative concerns over research initiatives Consequently, there may be limited resources and incentives available for educators and researchers to explore the nuances of washback effects in secondary education Additionally, the hierarchical structure of educational governance in Vietnam could contribute to a top-down approach that emphasizes compliance with policies rather than fostering a culture of research and inquiry at the secondary level These factors collectively contribute to the gap in literature and highlight the importance of addressing this research deficit to better understand the implications of high stakes testing in secondary education settings This research gap serves as the impetus for investigating the washback effects of a prominent high-stakes English test, namely the NHSGE English test, on Vietnamese high school teachers' teaching practices.

Moreover, the study aims to identify washback factors to develop a conceptual framework that elucidates the mechanisms or interactions among these factors within Vietnam's centralized educational system By undertaking this investigation, the study not only addresses theoretical gaps in the literature but also offers practical implications by understanding and mitigating potential negative washback effects.

Ultimately, the

findings of this research endeavor are anticipated to provide valuable insights for stakeholders aiming to enhance EFL teaching practices in Vietnamese high schools.

1.3 Research questions

This study aims to examine the washback effects of the English tests administered as part of the NHSGE on the teaching practices of Vietnamese EFL high school teachers Initially, the research aims to identify teacher-related factors as mediators of the NHSGE English test factors and contextual factors After exploring these teacher-related factors, including demographic details, perceptions of contextual elements, attitudes towards the NHSGE English test, and perspectives on effective teaching and test preparation, the study shifts its focus to understanding the tangible effects of test-induced washback across various domains of EFL teaching Specifically, the study seeks to address the following questions:

Question 1: What teacher factors are affected by the NHSGE English tests and related contextual factors?

Question 2: In what ways do the NHSGE English tests influence various aspects of EFL teaching?

The research questions implicitly underscore the research design's focus on identifying teacher factors which are affected by the NHSGE English tests and contextual factors prior to exploring the washback effects on EFL teaching Washback models commonly incorporate three factors context, test, and teacher factors -emphasizing the necessity of first identifying factors for the washback phenomenon to manifest This initial emphasis on teacher factors permits a thorough examination of specific factors unique to the study participants, which may not be discernible from existing literature, and their interaction with other washback components to generate washback effects Such an approach ensures a comprehensive understanding of washback dynamics and its implications for EFL teaching within the research context.

1.4 Scope of the study

The current study delineates its scope based on the tests under examination, the participants involved, and the research objectives Firstly, it narrows its focus to the English tests within the NHSGE framework, specifically excluding other high-stakes English tests These standardized multiple-choice tests are mandatory for Vietnamese grade 12 students, serving as a requirement for secondary education qualification and university admission.

Furthermore, the study is confined to investigating the washback effects of these tests on the EFL teaching practices of 58 Vietnamese high school teachers in Buon Ma Thuot city, Dak Lak province, Vietnam It specifically targets teachers working with grade 12 students, omitting grades 10 and 11, within the high schools they are employed in The investigation centers on three key aspects of EFL teaching influenced by the NHSGE English tests: the content covered by teachers, their instructional methods, and their in-class assessment practices This includes examining variables such as textbook coverage, i.e textbook teaching content, test-aligned content with a view to time allocation, and additional content in supplementary materials Additionally, attention is given to EFL teachers' selection of instructional activities and strategies for content presentation and test preparation, along with their decisions regarding the type, format, and content of in-class assessment methods, which may encompass projects, presentations, and testing conducted during regular class sessions to evaluate students' comprehension of the material covered.

Moreover, the research was conducted in Buon Ma Thuot city, situated in Dak Lak province in the Central Highlands of Vietnam While the Vietnamese education system operates under centralized guidelines set by the Ministry of Education and Training (MOET), the selected research site possesses distinct ethnographic characteristics Consequently, the teaching practices observed among the research

participants may not be reflective of those in other locations with differing ethnographic contexts.

Lastly, the chosen research design employs a mixed-methods approach involving the voluntary participation of fifty-eight EFL teachers from high schools in Buon Ma Thuot city Due to the inherently personalized and contextualized nature of washback effects, the findings of this study may offer insights relevant to EFL teachers in similar ethnographic settings However, caution must be exercised in generalizing these results to all EFL teachers across Vietnam or other educational contexts Instead, they contribute to an understanding of diverse washback mechanisms within specific test-use contexts Essentially, the aim of this study, akin to many washback studies, is not to propose a single washback mechanism universally applicable but to enrich the literature by exploring various washback phenomena across different testing contexts.

1.5 Significance of the study

The study holds both theoretical and practical significance Theoretically, it contributes by reviewing established washback models and constructing a comprehensive framework This framework delineates the factors triggering washback, particularly focusing on the intricate interplay of teacher-related factors within the washback mechanism Moreover, it identifies specific components within each factor group and their impact on various aspects of EFL teaching Consequently, the study enriches existing knowledge and offers insights transferable to similar educational contexts.

Practically, the findings raise EFL teachers' awareness of how high-stakes tests can influence their teaching practices By understanding these effects, teachers can adapt their approaches to maximize student learning outcomes Furthermore, the study's implications are expected to stimulate significant interest and discussion regarding the use of high-stakes English tests in Vietnamese educational settings Insights into the

influences of NHSGE English tests on teaching practices can prompt critical reflections among EFL teachers, fostering improvements in pedagogical strategies.

Additionally, the study results have the potential to initiate dialogue and action among stakeholders They highlight the importance of considering the NHSGE English test design and test use Policy makers, curriculum developers, teacher trainers, test designers and school authorities can utilize these findings to mitigate negative washback effects and promote effective teaching practices.

Moreover, this study serves as a foundation for future research endeavors Subsequent studies could explore washback effects on teachers with diverse ethnographic backgrounds or investigate additional factors contributing to the washback mechanism Additionally, there is an opportunity to examine the broader impact of these tests on various stakeholders beyond teachers, including students, administrators, test designers, and curriculum developers This collective body of research can provide comprehensive insights into the implications of high-stakes testing in Vietnaemse educational contexts.

1.6 Structure of the study

The study comprises five primary chapters, each serving a distinct purpose within the research framework.

Chapter 1 serves as the introductory section, providing an overview of the study's background, objectives, scope, and organizational structure.

Chapter 2 thoroughly explores the extant literature concerning the washback effects of high-stakes tests, providing a critical examination of key concepts, definitions, and interchangeable terms associated with washback effects Moreover, this chapter reviews diverse washback models from prior studies and introduces a conceptual framework customized to the unique context of the current research Additionally, it discusses the definition of EFL teaching, different teaching approaches, activities, and

various instructional domains Furthermore, this chapter presents findings from previous empirical research on the washback effects of tests across various educational settings, culminating in the proposal of a conceptual framework specifically designed for analyzing the NHSGE English tests.

Chapter 3 delineates the research methodology employed to investigate the two research questions It provides comprehensive details regarding the research participants, the research setting, data collection instruments, research approach, and study design Furthermore, this chapter examines aspects of validity, reliability, and ethical considerations to ensure the rigor and integrity of the study.

In Chapter 4, the findings of the study are comprehensively reported and interpreted based on the data collected Findings are organized into two main themes corresponding to the research questions, allowing for a thorough analysis and discussion of the results.

Finally, Chapter 5 offers a synthesis of the entire study, summarizing key findings and their implications Additionally, this chapter provides recommendations for utilizing the research results in practice and suggests avenues for future research endeavors, contributing to ongoing scholarly discourse in the field.

CHAPTER 2 LITERATURE REVIEW

This chapter adopts a dual approach, synthesizing both theoretical and integrative literature (Dudovskiy, 2022), to establish a comprehensive foundation for the current research Comprising four main sections, the chapter commences by an in-depth examination of test washback effects, including definitions, dimensions, factors, and models within diverse research contexts Following this, the exploration extends to defining EFL teaching and aspects of EFL teaching, preceding The third section presents a review of empirical research on the washback effects of high-stakes tests on EFL In conclusion, the fourth part introduces a conceptual framework tailored for the present study, aimed at understanding the washback effects of NHSGE English high-stakes tests on teaching.

2.1 Washback effects: definitions, dimensions, factors, and models

This section explores washback effects, encompassing definitions, dimensions, and factors by different researchers The study then explores various test washback models derived from prior research, laying the groundwork for the conceptual framework in the current study.

2.1.1 Definition of washback effects

In this part, various terms used interchangeably with "washback effects," such as impact, washback, backwash, and consequences or consequential validity are introduced Before delving into these terms, a brief explanation is provided regarding different test qualities, including impact Given the pivotal role of tests in numerous educational contexts, there has been a significant focus on quality control in the design and development of tests to ensure their usefulness According to Bachman and Palmer (1996), test usefulness is defined as a combination of six distinct test qualities: reliability, construct validity, authenticity, interactiveness, impact, and practicality, all intricately interconnected.

These six qualities collectively contribute to test usefulness and should not be assessed independently First, construct validity, as elucidated by Bachman and Palmer (1996), pertains to the appropriateness of test score interpretations—the extent to which interpretations reflect the measured abilities or constructs Justifications for score interpretation involve evidence, including content relevance, coverage, relatedness of concurrent criterion, and predictive utility In language learning, Coombe et al (2017) describe construct validity as the alignment between language learning theories, methodology, and assessment types An example is the use of communicative language tests when employing a communicative language learning approach Second, reliability, defined as "consistency of measurement" (Bachman & Palmer, 1996, p.19) or consistency of test scores (Coombe et al., 2017), represents the similarity of results obtained from a test administered to the same group of test takers across different times and settings Reliability can be influenced by test factors, administrative factors, and affective factors such as test anxiety and learning styles Together with construct validity, reliability justifies the use of test scores for inferences or decisions Third, authenticity, according to Bachman and Palmer (1996), is "the degree of correspondence of the characteristics of a given language test task to the features of a TLU task" (p.23) Authenticity connects test performance to target language use tasks, influencing generalization and test takers' perception of task authenticity, thus linking to construct validity Interactiveness, another test quality, describes the degree of involvement of test- takers' individual characteristics in fulfilling a test task (Bachman & Palmer, 1996) It considers individual characteristics like language ability, topical knowledge, and affective schemata, differing from authenticity by emphasizing the interaction between the individual and the task Practicality is the relationship between available and required resources (human, material, and time) for test implementation, development, or usage Although not directly tied to the use of test scores, practicality influences every stage of test design and development Finally, impact is defined as the influence of test use on individuals, the education system, and society at large

language testing, impact is used interchangeably with terms such as washback, consequences, backwash, and washback effects by researchers referring to varying scales and scopes of test effects.

In this study, the term "washback effects” is utilized as the primary terminology However, an analysis of empirical studies on the same topic reveals the interchangeable use of this term with several others by different researchers, including "consequences" (Shepard, 1993), "impact" (Hamp-Lyons, 1977), "backwash" (Alderson & Wall, 1993; Biggs, 2003; Spolsky, 1994 in Pan, 2009), and "washback" (Cheng, 1997; Messick, 1996; Shohamy et al.,1996).

Initially, "consequences" or “consequential validity”, proposed by Messick (1989) in Shepard (1993, refer to the positive or negative social consequences of a specific test The term was later broadened by Shepard (1993) this concept to encompass the social consequences of tests both within and beyond the classroom, covering aspects such as test uses, impacts on test takers and teachers, result interpretation by decision- makers, and potential misuse, abuse, and unintended usage of tests.

Cheng (1997) used “washback” and defined it as "an intended or unintended (accidental) direction and function of intended curriculum change on aspects of teaching and learning by means of the change of public examinations" (p.8) In Cheng's perspective, testing was seen as a catalyst for changes solely in the curriculum Shohamy et al (1996), also using the term “washback,” generalized it as the connection between testing and learning Bailey (1999) further divided the term into "washback to the learners," referring to test effects on students, and "washback to the program," denoting effects on teachers, administrators, curriculum developers, and counselors According to Bachman & Palmer (1996), "washback" fell within the scope of "impact," occurring at the micro level of participants (mainly teachers and learners), while "impact" encompassed macro influences on society Similarly, Wall (1997) suggested that "impact" refers to any effects tests may have on individuals, policies, or

practices across

various levels, from the classroom to the societal level In contrast, "washback" is narrower, focusing solely on the effects of tests on teaching and learning Messick (1996) viewed washback as a component of a test's validity and asserted that it influences validity only when it can be distinctly demonstrated to be an effect of the test itself and not of other educational factors at play on the scene (p.242) To clarify, Messick emphasized that only effects directly and unequivocally linked to the test should be considered as washback, excluding other test effects from this category Hamp-Lyons (2000) also distinguished "washback" as influences on teaching, teachers, and learning (including curriculum and materials), while "impact" is the term for broader influences of tests.

"Backwash" is considered a subset of "impact" or can be best understood within the scope of "impact" (Bachman & Palmer, 1996) Defined by Alderson & Wall (1993) as testing influences on teaching in general, and by Biggs (1995) as influences on both curriculum and teachers' methods and students' learning approaches, "backwash" has been interpreted as unintended effects and intended influences of tests on teaching and learning.

The term "washback effects" has been employed by Brown (1997), Cheng (1997), McKinley and Thompson (2018), Nguyen (2017), Pizarro (2010), and Xu and Liu (2018) Cheng (1997) defined it as "the influence of testing on teaching and learning," while McKinley and Thompson (2018) elaborated on it as the impact of language testing on curriculum design, teaching practices, and learning behaviors.

The diverse definitions above indicate that 'backwash,' 'washback,' and 'washback effects' refer to the same phenomenon with shared features, making them interchangeable However, the other terms are not entirely synonymous in terms of

scope and scale In this study, "washback effects" are operationally defined as theinfluences, either positive or negative, of tests on different aspects of teaching andlearning on a micro scale limited to the classroom In certain sections of Chapter

2, 'washback'

may be used instead of 'washback effects' when serving the working definitions Other terms are employed in Chapter 2 when cited by other researchers to present their empirical studies.

2.1.2 Washback dimensions

The perception of washback effects in testing is highly contingent upon the nature of specific tests and the chosen scope and scale of investigation by researchers into their influences This variability has led to the emergence of various approaches aimed at characterizing the dimensions of washback effects.

Wall and Anderson (1993), cited in Bachman & Palmer (1996), argued against a broad exploration of test washback studies solely in terms of their impact on teaching and learning Instead, they advocated for a more targeted focus on areas such as the content of teaching, teaching methodology, modes of assessment, and the presumed direction and extent of the impact (p.31) Going further from this, Cheng and Watanabe (2004) and Xu & Liu (2018) delineated six sub-dimensions within the concept of washback, namely specificity, intentionality, length, extent, intensity, and direction.

Washback specificity distinguishes between general effects applicable to any

test (e.g., learning motivation, feedback provision) and specific effects unique to a particular test or test type For instance, a general effect observed across various tests might be an increase in students' motivation to study due to the pressure of upcoming assessments On the other hand, a specific effect unique to a language proficiency test could be the adoption of certain teaching strategies tailored to the format and content of that specific test, such as intensive grammar instruction to prepare students for a grammar-focused language exam.

Washback intentionality encompasses both intended and unintended effects

resulting from a test Intended effects, such as motivating learners and providing feedback, are foreseen, and encouraged by test designers, while unintended effects arise

without the anticipation of test designers Examples of unintended washback include heightened anxiety and an excessive focus on test-taking techniques.

Washback length pertains to the duration of test-induced effects While certain

effects, like increased learning motivation, may diminish shortly after a test, students' habits of test preparation may persist over time For example, students might develop long-term study routines, such as regular practice sessions or the use of specific study materials, in response to the demands of a high-stakes exam like a college entrance test These habits could continue even after the test is completed, affecting their learning approaches in subsequent academic endeavors.

Washback extent denotes the range of influences exerted by tests, determining

whether their effects are confined to a school context or extend throughout the entire educational system The significance of a test correlates with its extent; high-stakes tests may wield nationwide influence, while the impact of classroom achievement tests is restricted to a specific class For instance, national standardized tests like the SAT or the national language proficiency exams in some countries can shape educational policies, curriculum design, and teaching practices across the entire country In contrast, quizzes or assessments conducted within individual classrooms primarily affect the students and teachers within that particular class, without broader implications for the educational system as a whole.

Washback intensity, as defined by Cheng (2004, p.33), refers to the degree of

the washback effect in specific areas of teaching and learning affected by an examination This intensity is often linked to the stakes associated with a test, where high-stakes tests exhibit more pronounced effects compared to their low-stakes counterparts Tests characterized by intense washback may impact teaching activities, influencing both the content and methods of instruction, as well as stakeholders’ attitudes toward teaching and learning In other words, tests with a strong effect will determine everything that happens in the classroom and will lead all teachers in the same way toward exams.

Otherwise, those with a weak effect will affect only a part of the classroom events, or only some teachers and students.

The term washback direction pertains to whether the impact of a test

encourages or discourages the intended forms of teaching or learning, emphasizing its potential to be either beneficial or damaging (Green, 2013) Washback is inherently neutral, capable of yielding positive or negative effects on individuals depending on the context (Shohamy et al., 1996) Stakeholders, as defined by Bachman and Palmer (1996, p.31), encompass those directly affected by the use of a specific test, such as test takers and teachers (Ahmad & Rao, 2012), thereby influencing their teaching and learning practices Pan's (2009) summary outlines the positive and negative washback effects of tests on teachers, learners, and the overall teaching-learning process Positive effects include inducing teachers to comprehensively cover syllabi within the stipulated time, motivating students to enhance their learning, and promoting the teaching and learning process by using tests as educational materials Conversely, negative washback effects occur when tests encourage a narrowed curriculum and loss of instructional time, evoke anxiety and distort the performance of teachers and students, guide students to focus solely on tested knowledge, and potentially demotivate learning Pan's (2009) descriptions illustrate the neutral direction of test washback, even though certain test types are more likely to generate positive effects In-class tests like free-recall tests (essays and short answers), cued-recall tests, and recognition tests (true/false tests, old/new recognition tests, and multiple-choice tests) are found to generate positive effects, whereas high-stakes standardized tests tend to produce negative effects on students, parents, schools, and teachers (Ford, 2018).

The term that is often used to discuss the direction of test washback is “teaching to the test” Teaching to the test refers to the practice in which instructors adjust their time allocation and priorities, concentrating more on the subject matter likely to appear in assessments, thus leading to a more focused curriculum (Cizek, 1998; Koretz, 2005;

Popham, 2002; Smith & Rottenberg; 1991; Stecher, 2002) It may also involve guiding students in developing test-taking skills This test-focused instructional approach, as outlined by Sieber (2009), presents several advantages Firstly, it often leads to improved test scores, as students become proficient in the specific content and skills emphasized in assessments This approach may enhance overall academic performance and achievement Secondly, in educational systems where test results are used for evaluating teacher or school effectiveness, teaching to the test ensures accountability Educators are prompted to cover essential topics thoroughly, contributing to a more transparent and measurable teaching process Lastly, test-focused instruction can establish clear objectives for both teachers and students, facilitating a targeted curriculum that aligns with the assessed content and skills However, this approach comes with notable disadvantages One major concern is the potential for a narrowed curriculum Relying exclusively on test-related content may lead to the exclusion of important subjects and skills crucial for holistic student development Moreover, the emphasis on test preparation may inadvertently undermine critical thinking and problem-solving skills, reducing the educational experience to mere memorization Additionally, the heightened focus on test performance may contribute to increased stress and pressure among students, negatively impacting their overall well-being and learning environment (Sieber, 2009).

In conclusion, the dimensions of washback effects associated with various tests exhibit variations based on their stakes and specific applications for distinct stakeholders Consequently, empirical studies become essential to obtain a comprehensive understanding of the washback phenomenon associated with a particular test within diverse educational contexts.

2.1.3 Factors generating test washback

Test washback manifests through various stakeholders or factors contingent on the test's utilization Regarding stakeholders, Alderson and Wall (1993), as cited in

Bachman and Palmer (1996), posit that not only individuals but also the educational system as a whole is impacted by washback Individuals encompass those with a vested interest in the test's application in a given context, directly affected parties like test-takers and teachers, and indirectly impacted individuals such as future classmates, co-workers, and employers (p.31) In essence, these researchers argue that every member of the societal and educational system experiences some degree of indirect impact due to test use According to Hughes (1993), there were three main factor groups, namely participants, process, and products being influenced by tests Participants encompass not only teachers and learners, as suggested by Alderson and Wall (1993), but also language testers, teacher trainers, parents, administrators, and curriculum designers Process encompasses all participants' actions throughout the learning journey, while products refer to learning content (facts, skills, etc.) and its quality (fluency) Similarly, Bailey (1996) contends that tests influence not only teachers and learners but also materials writers, curriculum designers, and researchers concurrently.

As found in Watanabe's (2004) framework, there are five distinct factor groups collectively contribute to test washback, including (1) test factors (e.g., test design and decisions based on test results), (2) test prestige factors (e.g., test stakes, test status), (3) personal factors (e.g., teachers' educational background, beliefs about optimal teaching or learning methods), (4) micro-context factors (school setting where the test is used), and (5) macro-context factors (society).

Synthesizing factors from various empirical studies, Shih (2009) proposed three major factor groups: contextual factors, test factors, and teacher factors The first group, contextual factors, comprises three sub-factors: national, social, or broader educational factors, school-level factors, and course-level factors (Grid 2.1).

Grid 2.1

Contextual Factors Generating Test Washback (Shih, 2009)

In these three-tiered contextual factors, the initial two contextual factors pertain to policymakers, test designers, and school high management Meanwhile, the third contextual factor at the course level is more student-centric, encompassing aspects such as proficiency level, feedback, and learning motivation Additional course-level factors involve the course objectives, class size, and influences from parents as stakeholders The second factor generating washback, test factors, comprises eight sub-factors that can be further classified into two sub-groups, as detailed in Table 2.1.

Table 2.1

Test Factors That Generate Washback (Shih, 2009)

TEST FACTORS

Test content related factorsTest stake related factors

The extent to which the test is counter to current teaching practices

Extra administrative work entailed by the test

Language skills tested The status of the language tested

This table illustrates Shih's (2009) emphasis on language aspects, such as knowledge and skills measured by the test, and the alignment between test content and teaching practices as crucial test factors However, it is noteworthy that factors related to test administration and stakes also play significant roles In comparison to Watanabe's (2004) classifications, Shih (2019) provides a more comprehensive list The third factor group identified by Shih (2009) as influential on teaching comprises 14 teacher factors, synthesized from studies by researchers like Anderson and Wall (1993), Green (2007), Shohamy (1993), and Watanabe (1996) These fourteen factors can be organized into three sub-groups: teachers' experience and training, their perceptions of tests, and their beliefs in effective training and test preparation, or what Watanabe (2004) terms as teachers’ personal factors (refer to Table 2.2).

Table 2.2

Teacher Factors That Generate Washback (Shih, 2009)

Teachers’ experience and

Learning experience Teachers’ perceived

importance of the test to the student

Teaching philosophy (e.g objection to test-driven instruction)

Leachers’ abilities in the language they teach

Teacher’s familiarity with the test and the educational context

Teachers’ familiarity with a range of teaching methods

commitment to the profession Other teaching commitment or various obligations

Teachers’ willingness and capability to innovate The impact of students’

performance on teachers’ job and teaching philosophy (e.g., objection to test-driven instruction)

To sum up, the factors influencing test washback can be categorized into three subgroups: contextual factors (encompassing micro and macro contexts), test factors, and teacher factors These main factors groups not only interact with each other but also exhibit interrelatedness among the specific factors within each subgroup In the present study framework, these three subgroups are employed with appropriate adjustments to suit the educational contexts of Vietnam.

2.1.4 Washback models

The factors or components of washback, as discussed in the preceding sections, interact with each other in a specific mechanism to produce washback The mechanism of washback is intricate (Bachman & Palmer, 1996; Nguyen & Gu, 2020) Various models have been proposed to explain how washback factors operate Notably, five models have been put forward by Alderson and Wall (1993), Bachman and Palmer (1996), Bailey (1996), Hughes (1993), and Shih (2009) The forthcoming descriptions of each model will primarily focus on their effects on teachers and their teaching, given that teachers are the main subjects of this study.

Alderson and Wall (1993) introduced 15 hypothetical influences of tests washback on teachers and learners, and their teaching and learning, which can be classified into two main categories of washback to teachers and teaching and washback on learners and learning as presented in Table 2 3 The letter H in each row of the

tables stands for hypothesis Those describing effects on both teachers and learners are

presented in unseparated rows According to Alderson and Wall (1993), the prerequisite for washback is the stakes of the tests; specifically, only high-stakes tests cause washback (H12 and 14) Once washback occurs, it affects all teachers and learners, although the individual effects may vary (H14 and H15) In this washback model, the impact on teaching encompasses influences on both what teachers teach (degree and depth, content of teaching) and how teachers teach (rate and sequence, method of teaching), along with the attitudes of teachers toward these two aspects Notably, Alderson and Wall (1993) focus on only two test washback participants: teachers and learners.