MINISTRY OF EDUCATION AND TRAINING HANOI NATIONAL UNIVERSITY OF EDUCATION NINH THI HANH DESIGNING AND USING THE COURSEWARE In TEACHING HISTORY GRADE 10 IN HIGH Schools Major: History Teaching Theories and Methodology Code: 9.14.01.11 SUMMARY OF EDUCATION DOCTORAL THESIS HANOI, 2019 This work was completed at Hanoi National University of Education Scientific Instructors: Dr NGUYEN THI BICH Dr HOANG THANH TU Reviewer 1: Assoc Prof, Dr Tran Viet Luu Central Propaganda and Training Commission Reviewer 2: Assoc Prof, Dr Nguyen Thi The Binh Hanoi National University Of Education Reviewer 3: Dr Vu Thi Ngoc Anh The Vietnam National Institute of Educational Sciences This thesis will be presented and defended at the Thesis Examination Board meeting at Hanoi National University of Education The thesis is available at: National Library of Vietnam, Hanoi Or Hanoi National University of Education’ Library A LIST OF RELATED RECOGNIZED STUDIES Hoàng Thanh Tú, Ninh Thị Hạnh (2011), Designing and using historical documentaries with the help of software Proshoww Gold, educational journal, issue 68, page 25-27;30 Ninh Thị Hạnh (2013), Using Google Sites service to design WebQuest to facilitate teaching in high schools Science journal, Hanoi II University of Education in high school, issue 27, page 106-112 Ninh Thị Hạnh (2014), Using software Hot Potatoes 6.0 to design testing tools to evaluate students’ result in History in High school, Science journal, Hanoi II University of Education in high school, issue 31, page 81-86 Ninh Thị Hạnh, Ngô Sách Đăng, Ngô Duy Nam (2016), Design and carry out lesson plan in High schools with assistance of technology tools, Journal of Science (Volume 42/2016), Hanoi Pedagogical University 2, ISSN 1859 – 2325, p.124 - 130 Ninh Thị Hạnh (2016), The applying of blended learning to teaching Vietnamese history in high school, The conference “Researching and Teaching History in the current context ", Nxb Political Theory Publishing, H 2016, p 304 – 314 Ninh Thị Hạnh (2017), The national digital learning resources network in Australia – Experience for designing Vietnamese history digital learning resources, The International Conference: History teacher training and professional development for curiculum and textbook reform, Vietnam National University Publishing, Ha Noi, 2017, p 485 – 491, ISBN: 9786046299769 Ninh Thị Hạnh (2018), Ninh Thị Hạnh (2018), Using the courseware to teaching history in high school, Journal of Science (Volume 55/2018), Hanoi Pedagogical University 2, ISSN 1859 – 2325, p 104 – 116 Hoang Thanh Tu, Ninh Thi Hanh (2018), Applying the Flipped Classroom to Teacher Training in Vietnam: A case study in Faculty of History in HPU2, The International Conference on Flipped Learning for Technological and Vocational Education, National Taiwan Normal University, p 107 – 116 INTRODUCTION Rationale 1 According to the modern pedagogy, educators consider the integration of all the technological products into the teaching the fourth educational revolution (after the advent of schools, letters, printing and books) The application of techniques and technology in teaching, especially Information Technology offers new ways of teaching, which encourages students‟ independence and proactive involvement In fact, “ 40 percent of population uses the Internet on the daily basis currently Imagine what we can when the ratio is 50, 60, 70 percent” This is a statement by Sheryl Sandberg - Facebook CEO, at the Annual Economic Forum was held in Davos, Switzerland (January 2015) This signifies the advancement of Information Technology and its continual influences on most domains, which is likely to comprehensively change economy, society and especially education Information Technology (IT) creates opportunities and challenges for education renovation in terms of syllabus contents, teaching methods, and usages of modern devices such as electronic material resources The growing development of the Internet together with popular applications of smart electronic gadgets allows people to have interaction with each other easily around the world This opens up opportunities for the growth and expansion of new teaching methods, besides the traditional methods (face to face) Blended learning is a new approach which is the combination between conventional teaching and elearning The application of binary teaching method including face-to-face and integrated-based teaching ones in general and history teaching in particular is the new trend, which is appropriate for the reality and current teaching renovation: “The application of Information Technology in contents and methods of teaching including the effective utilizations of devices, teaching software, virtual experiments, emulation software: fruitful uses of e-learning modern system in privileged areas, adapting elearning lectures, e-books, the use of online evaluation system ” 1.2 History teaching is typical of a subject Unlike other subjects, historical knowledge bears its own properties such as past, and non-recurrence History cannot be directly observed history, but indirectly perceived through historical sources as well as teaching facilities Thus, in history teaching, visual aids in general and courseware in particular play a key role Courseware presents information through multimedia, which enables the link to any database in computers via websites without the limit of time and space This is, therefore, significant to not only simply specify but also support learners to develop thinking and capabilities In Vietnam, the utilization of courseware in history teaching has been carried out on a large scale Nonetheless, its application has not been notably fruitful in enhancing learners‟ aptitudes, and creating learning incentives by dint of colossal factors like infrastructure condition, and teachers‟ approach 1.3 History for grade 10 plays a pivotal role in History curriculum in high school including international and Vietnamese History from primitive times to ancient -medieval era and the first phase of early modern period The material sources for this period are so available that teachers and students may easily access to them Hence, it is flexible for teacher to adapt and organize their lesson contents as well as opt for suitable materials matching with teaching approaches In reality, teachers, nevertheless, have difficulties in planning and utilizing electronic materials effectively From aforementioned rationales, we have chosen: “Designing and using courseware in teaching History for grade 10 in high school” as the title for Doctor of Education thesis on principles and methods in History teaching The subjects and scopes of the study 2.1 The subject of the study The subjects of the thesis is the planning procedures and the ways of using electronic materials in face to face teaching as well as blended teaching for grade 10 in High School 2.2 The scopes of the study - In terms of principles and methods of teaching History: The thesis puts an emphasis on the study of principles and methods of utilizing visual aids in general and teaching facilities in particular, especially modern teaching tools including the application of Information Technology in teaching History The thesis also focuses on the study of teaching methods in modern teaching perspectives - The scope of the study: + In terms of content: the thesis focuses on History for grade 10 in High Schools (standardized curriculum) so as to propose the knowledge which can be exploited to design courseware + In terms of courseware: The thesis concentrates on the recommendation of planning procedures and methods of using a specific courseware namely learning websites which is the combination of Face to face and Blended learning - In terms of the scope of research, situation investigation and experiment: +The survey was carried out in 67 high schools in 18 provinces with representatives in three main regions nationwide + The initial experiments and parallel experiments were exercised in five different high schools through lesson 27 and lesson 32 of History 10 Objectives and tasks of the thesis 3.1 Objectives Basing on the roles and significances of e-learning materials, the thesis constructs design procedures and contents of courseware in the form of learning websites as well as proposes the ways to use courseware in Face-to-face learning and Blended learning so as to improve the effectiveness of History teaching for grade 10 in high schools 3.2.Tasks With a view to fulfilling the above aims, the subject needs tackling the specific tasks as follows - Study the sound arguments of designing and using courseware in History teaching in high schools - Carry a survey related to designing and using courseware in History teaching in high schools, which is the practical foundations for the thesis - Design courseware in learning websites catering for teaching History for grade 10 - Recommend procedures, measures pertaining to using courseware in the form of Face-to-face and Blended learning to enhance the effectiveness of History teaching for grade 10 in high schools - Exercise educational experiments to verify the feasibility of the procedures as well as the proposed courseware uses Methodology and reasearch methods 4.1 Methodology The methodology of the thesis is the arguments of Marxism- Leninism related to Ho Chi Minh perception, ideology, and the guidelines of the Communist party on education in general and on History teaching in particular Research methods To study the subject, we use a wide range of medthods: - Study theories: examining the arguments on methodology, teaching methods, applications of Information Technology in teaching, especially the arguments on courseware and forms of teaching through reading, collecting and analysing documents like newspapers, books, maganizes, the Internet ; study curriculum, current History textbooks for grade 10 and projects for History teaching after the year of 2018 to select an option for suitable contents and topics to design courseware - Study practicality: Use a diverse methods of research such as surveys, interviews, observations to identify the reality of using courseware in History teaching in high schools and initially estimate the effectiveness of using proposed courseware Use the professional approach to preminilarly survey, appraise the quality of the designed courseware - Use the educational experimental method to verify the feasibility of the procedures, methods of using courseware in History teaching in five high schools in different provinces and cities namely Vinh Phuc, Hanoi, Ho Chi Minh city - Use methods of statistics to process, accumulate, analyze the collected data during educational experiment procedures and evaluate the feasibility, effectiveness of proposed measures of the study Scientific hypotheses The era of 4.0 industrial revolution has had a substantial influence on education, which makes courseware play an increasingly important role in teaching If design procedures are constructed and mearures of using proper courseware are taken, this can contribute to improving History teaching qualities in high schools in general and History teaching for grade 10 in particular Contributions of the study The results of the study will contribute to: - Affirm the roles, significances, necessities of designing and using courseware in History teaching generally and History teaching for grade 10 particularly -Evaluate the reality of utilizing courseware in the forms of Face-to-face and Blended learning in History teaching in high schools - Propose design procedures and measures of using courseware in the form of Face-to-face and Blended learning in History teaching for grade 10 in high schools The study’s scientific and practical significances The study is significant in terms of science and practicality: - In terms of science: The thesis diversify the arguments and methods of History teaching in the domain of using Information Technology in History teaching in general, design and using courseware in History teaching for grade 10 in particular - In terms practicality: The study is considered the documents guiding high school teachers to design and use courseware in teaching to improve teaching quality Besides, it can function as references for undergraduates and post-graduates specialized in Principles and Methods of History teaching The structure of the study In addition to Introduction, Conclusion, Appendix, Reference, the content of the study consists of four chapters Chapter Literature review Chapter Designing and using courseware in History teaching in high schoolstheory and practicality Chapter Designing courseware in History teaching for grade 10 in high schools Chapter Using courseware in History teaching for grade 10 in high schools Educational experiment Chapter LITERATURE REVIEW 1.1 Researches into courseware in teaching 1.1.1 In foreign countries The advancement of Information Technology leadsto a considerable increase in modern visual aids including e-learning materials Researches on Information Technology and e-learning started to flourish in the late 20th century The researches carried out by scholars from America, France, Australia, Singapore and UNESCO must be taken into account Most of these researches have affirmed that Blended learning is one of the effective measure which enhances the productivity of applying Information Technology in teaching in general and coursewarein particular Specifically, learning websites is one of the popular courseware used for Blended learning thanks to its unmatched advantages and its capability to meet the demands of educational renovation pertaining to the applications of Technology throughout the world 1.1.2 In Vietnam Researches into the application of Information Technology in teaching in general and using courseware in particular have become increasingly popular in the 21st century Many experts in teaching principles and methodology have taken the individual study of e-learning materials, especially learning websites into consideration, which have been represented by recognized articles in privileged journals Researches into courseware pertaining to History teaching 1.2.1 In foreign countries A colossal number of scholars have studied the tendency of the applications of Information Technology, specifically, the uses of courseware in History teaching These studies preliminarily shows the key role of using courseware in History teaching in general as well as takes an insight into the analysis of the roles, procedures, effectiveness of some particular courseware which can be utilized in History teaching like learning websites, documentaries, games, e-books, e-lesson plans, etc 1.2.2 In Vietnam In Vietnam, the study of using courseware was not diverse and systematic with the initial ones related to the study of principles of using visual aids in History teaching in general However, in the 21st century, together with the development of education throughout the world, the researches into using courseware in teaching in general and History teaching in particular have been increasingly various 1.3 Overall evaluation, inheritance and problem solving Taken together, some issues can be taken into consideration throughout the foreign and domestic study of using Information Technology, courseware in teaching in general and History teaching and in particular as follows First, affirm the necessity and importance of using Information Technology, courseware in teaching environment inclusing History teaching Second, catergorize courseware and take an in-depth study on specific e-learning materials Each type of courseware has its own merits when used in teaching in varied teaching conditions Third, identify the approach to make full use of courseware Three main ways of teaching comprising Face-to-face, Blended learning, E-learning are common nowadays Nonetheless, no researches have dealt with the following issues symtematically First: What are the significences of using courseware in History teaching in high schools in current educational renovation? Second: What are the merits and demerits of using courseware in History teaching in high schools? Third: How can courseware be used to be compatible with level of knowledge? Fourth: What are proper designed courseware used for History teaching in high schools currently? Fifth: How can the design procedures and methods used courseware in History in high schools in general and History teaching for grade 10 in particular be used effectively? On the basis of the previous achievements, the thesis puts an emphasis on addressing the following problems: First, explain the concepts, terms like courseware, courseware in History teaching, design and use E-learning in History teaching in high schools, classify the properties of courseware, face-to-face, blended learning Besides, on the ground of the objectives of the subject, analyze the historical knowledge identities, characteristics of students‟ perceptions in History teaching; analyze the roles, significances of using courseware in History teaching Second, examine the relationship between using courseware and other elements in History teaching in high school, demand for the change of teaching procedures and propose the fundemental requirements of using courseware in History teaching in high schools Third, have an overview of the reality of History teaching in high schools as well as take an insight into analyzing the advantages and disadvantages of using courseware on the survey results, which leads to the practical issues which need tackling Fourth, study the syllabus of History for grade 10 in high school and identify proper knowledge which can be suitable for courseware, opt for appropriate design procedures matching with the objectives To be specific, the thesis will design courseware in the form of learning websites directly support Face-to-face and Blended learning 10 types of schools such as gifted schools, public schools, private schools, international schools, centers for continuing training were surveyed The information from the data analysis is valuable for us to propose the measures of designing and using courseware in History teaching in high schools The results of the survey show that courseware play an important role in developing students„ aptitudes (80%) In reality, not many teachers are aware of using courseware in guiding students self-study, self-search and explore information (3.1%); give students guidance on Internet interaction (12.5%) Most teachers use courseware in Face-to-face (84.4%) The biggest challenge does not lie in the lack of facilities and modern teaching tools It is the students perplexity in opting and using courseware efficiently as well as teachers„ confusion and time-consumption in preparing a lesson plan with courseware that matters most 2.2.3 General assessments First, teachers seem to be unaware of students„ interests, demands, hobbies when designing and using E-learning materials Second, although teachers are concious of the importance of courseware in enhancing learners„ talents, the effectiveness of using courseware in History teaching has not come up to their expectation Third, teachers have used various forms of E-learning, but they just use the simple ones like text files, images, audios, videos, slides Teachers have not either made full use of high interactive courseware like interactive slideshow, inforgraphics, learning websites or updated new tools to increase the interaction in teaching Fourth, the shortage of a diverse E-learning material system with high reliability and appropriate procedures which can be suited to teaching ideas is a great hindrance to both teachers and students Chapter DESIGNING COURSEWARE IN HISTORY TEACHING FOR GRADE 10 IN HIGH SCHOOLS 3.1 Postition, objectives, contents of History syllabus for grade 10 3.2 The contents of History syllabus is suitable for designing courseware On the basis of analyzing the position, objectives and contents of History syllabus for grade 10, we confirm that the basic contents of History for grade 10 can be designed as courseware which may facilitate as follows: the knowledge containing ambiguous concept that students need to get to know its natures; the knowledge including a various historical events which are related to the course of events or the developmental stages of an event, a historical phenomenon in one or more than one periods; knowledge contents that students must analyze, evaluate, give remarks or give opinions; knowledge contents with multi-leveled integration In this 11 case, using courseware function as information supply as well as other subject complements which help students gain in-depth knowledge of historicals events and historical phenomena Teachers can use learning websites for all the aforementioned contents of knowledge as courseware is in the form of gathering all documents Moreover, learning websites are also appropriate for knowledge which is generalized or revision part at the end of the lesson, chapter summary or such part as a preliminary summary of Vietnamese history from its origin to the mid-19th century 3.3 Principles in designign and using courseware in History teaching in high schools Serving as a tool in teaching in general, History teaching in particular, designing courseware must be based on pedagogical science generally and on History teaching pedagogical science particularly as well as the social demand and school requirements in the renovation of History teaching method of the application IT in education To be specific, the reform must guarantee scientific, sytematic, interative, multimedia, technical, and aesthetic qualities 3.4 Procedures for designing courseware in History teaching for grade 10 in high schools To design courseware effectively, teacher should follow the steps in the flowchart below: Figure 3.1 Process of designing courseware for History teaching 12 3.5 An introduction of a learning website: HISTORY 10 (https://1095397.site123.me/) The website comprises home page and another links including: Introduction; General procedures; Learning task; Evaluation; Conclusion; Contact These links appear on the left of the learning website so that users can easily follow Each link has its own text with images Whenever someone wants to access the link, just click the intended link Homepage and each link are designed to excercise a particular task in teaching procedure Basing on the researches into the theories on visualization in teaching in general, the use of teaching tools and the researches on designing courseware in particular; basing on the current advancement of IT as well as the requirement for the application of IT in History teaching, we design the webpage with five particular properties: First, the interface is modern and friendly; Second, structures are in accordance with teaching procedures; Third, learning activities are designed to develop students’ talents; Fourth, the courseware system is abundant and reliable; Fifth, the possibility of interaction is high Chapter USING COURSEWARE IN TEACHING HISTORY 10 IN HIGH SCHOOLS - AN EDUCATIONAL EXPERIMENT Requirements for using courseware in History in High Schools Overall, to exploit the potentials of courseware, teachers should pay attention to the requirements for using courseware in teaching as a modern visual teaching aids All the requirements should be specific as follows: guarantee safety; guarantee 3D; guarantee effectiveness; guarantee practicality 4.2 Courseware use procedure in History 10 teaching in High schools Generally, if teaching procedure is considered a information transmission from teachers to students as well as process of interaction between students and infromation, teaching tools in general and courseware in particular function as information transmission tools to learners Though there is a difference between Face-to-face and Blended learning, the use of oursewares in History teaching must follow a common procedure including four following steps: 13 Figure 4.1 Process of using courseware for History teaching In face-to-face lessons, teachers can make a lesson plan that comprise three main parts: teaching procedure; types of courseware used; teaching and learning activities Table 4.1 A lesson plan using courseware in face-to-face Forms of courseware Procedur e Warm-up (review previous lesson, introduce new lesson) Knowledge, skill, new attitude formation Knowledge review and consolidation Evaluation Text Teaching activities Images, Audio, Worksheet Teachers Students graphs documentaries 14 In blended learning, the procedures consist of elements: Organizing activity process; blended learning, Face-to-face and online learning Here are the structures of teaching activities which utilize courseware in Blended learning Table 4.2: A Blended-learning lesson plan TOPIC/CONTENT A Procedure Blended learning Face-to-face Teachers Warm-Up (previous lesson review, new lesson introduction) Text introduction (content and method) New lesson introduction Online learning Students Answer questions (link to new lesson if necessary) Objective New introduction and Exchange knowledge, objectives of answers and skill, attitude each task support raise questions formation students directly Teachers Students Give marks Do tests Online support supervise individual and group fulfillment Do online tasks: perfect products, answer questions Review, consolidation Set requirements, Exchange Online support give remarks on answers and students‟ answers raise questions Evaluation Guide students to Students multiplequestions choice questions online Do task on Canva Modify Do Kahoot raise questions time multiple allowance if choice necessary questions 4.3 Methods of using courseware in history for grade 10 in high schools 4.3.1 Use courseware effectively in Face-to-face Face-to-face learning, teachers use courseware as modern visual aids which facilitate teachers to prepare, oganize and instruct students In addition to complying with standards of classroom and teaching necessities for history in high schools, to utilize courseware in Face-to-face in classrooms need to be equipped with other facilities like projectors, screen, large-screen TV with HDMI connected with laptops or computers 15 4.3.1.1 Use courseware to create learning situations and raise awareners 4.3.1.2 Exploit courseware to eventualize, phenominonize and historical figures 4.3.1.3 Give guidence on exclanging, analyzing courseware to draw conclusion 4.3.1.4 Give guidence on using courseware to practice, consilid the lessons 4.3.1.5 Give guidence on using courseware to prepare the homeworks 4.3.1.6 Give guidence on supporting the examination and assessment of students' learning results 4.3.2 Use courseware flexibly in Blended learning In blended learing, courseware, website history 10 is multifunctional, which means that it is both learning material resource and learning environment In order to use courseware in history teaching in blended learning, schools need to have good infrustructure, modern facilities Paticularly, school in big cities international schools, bilingual schools, multi-grade schools, national standard schools, gifted schools With these favorable conditions, teachers can carry out activities which utilize online materials, online interations classrooms need wify connection or internet connections, as well as computers can install basic software like web browse MS Powerpoint, Kahoot, Carva, Padlet, 4.3.2.1 Introduce courseware as well as objectives and tasks 4.3.2.2 Give guidence on self – study courseware to the gives tasks 4.3.2.3 Hold interactive activities to report products 4.3.2.4 Give guidence on basing on courseware to test to themselves 4.4 Educational experiments 4.4.1 Objectives First, educational experiments are scientific bases to test the feasibility of using courseware in history teaching in high school Second, evaluate the effects of using courseware in history teaching on results, attitudes, interest of learners Third, estimate the possibility of using courseware in approaches of teaching organization Fourth, scientifically affirm the hypothesis 4.2.2 Targets and experiment areas Experimental targets are 10 graders of schools in Vinh Phuc – Ha Noi and Ho Chi Minh city in school year 2017-2018 students who are choosen are in different regions: big cities, countrysides, plains including, gifted schools, public schools, 16 private schools, international schools to guarantee criteria in the experiment 4.4.3 Experimental approach In terms of website qualities:https://1095397.site123.me/ Using professional approach and surveys to estimate the designed materials namely learning website:https://1095397.site123.me/ To evaluate the quality, we conducted a survey to gather professionals‟ opinions, history teachers, students in high schools The contents of the survey mainly focus on the evaluation of content, structure, texture and significances In terms of the procedures of using courseware in Face-to-face and blended learning We carried out educational experiments on two lessons: Lesson 27: The process of country‟s construction and defence in the form of revision lesson and lesson 32: Industrial revolution in Europe in the form of knowledge provision We followed a 2step process Step1: Initial experimentation: confirm quality and modify the content, interface, structure, technical items of the website https://1095397.site123.me/ Step 2: Parallel experimentation, after the confirmation of the effectiveness of the website We used the website in the proposed process which was conducted on a wide range of targets Basing on the result, the efficiency of the website on the learning process is verified and pivotal conclusions are drawn The students who were chosen to be the targets of the empirical research as well as experimental research were equal in terms of the number as level of History learning, which met the research‟s requirements 4.4.4 Experiment procedure Table 4.3 Experiment procedures While- experiment PreB1 Initial B2 Parallel experiment experimentation experimentation Purpose - Survey of - Provide initial - Adjust and students' conclusions about improve the conditions for the quality of the website using site and oriented Verify the computers, to adjust to the effectiveness and smartphones experimental feasibility of the and Internet evaluation process using the connection site Postexperiment - Get feedback from students about issues: form, content of the website; student's interest in learning, 17 Tools Student - Questionnaire surveys through for experts and Google Forms teachers Direct observation of the classroom - Product of students Direct observation of the classroom Test of knowledge through Kahoot Product of students through Canva; Padlet difficulties and suggestions - Vote for feedback from students via Google Form In-depth interview with students 4.4.5 Experiment results The results obtained form the experiments are in two forms: qualitative and quantitavive information Qualitative information is formed on logical resolutions, which signifies the speculations through: + The students‟ ability to finish products after a lesson like: an introduction of a historical figure, posters, brochures, catalogue, worksheets, statistic tables, event comparisons, historical phenomena + Students‟ feedback after the experimental lesson + Teachers‟ observation and remarks on students‟ attitutes, behaviors after the lesson + Observers‟ comments on the effectiveness of the lesson Quantitative information based on arthimatical resolutions, the use of statistics through: + The result of the multiple choice test at the end of the class + The scores that groups get for their products: an introduction of a historical figure, posters, brochures, catalogues, worksheets, statistic tables, event comparisons, historical phenomena + Students‟ feedback result + Teachers‟ and obervers‟ comment result 4.4.5.1 Results of qualitative information analysis - First, the results of observations - Second, the results of in-depth interviews with students and teachers 4.4.5.2 Results of quantitative information analysis In terms of students’ interest We got feedback and interviewed 276 students who joined in the lessons 100% students said that they were interested in the lessons with learning website: https://1095397.site123.me as well as exploiting courseware 18 in learning activities The statistics shows that the proportion of wrong anwers are low (roughly a third of questions in multiple choice test) The average mark fluctuates between to 9.0, which presents that the efficiency of the use of learning websites in helping students with memorizing and understanding the lesson The results of marking students’ products: The marking is based on specific criteria for each product as well as criteria for the whole group work process of each student After the experiment, the results are as follows Average score - initial experiments and parallel experiments using courseware in Blended learning 70 60 50 40 30 20 10 EXCELLENT KH1-L3 KH2-V1 GOOD AVERAGE KH2-V2 KH2-F2 POOR KH2-F3 Figure 4.2 The result of using courseware in initial experiments and parallel experiments in blended learning Table 4.4 Average score table of students and results of marking students' products in initial experiments and parallel experiments TB 7,3 8,1 8,3 TB 7,6 STĐ SD 0,46 0,27 0,25 SD 0,44 SH2-V1 SH2-V2 8,4 0,31 8,6 0,30 3,47.10-6 5,42.10-7 SH2-F2 8,7 0,29 2,94.10-5 SH2-F3 8,9 0,26 1,03.10-4 Experiments Face to face Initial Parallel TT1-T4 TT2-Q14 TT2-Q15 Blended learning KH1-L3 Experiments Initial Parallel p 1,40.10-6 4,20.10-6 p 19 Statistics show that the use of courseware in Face-to-face learning and Blended learning is efficient, which can be seen in: The average point of classes in experiment research is higher than that of empirical research, which denotes that the proposed measures are more and more effective Standard deviation in classes of experimental research is smaller than that of empirical research, which denotes that the inattentive level of experimental class is lower than that of the empirical classes In other words, learning quality, the attentiveness of experimental class is better than the other P-value of experimental class