Three Layered Thinking Model for Designing Web-Based Educational Games 269 Curriculum goal Pedagogic Previous skill & C Level Game goal Knowledge enhancement O N T Design E Level X T Achievement M1.Pleasure M3.Skill enhance Level M2.Challenge M4.Concentration (4 Motivators to flow) Game Style Desi g n Game Task Desi g n Game Interface Design Fig. 1. Three Layered Thinking Model of Web-based Educational Game Design 4 Validating the Model In order to explain how the above model is useful in developing of educational games, three games were evaluated. We illustrated the expert review results followed with the survey from the learners. 4.1 Profile of Learners/Students The subjects consisted of 120 undergraduate students who had been taking an infor- mation literacy course provided to all students who are non-management information systems majors. Some 53% of the subjects were male while 47% were female; around 3% of the subjects were freshmen, 26% were sophomores, 37% were juniors and 34% were seniors. 4.2 Game Style, Task and Interface Design The games were developed by a team of researchers and designers for the Web-based course “Introduction of Software Applications”. The Web site is located at http://www.elearn.cc.nccu.edu.tw and is accessible with passwords. The first game was a simulation of how components are installed into a motherboard of a personal computer (PC), as shown in Figure 2 (a). Research has indicated that some novice users believed that the insides of a computer were bunches of wires connected to- gether in a mass tangle. The purpose of this game was to increase students’ under- standing of the shape and position of all components inside the shell of a PC. The second game was a diagnosis of the components required for a PC purchase. The purpose of this game was to increase students’ understanding of how to buy a new PC based on their needs. Suggestions on the requirements of the necessary PC components were given as feedback to the learner after a series Q & A regarding the user’s purpose, preference and budget. The third type of game was designed to en- hance players’ memory of operational procedures through stories. Many students are 270 F L. Fu and S C. Yu Game1. Installation of mother board (a) Game2. Suggestions of PC Components (b) Game3. Manipulations of Windows XP (c) Fig. 2. Demonstration of game interfaces interested in knowing more about problem-solving issues with regards to computer operating systems, including deleting cookies, rearranging hard disks, reinstalling software, and so forth. However, the steps involved in these tasks are difficult to re- member for rookie users. The games were thus designed to associate the procedures with a play serial dotted with quiet animators and light, and humorous conversations on a battle against crimes in a big city. The “big city” is akin to the operating system (Window XP) in a computer. The “crimes” are problems with computer usage, such as a kidnapping of the home page. Each scene in the story is displayed on the right side while corresponding problem-solving procedures are shown in a screen on the left part, as demonstrated in Figure 2 (c). After the player performs the correct proce- dure, he/she can click on an “arrow” to continue reading the next scene. 4.3 Skill Enhancement Design The following expert reviews were through subjective ratings. For each goal, all games were assigned a numerical value between one and five, indicating the extent to which the games supported that criterion. Values from one to five indicate “not at all,” “a little,” “average,” “good,” and “well done,” respectively. Pedagogic considerations focus on whether the curriculum goals are attainable by means of game goals, and whether the game style matches the knowledge taxonomy. The goals of Game 1 and 3 were rates to fit better their respective curricula goals than Game 2 because skills in strategic thinking are more difficult than skills in memory or recognition. Based on Bloom’s taxonomy of knowledge, the strategic thinking in- volved in PC components combination (as targeted in Game 2) requires a higher level of knowledge than the other games. The pedagogic method in the game designs in this study was the experiential theory mentioned in [5] and [11]. The earliest, initial learn- ing process in computer games is behavioral learning. Players learn by trial and error as well as stimulus associations. When the basic rules of a game are understood, play- ers start to think cognitively about how they should respond to a new situation and actively update existing knowledge to fit the new things they are confronted with in the game environment [13]. According to suggestions from Prensky [12], simulation games provide content that help develop the player’s system of understanding, while adventure games can improve the user’s development skills. Therefore the match between knowledge and game style was rated to be good for Game 1 and 3. The Three Layered Thinking Model for Designing Web-Based Educational Games 271 puzzle game, which was the game style of Game 2, was suggested to fit best with the learning goal of strategic reasoning [12]. 4.4 Challenge Design The games create scenarios and provide challenges to invoke the learners’ curiosity and keep them involved. Players immerse themselves in the game when the challenge provided by the game matches the skills they have [8]. If the challenge in the game is higher than the player’s skill, he/she will become anxious. If the challenge of the game is beneath the skill level of the player, then the player will become bored. Both situations impede progress [8]. We evaluated challenges provided in the games ac- cording to how well these problems are structured and how closely they match the players’ skills. There were fixed solutions and procedures in both the installation of a motherboard and the manipulation of Windows. The skill required in Game 1 was to recognize the positions of each component through trial and error. In the game that involves the manipulation of an operation system, the skill called for was to remem- ber the sequence of procedures that solve a problem. As the games were offered in an introduction to computers course, we assumed that these challenges matched the players’ skills. The problem provided in Game 2 was considered to be too easy in terms of how much it challenged players’ skills. 4.5 Pleasure Design All three games were single-player games that provide for interaction only with the software. There are two arguments concerning action-based drill and practice games such as Game 1. One point of view stipulates that in such games, players might simply keep on experimenting with actions until their scores improve. However, such behavior, based only on trial and error, may not bring about a higher level of intelligence [8]. On the other hand, the feedback delivered through rewards or punishments from the game indeed reinforces the stimuli and responses, and facilitates the learning of some behaviors [13]. The physiological pleasure comes from the graphics, sound, as well as the interaction with the systems. The psychological pleasure comes from positive feedback, such as score and/or applause. Both of the above satisfactions could encourage the player’s engagement in playing. All components in Game 1 were designed to resemble the real objects as closely as possible. Therefore, the physiological pleasure of the game to be rated “good”. Positive feedback is available when the computer “applauds” the player when he/she finishes the task of installation, so the psychological pleasure was rated “average”. Game 2 was a conversation-like diagnosis complete with a coloful interface. We rate the physiological pleasure as “average”. All the players could finish the task no matter what their skills or backgrounds were. The game does not rate the player’s performance. Therefore we see the psychological pleasure as lacking. Students probably garnered more pleasure from Game 3 due to its dramatic storyline, cartoonish characters and humorous dialogue. Therefore we rate the physiological pleasure of Game 3 as “well done”. The psychological pleasure is assessed to be average because players can finish the task simply by following the computer’s clear- cut directions. 272 F L. Fu and S C. Yu 4.6 Concentration Design The context factors involved in the design of educational games included the physical environment, equipment, technical support personnel, and so forth [3, 5]. The level of concentration is determined by the stimuli and the workload the game provided. Storylines and activities are considered as the stimuli while heavy demands on the player’s memory capacity are regarded as a high workload [15]. All the basic rules of playing these three games are comprehended by the players without the need for tu- tors to assist them [15]. The importance of the storyline to a game depends on the complexity of the game [8]. Game 3 used a serial of adventure story to help learners remember the procedures. The humorous stories were considered to help maintain players’ concentration. Game 2 created several scenarios in terms of PC component combinations. The scenarios themselves served as a small story. A serial of questions and answers before the diagnosis in Game 2 created a discontinuous gap between the facts and strategic results. The overly complicated screen design that showed both the story and operation demo in Game 3 was also considered to be a factor hampering players’ concentration. 4.7 Total Achievement on Educational Game Four achievements on Web-based educational games were goal (M1), challenge (M2), concentration (M3), and pleasure (M4). Total achievements on educational game flow can be calculated as the average score on individual motivators (Si) multiplied by its weight (Wi) in the following formula: Total Motivation = Wi * Si (1) Game 2 and Game 3 were considered “good “for achieving players’ experience flow. 4.8 Survey on Learners’ Engagement As suggested by [15], it is better to evaluate a game both by reviews of experts and by the players’ engagement behaviors. The engagement data on games was collected through system log file and a survey on the players. The users’ levels of engagement were measured in terms of the length of time they spent on the game and how Table 2. Empirical Results on Learners’ Engagement and Likeness Three Layered Thinking Model for Designing Web-Based Educational Games 273 frequently they played the game. The empirical results of the measurements – the frequency of play, time spent, and subjective rating of interest – are summarized in Table 2. We found that learners who played Game 1 and 3 had higher frequency lev- els than those who played Game 2. Also, while there was a high level of interest gen- erated from Game 3, this was not the case for Game 1. The empirical result is consis- tent with the results of our expert review. 5 Conclusions Designing a Web-based educational game is a complicated task. One must consider numerous factors such as rules, goals, experience flow, interaction, problem solving, challenge, conflict, storyline, etc. Much effort exerted in the designing of educational games should also be targeted at achieving the curriculum goal through relevant learning theories, contexts and learners’ characteristics. The primary intention of this paper is to present a three layered thinking model that makes designing and evaluating Web-based educational games less complicated and more effective. This model stresses the importance of focusing on the purposes in each step and their relation- ships to achieve of skill enhancement, challenge, concentration and pleasure. An empirical study was conducted to evaluate three games using the model. Cur- riculum goals can be reached using different game styles, game tasks and interfaces that produce separate results in terms of the players’ perceived challenge, concentra- tion, pleasure and developed skills. Empirical results on the players’ level of engage- ment are consistent with the expert’s evaluation results. The weight average of the four elements of game results can indeed predict the players’ engagement, while pleasure is verified to predict the subjective rating of interest. Due to the constraints imposed by the budget and the learning platform, the games in the study were only one-player games. Currently, leisure, social games are very popular online. We expect that in the future, this paper’s model can be further applied in the development of Web-based educational games with multiple users. References 1. Baumann, K., Thomas, B.: User Interface Design for Electronic Application, ch. 17. Tay- lor & Francis Inc., New York (2001) 2. Bloom, B.S.: Bloom Taxonomy of educational objectives, Allyn and Bacon. Pearson Edu- cation, Boston (1984) 3. Derntl, M., Hummel, K.A.: Modeling Context-Aware e-Learning Scenarios. In: Proceed- ings of the 3rd Int’l Conf. on Pervasive Computing and Communications Workshops (2005) 4. Foreman, J.: NEXT-Generation-education technology versus the lecture. EDUCAUSE Review, 17–30 ( July/August 2003) 5. Freitas, S.D., Olive, M.: How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? Computer & Education 46, 249–264 (2006) 6. Fu, F.L., Chen, W., Wiu, C.F.: Investigating the interest of on-line learning. In: Proceeding of E commerce and digital life conference, Taipei, Taiwan (March 2005) . problem-solving issues with regards to computer operating systems, including deleting cookies, rearranging hard disks, reinstalling software, and so forth. However, the steps involved in these. a battle against crimes in a big city. The “big city” is akin to the operating system (Window XP) in a computer. The “crimes” are problems with computer usage, such as a kidnapping of the home. because skills in strategic thinking are more difficult than skills in memory or recognition. Based on Bloom’s taxonomy of knowledge, the strategic thinking in- volved in PC components combination