Evaluating Multiple Aspects of Educational Computer Games: Literature Review and Case Study
---
paper_title: The state of the art in automating usability evaluation of user interfaces

paper_content:

Usability evaluation is an increasingly important part of the user interface design process. However, usability evaluation can be expensive in terms of time and human resources, and automation is therefore a promising way to augment existing approaches. This article presents an extensive survey of usability evaluation methods, organized according to a new taxonomy that emphasizes the role of automation. The survey analyzes existing techniques, identifies which aspects of usability evaluation automation are likely to be of use in future research, and suggests new ways to expand existing approaches to better support usability evaluation.
---
paper_title: Analysis in practical usability evaluation: a survey study

paper_content:

Analysis is a key part of conducting usability evaluations, yet rarely systematically studied. Thus, we lack direction on how to do research on supporting practitioners' analysis and lose an opportunity for practitioners to learn from each other. We have surveyed 155 usability practitioners on the analysis in their latest usability evaluation. Analysis is typically flexible and light-weight. At the same time, practitioners see a need to strengthen reliability in evaluation. Redesign is closely integrated with analysis; more than half of the respondents provide visual redesign suggestions in their evaluation deliverables. Analysis support from academic research, including tools, forms and structured formats, does not seem to have direct impact on analysis practice. We provide six recommendations for future research to better support analysis.
---
paper_title: A survey of software learnability: metrics, methodologies and guidelines

paper_content:

It is well-accepted that learnability is an important aspect of usability, yet there is little agreement as to how learnability should be defined, measured, and evaluated. In this paper, we present a survey of the previous definitions, metrics, and evaluation methodologies which have been used for software learnability. Our survey of evaluation methodologies leads us to a new question-suggestion protocol, which, in a user study, was shown to expose a significantly higher number of learnability issues in comparison to a more traditional think-aloud protocol. Based on the issues identified in our study, we present a classification system of learnability issues, and demonstrate how these categories can lead to guidelines for addressing the associated challenges.
---
paper_title: Extracting usability information from user interface events

paper_content:

Modern window-based user interface systems generate user interface events as natural products of their normal operation. Because such events can be automatically captured and because they indicate user behavior with respect to an application's user interface, they have long been regarded as a potentially fruitful source of information regarding application usage and usability. However, because user interface events are typically voluminos and rich in detail, automated support is generally required to extract information at a level of abstraction that is useful to investigators interested in analyzing application usage or evaluating usability. This survey examines computer-aided techniques used by HCI practitioners and researchers to extract usability-related information from user interface events. A framework is presented to help HCI practitioners and researchers categorize and compare the approaches that have been, or might fruitfully be, applied to this problem. Because many of the techniques in the research literature have not been evaluated in practice, this survey provides a conceptual evaluation to help identify some of the relative merits and drawbacks of the various classes of approaches. Ideas for future research in this area are also presented. This survey addresses the following questions: How might user interface events be used in evaluating usability? How are user interface events related to other forms of usability data? What are the key challenges faced by investigators wishing to exploit this data? What approaches have been brought to bear on this problem and how do they compare to one another? What are some of the important open research questions in this area?
---
paper_title: User testing of a language learning game for Mandarin Chinese

paper_content:

Polyglot Cubed is an educational game to facilitate the learning of multiple languages. The game is an implementation of contemporary theories in motivation, education and entertainment. This document provides the results from a formal user evaluation of the game. This evaluation was designed to determine user defined difficulties in game experience and understand user interest in its solution. Preliminary results indicate favorable interest in the game as a tool for learning Mandarin Chinese and minor challenges in gameplay experience.
---
paper_title: Methodology to evaluate interface of educational computer game

paper_content:

Game is an application that is popularly used in teaching and learning by some of educational institution to-date. Educational computer game (ECG) is a game being used for teaching and learning. Comprehensive methodology that consist of techniques and tools need to be developed to assist the development of ECG. ECG is an application that combines elements of fun and education and the evaluation criteria need to be able to covers all the important criteria. There are 5 heuristics identified; interface, pedagogical, content, playability and multimedia. Two techniques are identified to be used in evaluation process; PHEG for expert evaluators and PAEG for real user. The tool, online evaluation system (AHP_HeGES) is the tool that can be used to simplify the evaluation process. This research is important because it can help the game developers and educationist to conform the important elements that need to be exists in any ECG through evaluation process.
---
paper_title: Heuristic evaluation for games: usability principles for video game design

paper_content:

Most video games require constant interaction, so game designers must pay careful attention to usability issues. However, there are few formal methods for evaluating the usability of game interfaces. In this paper, we introduce a new set of heuristics that can be used to carry out usability inspections of video games. The heuristics were developed to help identify usability problems in both early and functional game prototypes. We developed the heuristics by analyzing PC game reviews from a popular gaming website, and the review set covered 108 different games and included 18 from each of 6 major game genres. We analyzed the reviews and identified twelve common classes of usability problems seen in games. We developed ten usability heuristics based on the problem categories, and they describe how common game usability problems can be avoided. A preliminary evaluation of the heuristics suggests that they help identify game-specific usability problems that can easily be overlooked otherwise.
---
paper_title: Accessibility and Usability of Educational Gaming Environments for Disabled Students

paper_content:

This tutorial discusses accessibility and usability issues for disabled students. It briefly reports on a study of disabled students and teachers in the UK, Italy and Poland and presents guidelines and recommendations.
---
paper_title: Using heuristics to evaluate the playability of games

paper_content:

Heuristics have become an accepted and widely used adjunct method of usability evaluation in Internet and software development. This report introduces Heuristic Evaluation for Playability (HEP), a comprehensive set of heuristics for playability, based on the literature on productivity and playtesting heuristics that were specifically tailored to evaluate video, computer, and board games. These heuristics were tested on an evolving game design to assess their face validity and evaluation effectiveness compared to more standard user testing methodologies. The results suggest that HEP identified qualitative similarities and differences with user testing and that HEP is best suited for evaluating general issues in the early development phases with a prototype or mock-up. Combined with user studies, HEP offers a new method for the HCI game community that can result in a more usable and playable game.
---
paper_title: Eyes-free interfaces for educational games

paper_content:

Any new technology introduced in the classroom has a potential risk of increasing the divide for students with disabilities. This impact becomes greater as the new complexity of new technologies increases. In this sense, one of the most complex technologies currently being considered are educational videogames. And while the potential benefits of videogames are great also pose significant challenges from an accessibility perspective. In this paper we investigate interfaces that may facilitate access to educational games for blind students. The long-term goal is to integrate these interfaces into educational game engines or authoring tools, facilitating the adaptation of educational games to avoid a future problem of digital divide. Three eyes-free interfaces have been developed: 1) a navigation system that allows the user to browse and interact with elements using arrow and action keys; 2) a sonar that helps blind users find interactive elements in the game universe with the mouse; and 3) an interface that interprets commands typed in natural language.
---
paper_title: Expert review method in game evaluations: comparison of two playability heuristic sets

paper_content:

The expert review method is not yet widely adopted in game evaluations, although it has been used successfully in productivity software evaluations for years. In order to use the method effectively, there need to be playability heuristics that take into account the characteristics of the videogames. There are a few playability heuristic sets available, but they have several differences, and they have not been compared to discover their strengths and weaknesses in game evaluations. In this paper, we report on a first study comparing two playability heuristic sets when evaluating the playability of a videogame. The results indicate that the heuristics can assist the evaluators in evaluating both the user interface and the gameplay aspects of the game. However, playability heuristics need to be developed further before they can be utilized by the practitioners. Especially, the clarity and comprehensibility of the heuristics need to be improved, and the optimal number of heuristics is still open.
---
paper_title: Heuristic Evaluation of Digital Game Based Learning: A Case Study

paper_content:

In the past decade, computer games have been gradually seen as one of effective teaching methods and environments, and they have been widely applied in various fields. The purpose of this case study was to investigate the usability of the computer Game, Crazy Machines 2, as a digital game-based learning material through heuristic evaluation to ascertain whether the game conform to interface design principles. Research results indicate that the highest-point indicator is the game interface which provides immediate feedback to solve related missions or problems. The second-highest is game rules which appropriately provide missions or constructive feedback icons to students, enabling them to know what is happening. The lower-point indicator is the social interaction feature, experts consider that if students have different media and the best communication tools, they will achieve the best learning effect. This study recommends future research towards adding the indicators of playfulness and joyfulness into the heuristic evaluation tools.
---
paper_title: Heuristics and Usability Guidelines for the Creation and Evaluation of Fun in Video Games

paper_content:

This study examines the implicit and explicit heuristics and usability evaluation processes utilized by a leading game developer. Five people from a single game team, each contributing in a different way to the game development process, were observed for one business day and interviewed. At the time of this study, the participating game team was at the tail end of their first month of prototyping in the pre-production phase of development. The data collected and literature reviewed combine to suggest that instituting more formal usability evaluation processes could be helpful to the game development process. The heuristics created in this study are a starting point for the construction of a standard list of game heuristics for use by the game development community.
---
paper_title: Development and potential analysis of Heuristic Evaluation for Educational Computer Game (PHEG)

paper_content:

Heuristic Evaluation has expended tremendously from general interface evaluation to more specific evaluation including for computer games application. Computer game plays an important leisure tool nowadays and the potential of it has been explored and being used in teaching and learning. Combination of evaluation method and computer game application that focus on educational purposes, lead to the realization of having comprehensive evaluation method that able to evaluate elements of fun and educational. As a result, Playability Heuristics Evaluation for Educational Computer Game (PHEG) was proposed because of the needs to address important elements especially fun element in educational computer games so that the produced application is fun to be used as well as it incorporates educational elements to help users during the learning process. The development process of PHEG involved five major steps; initial study, proposed heuristics, experts review, evaluation process and final heuristics. The discussions of this paper only focus on the fourth step which is evaluation process on PHEG. At the end of this paper, a potential analysis for PHEG using quantitative method is proposed.
---
paper_title: Guidelines of Serious Game Accessibility for the Disabled

paper_content:

Web accessibility/mobile app accessibility for the disabled has been studied for the past 10 years. The legislation of web accessibility guidelines can guarantee easy access to web contents of the disabled but not game playing, because web/ mobile app contents consists of rather simple information compared with game contents. Game contents include many PCs (player characters)/NPCs (non PCs) and the conflicts among them, as well as upgrade of a player character by completion of a quest in competition. Therefore, it is necessary to analyze and classify game accessibility so as to make the game accessibility guidelines.
---
paper_title: Playability Heuristics Evaluation (PHE) approach for Malaysian educational games

paper_content:

Heuristics evaluation is one of the common techniques being used for usability evaluation. The potential of HE has been explored in games design and development and later playability heuristics evaluation (PHE) is generated. PHE has been used in evaluating games. Issues in games usability covers forms of game usability, game interface, game mechanics, game narrative and game play. This general heuristics has the potential to be further explored in specific domain of games that is educational games. Combination of general heuristics of games (tailored based on specific domain) and education heuristics seems to be an excellent focus in order to evaluate the usability issues in educational games especially educational games produced in Malaysia.
---
paper_title: Learning by gaming - evaluation of an online game for children

paper_content:

Playing computer games is widely popular among children and teenagers as an entertainment activity; meanwhile, playing computer games also provides a learning opportunity. For example, the rules of the game have to be learned by the player in order to improve his/her performance. Based on that principle, the City eHealth Research Centre (CeRC) developed a web game for 13–15 year olds, whereby the player becomes an investigator who attends the scene of an incident that involves microbes. There are four missions in total, each involving a mystery that the player needs to solve and learning objectives that need to be taught - such as antibiotic resistance and the importance of hygiene. This paper presents the results from a game evaluation that took place between July of 2009, in four UK schools (Glasgow, Gloucester, London), with 129 students; whereby 98% of the students commented positively about playing the game. Subsequently, CeRC has improved the game and developed an interactive educational games portal (www.edugames4all.com) for different age groups of web game enthusiasts.
---
paper_title: User testing of a language learning game for Mandarin Chinese

paper_content:

Polyglot Cubed is an educational game to facilitate the learning of multiple languages. The game is an implementation of contemporary theories in motivation, education and entertainment. This document provides the results from a formal user evaluation of the game. This evaluation was designed to determine user defined difficulties in game experience and understand user interest in its solution. Preliminary results indicate favorable interest in the game as a tool for learning Mandarin Chinese and minor challenges in gameplay experience.
---
paper_title: Methodology to evaluate interface of educational computer game

paper_content:

Game is an application that is popularly used in teaching and learning by some of educational institution to-date. Educational computer game (ECG) is a game being used for teaching and learning. Comprehensive methodology that consist of techniques and tools need to be developed to assist the development of ECG. ECG is an application that combines elements of fun and education and the evaluation criteria need to be able to covers all the important criteria. There are 5 heuristics identified; interface, pedagogical, content, playability and multimedia. Two techniques are identified to be used in evaluation process; PHEG for expert evaluators and PAEG for real user. The tool, online evaluation system (AHP_HeGES) is the tool that can be used to simplify the evaluation process. This research is important because it can help the game developers and educationist to conform the important elements that need to be exists in any ECG through evaluation process.
---
paper_title: Heuristic evaluation for games: usability principles for video game design

paper_content:

Most video games require constant interaction, so game designers must pay careful attention to usability issues. However, there are few formal methods for evaluating the usability of game interfaces. In this paper, we introduce a new set of heuristics that can be used to carry out usability inspections of video games. The heuristics were developed to help identify usability problems in both early and functional game prototypes. We developed the heuristics by analyzing PC game reviews from a popular gaming website, and the review set covered 108 different games and included 18 from each of 6 major game genres. We analyzed the reviews and identified twelve common classes of usability problems seen in games. We developed ten usability heuristics based on the problem categories, and they describe how common game usability problems can be avoided. A preliminary evaluation of the heuristics suggests that they help identify game-specific usability problems that can easily be overlooked otherwise.
---
paper_title: Accessibility and Usability of Educational Gaming Environments for Disabled Students

paper_content:

This tutorial discusses accessibility and usability issues for disabled students. It briefly reports on a study of disabled students and teachers in the UK, Italy and Poland and presents guidelines and recommendations.
---
paper_title: Using heuristics to evaluate the playability of games

paper_content:

Heuristics have become an accepted and widely used adjunct method of usability evaluation in Internet and software development. This report introduces Heuristic Evaluation for Playability (HEP), a comprehensive set of heuristics for playability, based on the literature on productivity and playtesting heuristics that were specifically tailored to evaluate video, computer, and board games. These heuristics were tested on an evolving game design to assess their face validity and evaluation effectiveness compared to more standard user testing methodologies. The results suggest that HEP identified qualitative similarities and differences with user testing and that HEP is best suited for evaluating general issues in the early development phases with a prototype or mock-up. Combined with user studies, HEP offers a new method for the HCI game community that can result in a more usable and playable game.
---
paper_title: Eyes-free interfaces for educational games

paper_content:

Any new technology introduced in the classroom has a potential risk of increasing the divide for students with disabilities. This impact becomes greater as the new complexity of new technologies increases. In this sense, one of the most complex technologies currently being considered are educational videogames. And while the potential benefits of videogames are great also pose significant challenges from an accessibility perspective. In this paper we investigate interfaces that may facilitate access to educational games for blind students. The long-term goal is to integrate these interfaces into educational game engines or authoring tools, facilitating the adaptation of educational games to avoid a future problem of digital divide. Three eyes-free interfaces have been developed: 1) a navigation system that allows the user to browse and interact with elements using arrow and action keys; 2) a sonar that helps blind users find interactive elements in the game universe with the mouse; and 3) an interface that interprets commands typed in natural language.
---
paper_title: Expert review method in game evaluations: comparison of two playability heuristic sets

paper_content:

The expert review method is not yet widely adopted in game evaluations, although it has been used successfully in productivity software evaluations for years. In order to use the method effectively, there need to be playability heuristics that take into account the characteristics of the videogames. There are a few playability heuristic sets available, but they have several differences, and they have not been compared to discover their strengths and weaknesses in game evaluations. In this paper, we report on a first study comparing two playability heuristic sets when evaluating the playability of a videogame. The results indicate that the heuristics can assist the evaluators in evaluating both the user interface and the gameplay aspects of the game. However, playability heuristics need to be developed further before they can be utilized by the practitioners. Especially, the clarity and comprehensibility of the heuristics need to be improved, and the optimal number of heuristics is still open.
---
paper_title: Heuristic Evaluation of Digital Game Based Learning: A Case Study

paper_content:

In the past decade, computer games have been gradually seen as one of effective teaching methods and environments, and they have been widely applied in various fields. The purpose of this case study was to investigate the usability of the computer Game, Crazy Machines 2, as a digital game-based learning material through heuristic evaluation to ascertain whether the game conform to interface design principles. Research results indicate that the highest-point indicator is the game interface which provides immediate feedback to solve related missions or problems. The second-highest is game rules which appropriately provide missions or constructive feedback icons to students, enabling them to know what is happening. The lower-point indicator is the social interaction feature, experts consider that if students have different media and the best communication tools, they will achieve the best learning effect. This study recommends future research towards adding the indicators of playfulness and joyfulness into the heuristic evaluation tools.
---
paper_title: Development and potential analysis of Heuristic Evaluation for Educational Computer Game (PHEG)

paper_content:

Heuristic Evaluation has expended tremendously from general interface evaluation to more specific evaluation including for computer games application. Computer game plays an important leisure tool nowadays and the potential of it has been explored and being used in teaching and learning. Combination of evaluation method and computer game application that focus on educational purposes, lead to the realization of having comprehensive evaluation method that able to evaluate elements of fun and educational. As a result, Playability Heuristics Evaluation for Educational Computer Game (PHEG) was proposed because of the needs to address important elements especially fun element in educational computer games so that the produced application is fun to be used as well as it incorporates educational elements to help users during the learning process. The development process of PHEG involved five major steps; initial study, proposed heuristics, experts review, evaluation process and final heuristics. The discussions of this paper only focus on the fourth step which is evaluation process on PHEG. At the end of this paper, a potential analysis for PHEG using quantitative method is proposed.
---
paper_title: Guidelines of Serious Game Accessibility for the Disabled

paper_content:

Web accessibility/mobile app accessibility for the disabled has been studied for the past 10 years. The legislation of web accessibility guidelines can guarantee easy access to web contents of the disabled but not game playing, because web/ mobile app contents consists of rather simple information compared with game contents. Game contents include many PCs (player characters)/NPCs (non PCs) and the conflicts among them, as well as upgrade of a player character by completion of a quest in competition. Therefore, it is necessary to analyze and classify game accessibility so as to make the game accessibility guidelines.
---
paper_title: Learning by gaming - evaluation of an online game for children

paper_content:

Playing computer games is widely popular among children and teenagers as an entertainment activity; meanwhile, playing computer games also provides a learning opportunity. For example, the rules of the game have to be learned by the player in order to improve his/her performance. Based on that principle, the City eHealth Research Centre (CeRC) developed a web game for 13–15 year olds, whereby the player becomes an investigator who attends the scene of an incident that involves microbes. There are four missions in total, each involving a mystery that the player needs to solve and learning objectives that need to be taught - such as antibiotic resistance and the importance of hygiene. This paper presents the results from a game evaluation that took place between July of 2009, in four UK schools (Glasgow, Gloucester, London), with 129 students; whereby 98% of the students commented positively about playing the game. Subsequently, CeRC has improved the game and developed an interactive educational games portal (www.edugames4all.com) for different age groups of web game enthusiasts.
---
paper_title: AHP_HeGES: Tools to evaluate usability of educational computer game (UsaECG)

paper_content:

Development of computer game for teaching and learning become popular nowadays. In order to develop a game that able to contribute to the teaching and learning process and at the same time, fun to be played by the users, a comprehensive evaluation technique is needed. Experts' involvement in the evaluation process able to help developers detect usability problem before the game can be used by users. Gathering experts to conduct an evaluation process is not an easy task. This is because experts have their own commitments. To overcome this problem, a tool known as AHP_HeGES was developed to assist in evaluation process. This tool is an online system that can be used by experts to conduct the evaluation process. Evaluation technique used are based on heuristic evaluation, known as Playability Heuristic Evaluation for Educational Computer Game (PHEG) that consists of five heuristics; interface, educational element, content, playability, and multimedia. This research is important to facilitate game developers in identifying elements that need to be improved before the final version is ready to be released.
---
paper_title: Methodology to evaluate interface of educational computer game

paper_content:

Game is an application that is popularly used in teaching and learning by some of educational institution to-date. Educational computer game (ECG) is a game being used for teaching and learning. Comprehensive methodology that consist of techniques and tools need to be developed to assist the development of ECG. ECG is an application that combines elements of fun and education and the evaluation criteria need to be able to covers all the important criteria. There are 5 heuristics identified; interface, pedagogical, content, playability and multimedia. Two techniques are identified to be used in evaluation process; PHEG for expert evaluators and PAEG for real user. The tool, online evaluation system (AHP_HeGES) is the tool that can be used to simplify the evaluation process. This research is important because it can help the game developers and educationist to conform the important elements that need to be exists in any ECG through evaluation process.
---
paper_title: Quantitive analysis in a heuristic evaluation for usability of Educational Computer Game (UsaECG)

paper_content:

Usability problems found in most of heuristic evaluation process normally being presented based on mean value of severity rating. In some cases, mean value of severity rating is not accurate to determine level of usability problems found. This is because mean value will ignore the most critical problems found in specific part. One of the possible solutions to overcome this problem is by analyzing the numbers of severity rating for critical problem found. Pilot study was conducted using Playability Heuristic Evaluation for Educational Computer Game (PHEG) that being developed based on Heuristic Evaluation technique. Quantitative analysis for PHEG was use as the basis of this analysis. Numbers of subheuristics for each heuristic were used in order to calculate the percentage for each heuristic. Result shows that the most critical problem found in Educational Element and Content heuristics where there are 57.14% critical problems found for each heuristics. The least critical problem found is in Playability heuristic where the value is 18.18%. Mean value (42.42%) for the overall critical problem found in ECG can be referred as an indicator for the overall usability level of the ECG.
---
paper_title: Challenges in the evaluation of educational computer games

paper_content:

Computer games have been used for educational purposes nowadays. The implementations of educational computer games bring some challenges and issues that need to be highlighted. There are three challenges in evaluating educational computer games which are the evaluation criteria, the evaluators and the evaluation process. Evaluation criteria have been identified and it consists of five issues which are Interface, Educational, Content, Playability and Multimedia. The evaluators are real and surrogate users. The third challenge is regarding evaluation process and one of the suitable processes is by implementing formative evaluation in order to develop a comprehensive and practical educational computer games.
---
paper_title: Heuristics evaluation in computer games

paper_content:

Heuristics Evaluation is commonly used in the evaluation process of computer games and it has been expended from general evaluation to more specific computer games evaluation such as mobile games, video games and MMORPg. Developed heuristics evaluation of computer games were thoroughly analyzed based on specific criteria being used in the heuristics. As a conclusion, Playability Heuristics Evaluation for Educational Games (PHEG) were proposed to be developed because of the important elements of educational games that need to be addressed in order to develop a usable and acceptable educational games.
---
paper_title: Development and potential analysis of Heuristic Evaluation for Educational Computer Game (PHEG)

paper_content:

Heuristic Evaluation has expended tremendously from general interface evaluation to more specific evaluation including for computer games application. Computer game plays an important leisure tool nowadays and the potential of it has been explored and being used in teaching and learning. Combination of evaluation method and computer game application that focus on educational purposes, lead to the realization of having comprehensive evaluation method that able to evaluate elements of fun and educational. As a result, Playability Heuristics Evaluation for Educational Computer Game (PHEG) was proposed because of the needs to address important elements especially fun element in educational computer games so that the produced application is fun to be used as well as it incorporates educational elements to help users during the learning process. The development process of PHEG involved five major steps; initial study, proposed heuristics, experts review, evaluation process and final heuristics. The discussions of this paper only focus on the fourth step which is evaluation process on PHEG. At the end of this paper, a potential analysis for PHEG using quantitative method is proposed.
---
paper_title: Playability Heuristics Evaluation (PHE) approach for Malaysian educational games

paper_content:

Heuristics evaluation is one of the common techniques being used for usability evaluation. The potential of HE has been explored in games design and development and later playability heuristics evaluation (PHE) is generated. PHE has been used in evaluating games. Issues in games usability covers forms of game usability, game interface, game mechanics, game narrative and game play. This general heuristics has the potential to be further explored in specific domain of games that is educational games. Combination of general heuristics of games (tailored based on specific domain) and education heuristics seems to be an excellent focus in order to evaluate the usability issues in educational games especially educational games produced in Malaysia.
---
paper_title: Development and potential analysis of Heuristic Evaluation for Educational Computer Game (PHEG)

paper_content:

Heuristic Evaluation has expended tremendously from general interface evaluation to more specific evaluation including for computer games application. Computer game plays an important leisure tool nowadays and the potential of it has been explored and being used in teaching and learning. Combination of evaluation method and computer game application that focus on educational purposes, lead to the realization of having comprehensive evaluation method that able to evaluate elements of fun and educational. As a result, Playability Heuristics Evaluation for Educational Computer Game (PHEG) was proposed because of the needs to address important elements especially fun element in educational computer games so that the produced application is fun to be used as well as it incorporates educational elements to help users during the learning process. The development process of PHEG involved five major steps; initial study, proposed heuristics, experts review, evaluation process and final heuristics. The discussions of this paper only focus on the fourth step which is evaluation process on PHEG. At the end of this paper, a potential analysis for PHEG using quantitative method is proposed.
---
paper_title: Applying the Method for Evaluation of Motivational Aspects on Karucha Ships Invaders Educational Game with Brazilian Students of Japanese

paper_content:

The increasing use of educational games in several learning contexts is noticed, and to reinforce this growth, it is very important to evaluate these tools in several of their aspects. An important aspect is the influence this kind of tool exercises in the learning process. This paper presents the evaluation of motivational aspects of Karuchā Ships Invaders educational game, through a method developed earlier by the authors, demonstrating that these aspects influence positively on the user’s will to continue the learning and practice of Japanese language. The study was applied with Brazilian students of Japanese, and the results show that Karuchā Ships Invaders is a good educational tool for Japanese language learning.
---

Title: Evaluating Multiple Aspects of Educational Computer Games: Literature Review and Case Study

Section 1: Introduction
Description 1: This section introduces the main topic and objective of the paper, providing background information on the evaluation of educational computer games and the importance of such evaluations for improving user experience and educational outcomes.

Section 2: Literature Review
Description 2: This section presents a comprehensive review of existing literature on the evaluation of multiple interface aspects of educational computer games, divided into general methods and specific methods for evaluating educational games.

Section 3: Interface Evaluation in General
Description 3: This subsection details the search methodology and findings regarding general interface evaluation methods, including manual, semiautomatic, and automatic approaches, and their applicability to different contexts.

Section 4: Evaluation of Multiple Aspects of Educational Games
Description 4: This subsection focuses on methods for evaluating educational games specifically, discussing various heuristics, user-centric, and guideline-based approaches.

Section 5: Synthesis and Discussion
Description 5: This section synthesizes the reviewed literature, summarizing key findings and trends while highlighting methods that are particularly suitable for evaluating educational games.

Section 6: Case Study
Description 6: This section outlines the case study of evaluating the Karuchā Ships Invaders game, including a description of the game, the methodology used for evaluation, and the results obtained.

Section 7: Lessons Learned
Description 7: This section discusses the insights and lessons gained from conducting the evaluation, including the importance of incorporating both expert and end-user perspectives.

Section 8: Conclusions
Description 8: This section concludes the paper by summarizing the main findings, contributions, and implications of the study, and suggests potential directions for future research.