Assessment in and of Serious Games: An Overview
---
paper_title: Cognitive science implications for enhancing training effectiveness in a serious gaming context

paper_content:

Serious games use entertainment principles, creativity, and technology to meet government or corporate training objectives, but these principles alone will not guarantee that the intended learning will occur. To be effective, serious games must incorporate sound cognitive, learning, and pedagogical principles into their design and structure. In this paper, we review cognitive principles that can be applied to improve the training effectiveness in serious games and we describe a process we used to design improvements for an existing game-based training application in the domain of cyber security education.
---
paper_title: What video games have to teach us about learning and literacy

paper_content:

Good computer and video games like System Shock 2, Deus Ex, Pikmin, Rise of Nations, Neverwinter Nights, and Xenosaga: Episode 1 are learning machines. They get themselves learned and learned well, so that they get played long and hard by a great many people. This is how they and their designers survive and perpetuate themselves. If a game cannot be learned and even mastered at a certain level, it won't get played by enough people, and the company that makes it will go broke. Good learning in games is a capitalist-driven Darwinian process of selection of the fittest. Of course, game designers could have solved their learning problems by making games shorter and easier, by dumbing them down, so to speak. But most gamers don't want short and easy games. Thus, designers face and largely solve an intriguing educational dilemma, one also faced by schools and workplaces: how to get people, often young people, to learn and master something that is long and challenging--and enjoy it, to boot.
---
paper_title: Digital Game-Based Learning: It's Not Just the Digital Natives Who Are Restless.

paper_content:

After years of research and proselytizing, the proponents of digital game-based learning (DGBL) have been caught unaware. Like the person who is still yelling after the sudden cessation of loud music at a party, DGBL proponents have been shouting to be heard above the prejudice against games. But now, unexpectedly, we have everyone's attention. The combined weight of three factors has resulted in widespread public interest in games as learning tools. The first factor is the ongoing research conducted by DGBL proponents. In each decade since the advent of digital games, researchers have published dozens of essays, articles, and mainstream books on the power of DGBL—including, most recently, Marc Prensky's Digital Game-Based Learning (2001), James Paul Gee's What Video Games Have to Teach Us about Learning and Literacy (2003), Clark Aldrich's Simulations and the Future of Learning: An Innovative (and Perhaps Revolutionary) Approach to e-Learning (2004), Steven Johnson's Everything Bad Is Good for You: How Today's Popular Culture Is Actually Making Us Smarter (2005), Prensky's new book “Don’t Bother Me, Mom, I'm Learning!”: How Computer and Video Games Are Preparing Your Kids for 21st Century Success and How You Can Help! (2006), and the soon-to-be-published Games and Simulations in Online Learning: Research and Development Frameworks, edited by David Gibson, Clark Aldrich, and Marc Prensky. The second factor involves today’s “Net Generation,” or “digital natives,” who have become disengaged with traditional instruction. They require multiple streams of information, prefer inductive reasoning, want frequent and quick interactions with
---
paper_title: Uncharted waters?: exploring experts' opinions on the opportunities and limitations of serious games for foreign language learning

paper_content:

The use of serious games has seen a remarkable growth in the past decade. This resulted in a substantial number of people with hands-on experience. However, to our knowledge, no research has been performed to harvest this source of information. By means of a survey with closed and open-ended questions, we explore the opinions of 50 serious game and CALL experts on serious games' potential for foreign language learning. The first part of the paper discusses attitudes on serious games and learning. In general, we discern a rather strong belief in the potential of learning games. The second part of the paper zooms in on foreign language learning through games whereby some remarkable results emerge on the possibilities and limitations of foreign language learning games. Next, we discuss respondents' opinions on issues regarding the integration of foreign language learning games in a classroom context and on their design. The final part of the paper elaborates on a SWOT analysis of foreign language learning games resulting in a nuanced view on the opportunities and limitations of foreign language learning games. As a consequence, this paper not only identifies topics which bear a broad consensus among experts, but also shows that strong differences in opinion exist.
---
paper_title: Do Serious Games Work? Results from Three Studies

paper_content:

Three studies in higher education look at whether serious games (or video games whose primary purpose is something other than entertainment, such as military training, education, physical therapy) really do change learning outcomes.
---
paper_title: Interactivity and Reward-Related Neural Activation during a Serious Videogame

paper_content:

This study sought to determine whether playing a “serious” interactive digital game (IDG) – the Re-Mission videogame for cancer patients – activates mesolimbic neural circuits associated with incentive motivation, and if so, whether such effects stem from the participatory aspects of interactive gameplay, or from the complex sensory/perceptual engagement generated by its dynamic event-stream. Healthy undergraduates were randomized to groups in which they were scanned with functional magnetic resonance imaging (FMRI) as they either actively played Re-Mission or as they passively observed a gameplay audio-visual stream generated by a yoked active group subject. Onset of interactive game play robustly activated mesolimbic projection regions including the caudate nucleus and nucleus accumbens, as well as a subregion of the parahippocampal gyrus. During interactive gameplay, subjects showed extended activation of the thalamus, anterior insula, putamen, and motor-related regions, accompanied by decreased activation in parietal and medial prefrontal cortex. Offset of interactive gameplay activated the anterior insula and anterior cingulate. Between-group comparisons of within-subject contrasts confirmed that mesolimbic activation was significantly more pronounced in the active playgroup than in the passive exposure control group. Individual difference analyses also found the magnitude of parahippocampal activation following gameplay onset to correlate with positive attitudes toward chemotherapy assessed both at the end of the scanning session and at an unannounced one-month follow-up. These findings suggest that IDG-induced activation of reward-related mesolimbic neural circuits stems primarily from participatory engagement in gameplay (interactivity), rather than from the effects of vivid and dynamic sensory stimulation.
---
paper_title: A Review of Scholarship on Assessing Experiential Learning Effectiveness

paper_content:

This article focuses on the research and scholarship dealing with the assessment of the experiential approach in both its computer-based and human-based forms. It covers two forms of assessment. First it covers explicit attempts to assess the validity of experiential learning, and second it focuses on measuring the effectiveness of the experiential approach. The authors classify the literature on both validity and measurement into critiques and prescriptions for ideal research on one hand and empirical research studies on the other. The empirical research reviewed supports the notion that experiential learning is effective. However, the studies showing these results reflect a long-standing trend of not meeting the highest of research design and measurement standards. Thus, the authors believe any conclusion about the effectiveness of these teaching approaches must be tentative.
---
paper_title: A systematic literature review of empirical evidence on computer games and serious games

paper_content:

This paper examines the literature on computer games and serious games in regard to the potential positive impacts of gaming on users aged 14 years or above, especially with respect to learning, skill enhancement and engagement. Search terms identified 129 papers reporting empirical evidence about the impacts and outcomes of computer games and serious games with respect to learning and engagement and a multidimensional approach to categorizing games was developed. The findings revealed that playing computer games is linked to a range of perceptual, cognitive, behavioural, affective and motivational impacts and outcomes. The most frequently occurring outcomes and impacts were knowledge acquisition/content understanding and affective and motivational outcomes. The range of indicators and measures used in the included papers are discussed, together with methodological limitations and recommendations for further work in this area.
---
paper_title: A serious game model for cultural heritage

paper_content:

Serious games present a promising opportunity for learning, but the genre still lacks methodologies and tools for efficient and low-cost production, particularly for teacher and domain experts. This article gives an authoring framework that aims to provide structured support, from content design to final implementation. In particular, we have abstracted a conceptual model—the SandBox Serious Game - which relies on a generalization of task-based learning theory. The model invites players to perform cognitive tasks contextually while exploring information-rich virtual environments. We consider it particularly suited for cultural heritage entertainment applications. The model defines games that are set in realistic virtual worlds enriched with embedded educational tasks, which we have implemented as minigames. This approach simplifies the authoring work, which can easily be supported by visual authoring tools for ontology-based urban 3D modeling and implementation tasks, thus allowing an approach similar to the mind-maps concept. We propose a top-down methodology for content preparation, starting from a city-level analysis down to the single points of interest and associated tasks, which are instances of simple predefined minigame/quiz typologies. We provide examples and discuss criteria for selecting task typologies according to the authors' cognitive targets. Finally, we discuss the results of a user test, which took place in a lab, aimed at verifying the acquisition of cultural heritage knowledge in a pleasant and engaging way. Games appear particularly suited for supporting the study of images, especially of iconography. Compared to reading text, a game forces the player to focus more strongly on problems, which favors knowledge acquisition and retention. Learning complex concepts requires an investigative attitude, which can be spurred by well-designed games. Good design involves usability, graphic appeal, appropriate content, and the presence of connections which a player must discover in the content. Players should be asked to pay attention to and reason about their whole game activity - including the relationships between the game content, the brief introduction, and concluding texts. More comprehensive tests are needed to better investigate the educational effectiveness—however, the first results are promising, especially in terms of user motivation and creation of new opportunities for learning about CH.
---
paper_title: Developing Serious Games

paper_content:

Introduction Chapter 1: Historical Perspective Chapter 2: Working Context Chapter 3: Technology Trends Chapter 4: Standards Chapter 5: Best Practices Chapter 6: Tools Chapter 7: Serious Business Chapter 8: Serious Game Design Chapter 9: Outlook Appendix A: Glossary Appendix B: Acronyms Appendix C: File Formats Appendix D: Serious Games Design Document Outline Appendix E: A Brief Business Plan Primer Appendix F: Resources Index
---
paper_title: What video games have to teach us about learning and literacy

paper_content:

Good computer and video games like System Shock 2, Deus Ex, Pikmin, Rise of Nations, Neverwinter Nights, and Xenosaga: Episode 1 are learning machines. They get themselves learned and learned well, so that they get played long and hard by a great many people. This is how they and their designers survive and perpetuate themselves. If a game cannot be learned and even mastered at a certain level, it won't get played by enough people, and the company that makes it will go broke. Good learning in games is a capitalist-driven Darwinian process of selection of the fittest. Of course, game designers could have solved their learning problems by making games shorter and easier, by dumbing them down, so to speak. But most gamers don't want short and easy games. Thus, designers face and largely solve an intriguing educational dilemma, one also faced by schools and workplaces: how to get people, often young people, to learn and master something that is long and challenging--and enjoy it, to boot.
---
paper_title: The Inhibition of Socially Rejecting Information Among People with High Versus Low Self-Esteem: The Role of Attentional Bias and the Effects of Bias Reduction Training

paper_content:

In two studies, we examined the inhibition of rejection information. In Study 1, we developed a Rejection Stroop task with the purpose of measuring an attentional bias to rejection words hypothesized to characterize individuals with low self-esteem. Results indicated that people with low self-esteem experienced significantly more interference on rejection words than on acceptance words, whereas for people with high self-esteem there was no such difference. In Study 2, we developed a task to train the response of inhibiting rejection information by repeatedly identifying the smiling/accepting face in a 4 × 4 matrix of frowning faces. Results showed that after this inhibition training, people with chronic low self-esteem experienced significantly less interference on rejection words on the Rejection Stroop than their counterparts in the control condition. People with high self-esteem, on the other hand, did not exhibit different amounts of interference on rejection or acceptance words between conditions. The present findings suggest that it is possible to measure people’s attentional bias to rejection and teach people skills that help them deal with negative social information.
---
paper_title: Designing Effective Serious Games: Opportunities and Challenges for Research

paper_content:

Serious Games represent an acknowledged potential for instruction, because they are able to strongly motivate learners. They can also provide immersive environments where advanced users can practice knowledge and skills, also exploiting multimodal interaction. They can combine the effectiveness of computer processing and data storage, with high levels of attractiveness. Our work has investigated the state of the art research on SGs, starting from the cognitive aspects, that are necessary in order to root technological development and applications in sound theoretical foundations. The paper discusses some key aspects about SG design and exploitation: choice of components-off-the-shelf or from-scratch design, tools and methodologies for development or adaptation, intelligent tutoring, virtual coaches and affective learning, living worlds, game mechanics, Human-Computer Interaction. ::: While several SGs have been developed, still the literature stresses a lack of significant, extensive user tests. Further research is necessary to investigate in greater detail the real effectiveness of the various types of SGs. The paper proposes several research questions - that range from requirements elicitation to design and from deployment to use and evaluation - to be answered in order to avoid technology pushing and drive technological research according to the requirements of the end-users and stakeholders. ::: We believe that deepening the analysis about these issues is key to strengthen the foundations of SG research, for which we identify four major directions: definition of metrics and learning progress evaluation tools; methodologies and tools for designing games from various topics and for various users; computing and communication architectures; technologies that can enhance the overall system performance.
---
paper_title: A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial

paper_content:

OBJECTIVE ::: Suboptimal adherence to self-administered medications is a common problem. The purpose of this study was to determine the effectiveness of a video-game intervention for improving adherence and other behavioral outcomes for adolescents and young adults with malignancies including acute leukemia, lymphoma, and soft-tissue sarcoma. ::: ::: ::: METHODS ::: A randomized trial with baseline and 1- and 3-month assessments was conducted from 2004 to 2005 at 34 medical centers in the United States, Canada, and Australia. A total of 375 male and female patients who were 13 to 29 years old, had an initial or relapse diagnosis of a malignancy, and currently undergoing treatment and expected to continue treatment for at least 4 months from baseline assessment were randomly assigned to the intervention or control group. The intervention was a video game that addressed issues of cancer treatment and care for teenagers and young adults. Outcome measures included adherence, self-efficacy, knowledge, control, stress, and quality of life. For patients who were prescribed prophylactic antibiotics, adherence to trimethoprim-sulfamethoxazole was tracked by electronic pill-monitoring devices (n = 200). Adherence to 6-mercaptopurine was assessed through serum metabolite assays (n = 54). ::: ::: ::: RESULTS ::: Adherence to trimethoprim-sulfamethoxazole and 6-mercaptopurine was greater in the intervention group. Self-efficacy and knowledge also increased in the intervention group compared with the control group. The intervention did not affect self-report measures of adherence, stress, control, or quality of life. ::: ::: ::: CONCLUSIONS ::: The video-game intervention significantly improved treatment adherence and indicators of cancer-related self-efficacy and knowledge in adolescents and young adults who were undergoing cancer therapy. The findings support current efforts to develop effective video-game interventions for education and training in health care.
---
paper_title: A Review of Scholarship on Assessing Experiential Learning Effectiveness

paper_content:

This article focuses on the research and scholarship dealing with the assessment of the experiential approach in both its computer-based and human-based forms. It covers two forms of assessment. First it covers explicit attempts to assess the validity of experiential learning, and second it focuses on measuring the effectiveness of the experiential approach. The authors classify the literature on both validity and measurement into critiques and prescriptions for ideal research on one hand and empirical research studies on the other. The empirical research reviewed supports the notion that experiential learning is effective. However, the studies showing these results reflect a long-standing trend of not meeting the highest of research design and measurement standards. Thus, the authors believe any conclusion about the effectiveness of these teaching approaches must be tentative.
---
paper_title: The Guide to Computer Simulations and Games

paper_content:

The first computer simulation book for anyone designing or building a gameAnswering the growing demand for a book catered for those who design, develop, or use simulations and games this book teaches you exactly what you need to know in order to understand the simulations you build or use all without having to earn another degree. Organized into three parts, this informative book first defines computer simulations and describes how they are different from live-action and paper-based simulations.The second section builds upon the previous, with coverage of the technical details of simulations, a detailed description of how models are built, and an explanation of how those models are translated into simulations. Finally, the last section develops four examples that walk you through the process from model to finished and functional simulation, all of which are created using freely available software and all of which can be downloaded.Targets anyone interested in learning about the inner workings of a simulation or game, but may not necessarily be a programmer or scientistOffers technical details on what simulations are and how they are built without overwhelming you with intricate jargonBreaks down simulation vs. modeling and traditional vs. computer simulationsExamines verification and validation and discusses simulation toolsWhether you need to learn how simulations work or it's something you've always been curious about but couldn't find the right resource, look no further. The Guide to Computer Simulations and Games is the ideal book for getting a solid understanding of this fascinating subject.
---
paper_title: ASSESSING WHAT PLAYERS LEARNED IN SERIOUS GAMES: IN SITU DATA COLLECTION, INFORMATION TRAILS, AND QUANTITATIVE ANALYSIS

paper_content:

What do students really learn as they play videogames for learning? Are they learning the content presented to them, or merely how to play the game? Educators want serious games that “inform, monitor, assess and appraise” students throughout the games and scientific evidence verifying the process. Likewise, policy-makers often require rigorous, large-scale empirical studies to help them determine if new technology, such as the serious games, could be effective in practice.
---
paper_title: The Concept of Formative Assessment.

paper_content:

While many educators are highly focused on state tests, it is important to consider that over the course of a year, teachers can build in many opportunities to assess how students are learning and then use this information to make beneficial changes in instruction. This diagnostic use of assessment to provide feedback to teachers and students over the course of instruction is called formative assessment. It stands in contrast to summative assessment, which generally takes place after a period of instruction and requires making a judgment about the learning that has occurred (e.g., by grading or scoring a test or paper). This article addresses the benefits of formative assessment and provides examples and resources to support its implementation.
---
paper_title: Analysis of Pre‐test‐Post‐test Control Group Designs in Educational Research

paper_content:

Abstract Pre‐test‐post‐test control group designs are well suited to investigating effects of educational innovations and are common in educational research. They are frequently analysed by means of an ANOVA on change scores, or, what amounts to the same thing, a repeated measures ANOVA to test the treatment by occasion interaction. Although the analysis of change scores has intuitive appeal, such analyses are often inappropriate. An ANCOVA on post‐test scores, with pre‐test scores as co‐variate usually provides a more appropriate and informative analysis. Advantages of the ANCOVA approach are explained and illustrated using SPSS‐X analyses of fabricated data sets.
---
paper_title: Innovation Diffusion: Assessment of Strategies Within the DIFFUSION SIMULATION GAME

paper_content:

Educators increasingly view the high level of engagement and experiential learning offered by games as a means to promote learning. However, as with any designed learning experience, player experiences should provide an accurate representation of content to be learned. In this study, the authors investigated the DIFFUSION SIMULATION GAME (DSG) to assess the consistency of strategies effective in the game with those implied to be effective by the diffusion of innovations theory on which the game is based. They analyzed records from 2,361 completed game sessions of the DSG and compared successful and unsuccessful strategies. They further compared successful gameplay strategies with strategies suggested by the diffusion of innovations theory. The data analysis indicated that four of the seven winning strategies were inconsistent with what the theory predicts. However, this conclusion is tentative, given limitations of temporal detail in available data. These limitations imply how data should be collected to better investigate strategies that result in successful DSG gameplay. In addition, the study provides a case in which objective methods were used to analyze patterns of gameplay and offers insight on how data should be collected to analyze patterns more effectively.
---
paper_title: GameFlow: a model for evaluating player enjoyment in games

paper_content:

Although player enjoyment is central to computer games, there is currently no accepted model of player enjoyment in games. There are many heuristics in the literature, based on elements such as the game interface, mechanics, gameplay, and narrative. However, there is a need to integrate these heuristics into a validated model that can be used to design, evaluate, and understand enjoyment in games. We have drawn together the various heuristics into a concise model of enjoyment in games that is structured by flow. Flow, a widely accepted model of enjoyment, includes eight elements that, we found, encompass the various heuristics from the literature. Our new model, GameFlow, consists of eight elements -- concentration, challenge, skills, control, clear goals, feedback, immersion, and social interaction. Each element includes a set of criteria for achieving enjoyment in games. An initial investigation and validation of the GameFlow model was carried out by conducting expert reviews of two real-time strategy games, one high-rating and one low-rating, using the GameFlow criteria. The result was a deeper understanding of enjoyment in real-time strategy games and the identification of the strengths and weaknesses of the GameFlow model as an evaluation tool. The GameFlow criteria were able to successfully distinguish between the high-rated and low-rated games and identify why one succeeded and the other failed. We concluded that the GameFlow model can be used in its current form to review games; further work will provide tools for designing and evaluating enjoyment in games.
---
paper_title: Toward an understanding of flow in video games

paper_content:

In the domain of computer games, research into the interaction between player and game has centred on 'enjoyment', often drawing in particular on optimal experience research and Csikszentmihalyi's 'Flow theory'. Flow is a well-established construct for examining experience in any setting and its application to game-play is intuitive. Nevertheless, it's not immediately obvious how to translate between the flow construct and an operative description of game-play. Previous research has attempted this translation through analogy. In this article we propose a practical, integrated approach for analysis of the mechanics and aesthetics of game-play, which helps develop deeper insights into the capacity for flow within games. The relationship between player and game, characterized by learning and enjoyment, is central to our analysis. We begin by framing that relationship within Cowley's user-system-experience (USE) model, and expand this into an information systems framework, which enables a practical mapping of flow onto game-play. We believe this approach enhances our understanding of a player's interaction with a game and provides useful insights for games' researchers seeking to devise mechanisms to adapt game-play to individual players.
---
paper_title: Affective Ludology : Scientific Measurement of User Experience in Interactive Entertainment

paper_content:

Digital games provide the most engaging interactive experiences. Researching gameplay experience is done mainly in the science and technology (e.g., human-computer interaction, physiological and en ...
---
paper_title: EGameFlow : A scale to measure learners ’ enjoyment of e-learning games

paper_content:

In an effective e-learning game, the learner's enjoyment acts as a catalyst to encourage his/her learning initiative. Therefore, the availability of a scale that effectively measures the enjoyment offered by e-learning games assist the game designer to understanding the strength and flaw of the game efficiently from the learner's points of view. E-learning games are aimed at the achievement of learning objectives via the creation of a flow effect. Thus, this study is based on Sweetser's & Wyeth's framework to develop a more rigorous scale that assesses user enjoyment of e-learning games. The scale developed in the present study consists of eight dimensions: Immersion, social interaction, challenge, goal clarity, feedback, concentration, control, and knowledge improvement. Four learning games employed in a university's online learning course ''Introduction to Software Application'' were used as the instruments of scale verification. Survey questionnaires were distributed to students taking the course and 166 valid samples were subsequently collected. The results showed that the validity and reliability of the scale, EGameFlow, were satisfactory. Thus, the measurement is an effective tool for evaluating the level of enjoyment provided by e-learning games to their users.
---
paper_title: Flow: The Psychology of Optimal Experience

paper_content:

This article presents a review of the book “Flow: The Psychology of Optimal Experience,” by Michael Csikszentmihalyi.
---
paper_title: Measuring the experience of digital game enjoyment

paper_content:

Wijnand IJsselsteijn, Wouter van den Hoogen, Christoph Klimmt, Yvonne de Kort, Craig Lindley, Klaus Mathiak, Karolien Poels, Niklas Ravaja, Marko Turpeinen, and Peter Vorderer 1 Eindhoven University of Technology, Eindhoven, The Netherlands, w.a.ijsselsteijn.tue.nl 2 Hochschule fur Musik und Theater, Hannover, Germany 3 Blekinge Institute of Technology, Karlshamn, Sweden 4 RWTH Aachen University, Aachen, Germany 5 CKIR, Helsinki School of Economics, Helsinki, Finland 6 Helsinki Institute for Information Technology, Helsinki, Finland 7 CAMeRA, Free University of Amsterdam, Amsterdam, The Netherlands
---
paper_title: The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing

paper_content:

Deep engagement in video game-playing has the potential to be to be one important determinant of the impact of playing violent video games, but there are currently no reliable measures of this subjective experience. To fill this gap, the Game Engagement Questionnaire (GEQ) was developed using both classical and Rasch analyses. In Study 1 Rasch analyses provide support for the reliability and functionality of the GEQ scores. Rasch analyses also demonstrate that the GEQ has adequate separation, fit, rating scale functioning, and dimensionality, suggesting that one’s tendency to become engaged in video game-playing is a quantifiable construct. In Study 2, behavioral and questionnaire data supported the reliability and validity of the GEQ for predicting engagement in violent video games. The GEQ provides a psychometrically strong measure of levels of engagement specifically elicited while playing video games, and thus shows promise for future research examining risk and protective factors for negative game impact.
---
paper_title: Oscillatory Brain Responses Evoked by Video Game Events: The Case of Super Monkey Ball 2

paper_content:

Electroencephalography (EEG) was recorded while the subjects played a video game where the player had to steer a monkey into a goal while collecting bananas for extra points and had to avoid falling off the edge of the game board. Each of these three studied events evoked differential EEG oscillatory changes. Picking up bananas evoked decreased theta activation on central electrodes, decreased high alpha activation on frontal electrodes, and increased beta activation on frontal electrodes. Falling off the game board evoked decreased central theta activation and increased fronto-central beta activation. Reaching the goal evoked increased theta activation on parietal electrodes, increased low alpha activation on frontal electrodes, increased high alpha activation on frontal, central, and parietal electrodes, and increased beta activation on frontal and central electrodes. It is suggested that the EEG oscillations evoked by picking up bananas reflect increased cortical activation and arousal; the oscillation...
---
paper_title: Heart rate asymmetry and emotional response to robot-assist task challenges in post-stroke patients

paper_content:

The level of motivation or stress influences learning the use of a robot-assist device for walking. Heart rate asymmetry (HRA) indicates the level of parasympathetic (HRA 0.5) involvement in heart rate regulation. Three patients and seven controls were presented increasing levels of task difficulty. During training patients showed an increase in stress as indicated by the HRA index (0.524±0.02) in contrast to control participants (0.485±0.03). As the task complexity increased, the HRA in the patient group was atypical and falling below 0.5, compared to control (HRA>0.5). The latter result reflects an increased cognitive involvement and a higher sympathetic predominance in accordance with an increased task difficulty. Thus stroke affected the response to the task challenges in that the patient response to increasing task challenge leads to an inversion of HRA associated with a decreased mental engagement and higher risk of sudden cardiac death
---
paper_title: Innovation Diffusion: Assessment of Strategies Within the DIFFUSION SIMULATION GAME

paper_content:

Educators increasingly view the high level of engagement and experiential learning offered by games as a means to promote learning. However, as with any designed learning experience, player experiences should provide an accurate representation of content to be learned. In this study, the authors investigated the DIFFUSION SIMULATION GAME (DSG) to assess the consistency of strategies effective in the game with those implied to be effective by the diffusion of innovations theory on which the game is based. They analyzed records from 2,361 completed game sessions of the DSG and compared successful and unsuccessful strategies. They further compared successful gameplay strategies with strategies suggested by the diffusion of innovations theory. The data analysis indicated that four of the seven winning strategies were inconsistent with what the theory predicts. However, this conclusion is tentative, given limitations of temporal detail in available data. These limitations imply how data should be collected to better investigate strategies that result in successful DSG gameplay. In addition, the study provides a case in which objective methods were used to analyze patterns of gameplay and offers insight on how data should be collected to analyze patterns more effectively.
---
paper_title: GameFlow: a model for evaluating player enjoyment in games

paper_content:

Although player enjoyment is central to computer games, there is currently no accepted model of player enjoyment in games. There are many heuristics in the literature, based on elements such as the game interface, mechanics, gameplay, and narrative. However, there is a need to integrate these heuristics into a validated model that can be used to design, evaluate, and understand enjoyment in games. We have drawn together the various heuristics into a concise model of enjoyment in games that is structured by flow. Flow, a widely accepted model of enjoyment, includes eight elements that, we found, encompass the various heuristics from the literature. Our new model, GameFlow, consists of eight elements -- concentration, challenge, skills, control, clear goals, feedback, immersion, and social interaction. Each element includes a set of criteria for achieving enjoyment in games. An initial investigation and validation of the GameFlow model was carried out by conducting expert reviews of two real-time strategy games, one high-rating and one low-rating, using the GameFlow criteria. The result was a deeper understanding of enjoyment in real-time strategy games and the identification of the strengths and weaknesses of the GameFlow model as an evaluation tool. The GameFlow criteria were able to successfully distinguish between the high-rated and low-rated games and identify why one succeeded and the other failed. We concluded that the GameFlow model can be used in its current form to review games; further work will provide tools for designing and evaluating enjoyment in games.
---
paper_title: Toward an understanding of flow in video games

paper_content:

In the domain of computer games, research into the interaction between player and game has centred on 'enjoyment', often drawing in particular on optimal experience research and Csikszentmihalyi's 'Flow theory'. Flow is a well-established construct for examining experience in any setting and its application to game-play is intuitive. Nevertheless, it's not immediately obvious how to translate between the flow construct and an operative description of game-play. Previous research has attempted this translation through analogy. In this article we propose a practical, integrated approach for analysis of the mechanics and aesthetics of game-play, which helps develop deeper insights into the capacity for flow within games. The relationship between player and game, characterized by learning and enjoyment, is central to our analysis. We begin by framing that relationship within Cowley's user-system-experience (USE) model, and expand this into an information systems framework, which enables a practical mapping of flow onto game-play. We believe this approach enhances our understanding of a player's interaction with a game and provides useful insights for games' researchers seeking to devise mechanisms to adapt game-play to individual players.
---
paper_title: A Metric for Entertainment of Boardgames: Its Implication for Evolution of Chess Variants

paper_content:

In this paper we investigate the evolutionary changes of chess variants. For this purpose we propose a measure of the game’s entertaining impact. The measure is derived from grandmaster games and is applicable to chess variants independent of the game under consideration. For the case where no grandmaster games are available we perform computer self-play experiments to estimate the measure. For several old chess variants the measure as well as other measures, such as the search-space complexity and draw rate, are calculated. Some relationships between the measures investigated are found. Based on these relationships the evolutionary changes of chess variants are discussed.
---
paper_title: Flow: The Psychology of Optimal Experience

paper_content:

This article presents a review of the book “Flow: The Psychology of Optimal Experience,” by Michael Csikszentmihalyi.
---
paper_title: Enhancing the educational value of video games

paper_content:

Lowering the barrier between education and real entertainment is an important challenge in order to better exploit the potential of computers and reach a demographic that is traditionally averse to learning. To this end, it is important to investigate how to exploit the appeal of video games (VGs) to also favor and induce learning via playing video games. Achieving this goal is not only a matter of content, since simply “superimposed” educational content risks being perceived as boring. Hence we believe that the game should feature mechanisms for acquiring knowledge and skill that are smoothly embedded in a meaningful, homogeneous, and compelling whole. Thus, there is a need to compartmentalize components of a game engine so that it becomes easy and efficient to integrate the graphics/interface—which has already been done very well by state-of-the-art successful video games and the educational aspect which is typically poor in those same games. Hence we have defined a general set of mechanisms and modules that can be inserted in state-of-the-art VG environments and are aimed at promoting various kinds of knowledge and procedural skill acquisition. In order to investigate and validate this concept, we have built an educational game, SeaGame, using a state-of-the-art commercial game development approach, and enriched the environment with instances of developed educational modules. Analyzing user test results, we conclude that SeaGame is perceived quite similarly to commercial VGs, which suggests that the proposed mechanisms do not compromise the overall enjoyability of the game, which is key to attracting a wide demographic that is not currently involved in educational activities during their leisure time. The results of this research can be generalized, since the standards of commercial games and the proposed educational enhancements can be instantiated in a variety of educational contexts and applied to different types of content.
---
paper_title: Capturing Design Space From a User Perspective : The Repertory Grid Technique Revisited

paper_content:

The design of an artifact (e.g., software system, household appliance) requires a multitude of decisions. In the course of narrowing down the design process, "good ideas" have to be divided from "bad ideas." To accomplish this, user perceptions and evaluations are of great value. The individual way people perceive and evaluate a set of prototypes designed in parallel may shed light on their general needs and concerns. The Repertory Grid Technique (RGT) is a method of elucidating the so-called personal constructs (e.g., friendly-hostile, bad-good, playful-expert-like) people employ when confronted with other individuals, events, or artifacts. We assume that the personal constructs (and the underlying topics) generated as a reaction to a set of artifacts mark the artifacts' design space from a user's perspective and that this information may be helpful in separating valuable ideas from the not so valuable. This article explores the practical value of the RGT in gathering design-relevant information about th...
---
paper_title: Evolving Opponents for Interesting Interactive Computer Games

paper_content:

In this paper we introduce experiments on neuro-evolution mechanisms applied to predator/prey multi-character computer games. Our test-bed is a modified version of the well-known Pac-Man game. By viewing the game from the predators’ (i.e. opponents’) perspective, we attempt off-line to evolve neural-controlled opponents capable of playing effectively against computer-guided fixed strategy players. However, emergent near-optimal behaviors make the game less interesting to play. We therefore discuss the criteria that make a game interesting and, furthermore, we introduce a generic measure of predator/prey computer games’ interest. Given this measure, we present an evolutionary mechanism for opponents that keep learning from a player while playing against it (i.e. on-line) and we demonstrate its efficiency and robustness in increasing and maintaining the game’s interest. Computer game opponents following this on-line learning approach show high adaptability to changing player strategies which provides evidence for the approach’s effectiveness against human players.
---
paper_title: Technology-Enhanced Formative Assessment: A Research-Based Pedagogy for Teaching Science with Classroom Response Technology

paper_content:

Classroom response systems (CRSs) are a promising instructional technology, but most literature on CRS use fails to distinguish between technology and pedagogy, to define and justify a pedagogical perspective, or to discriminate between pedagogies. Technology-enhanced formative assessment (TEFA) is our pedagogy for CRS-based science instruction, informed by experience and by several traditions of educational research. In TEFA, four principles enjoin the practice of question-driven instruction, dialogical discourse, formative assessment, and meta-level communication. These are enacted via the question cycle, an iterative pattern of CRS-based questioning that can serve multiple instructional needs. TEFA should improve CRS use and help teachers “bridge the gap” between educational research findings and practical, flexible classroom strategies for science instruction.
---
paper_title: Web-based self-assessments in pathology with Questionmark Perception.

paper_content:

Formative assessment with appropriate feedback is an effective method of promoting learning. Software tools are now available to facilitate the delivery of formative self-assessments via the World Wide Web. For the past 5 years, we have offered on-line formative self-assessments in pathology to undergraduate medical students. These assessments, which have been authored using Questionmark Perception, have proved particularly popular and have received high ratings in course evaluation surveys. Furthermore, they appear to have been effective in promoting student learning, with evidence that students made a genuine attempt to answer the questions in the assessment and that they learnt from doing so. The potential of on-line formative self-assessments for postgraduate training and continuing education in pathology so far remains untapped.
---
paper_title: The automatic assessment of free text answers using a modified BLEU algorithm

paper_content:

e-Learning plays an undoubtedly important role in today's education and assessment is one of the most essential parts of any instruction-based learning process. Assessment is a common way to evaluate a student's knowledge regarding the concepts related to learning objectives. In this paper, a new method for assessing the free text answers of students based on the BLEU algorithm is presented. We modify the BLEU algorithm so that it is suitable for assessing free text answers and call the new algorithm the modified BLEU (M-BLEU). To perform an assessment, it is necessary to establish a repository of reference answers written by course instructors or related experts. Several reference answers are included for each question. The M-BLEU algorithm is used to identify the most similar reference answer to a student answer; a similarity score is calculated and applied to score the answers provided by students. Evaluation results show that the proposed method achieves the highest correlation with human expert scores compared to other assessment methods such as latent semantic analysis (LSA) and n-gram co-occurrence.
---
paper_title: The Use of Question Mark Software for Formative and Summative Assessment in Two Universities

paper_content:

SUMMARY This paper describes two methods of introducing computer‐assisted assessment (CAA) using Question Mark software within two universities. Both the University of Luton and Loughborough University have been using CAA to assess student learning. The University of Luton uses CAA for summative assessment and finds that the range of skills and abilities being tested can be extended when an appropriate balance of assessment methods is used. It may also release academic staff time when effectively implemented. Loughborough University uses CAA mainly for formative assessment to enable students to reflect critically on their performance though the provision of instant feedback. Both universities have set up central units to provide support and development for staff wishing to use CAA in their teaching. Students at each university react favourably to each method. The differences between methods is discussed.
---
paper_title: An evaluation of undergraduate students' online assessment performances

paper_content:

Irish universities are adopting eLearning technologies to improve the quality of learning for third level students, across a variety of disciplines. As the technology to support blended learning is becoming more stable, there is an emerging awareness that the key source of value for students is derived from the pedagogic application of that technology rather than the technology itself. This research examines the quality of students' learning as measured by assessment performances in a hybrid technology-supported education model. Tests are conducted on the efficacy of this model by examining undergraduate accounting students' online assessment performances in comparison to similar off-line assessments. Students' performances are then related to the frequency/time spent online and the student's preferred learning style. We conclude that the introduction of eAssessments does improve the offine performances of undergraduate accounting students. However, these improvements are not universally observed in this heterogeneous group. Therefore, we recommend that in order to harness the benefits of learning technologies, pedagogic consideration should be given to the choice, variety, and level of difficulty of eAssessments offered to students.
---
paper_title: Can online course-based assessment methods be fair and equitable? Relationships between students' preferences and performance within online and offline assessments

paper_content:

To address concerns raised regarding the use of online course-based summative assessment methods, a quasi-experimental design was implemented in which students who completed a summative assessment either online or offline were compared on performance scores when using their self-reported preferred or non-preferred modes. Performance scores were found not to differ depending on whether the assessment was completed in the preferred or non-preferred mode. These findings provide preliminary support for the validity of online assessment methods. Future studies could help determine the extent to which this finding generalizes beyond the assessment procedures and type of sample used here. Suggestions for follow-up studies are offered, including exploring the validity of more complex computer-related online assessment tasks and investigating the impact of using preferred and non-preferred modes upon the quality of the student experience. © 2012 Wiley Periodicals, Inc.
---
paper_title: Improving Student Performance Using Self-Assessment Tests

paper_content:

Testing is the most generic and perhaps most widely used mechanism for student assessment. Most tests are based on the classical test theory, which says that a student's score is the sum of the scores obtained in all questions plus some kind of error. The most relevant is that the student test result depends heavily on the individual's learning preferences or abilities and also on the actual test's format. According to this theory, tests aren't necessarily useful in intelligent educational systems, which require accurately obtaining the student's knowledge state to guide the learning process. Yet the Web has created a new generation of intelligent systems-adaptive hypermedia systems which offer new types of instructional interaction. Educational AHSs adapt the learning process on the basis of the student's learning preferences, knowledge, and availability. One such Web-based tool is Siette (the system of intelligent evaluation using rests), which infers student knowledge using adaptive testing.
---
paper_title: Assessment for learning ? The potential of short-answer free-text questions with tailored feedback

paper_content:

A natural language based system has been used to author and mark short-answer free-text assessment tasks. Students attempt the questions online and are given tailored and relatively detailed feedback on incorrect and incomplete responses, and have the opportunity to repeat the task immediately so as to learn from the feedback provided. The answer matching has been developed in the light of student responses to the questions. A small number of the questions are now in low-stakes summative use, alongside other e-assessment tasks and tutor-marked assignments, to give students instantaneous feedback on constructed response items, to help them to monitor their progress and to encourage dialogue with their tutor. The answer matching has been demonstrated to be of similar or greater accuracy than specialist human markers. Students have been observed attempting the questions and have been seen to respond in differing ways to both the questions themselves and the feedback provided. We discuss features of appropriate items for assessment of this type.
---
paper_title: Classroom Response Systems: A Review of the Literature

paper_content:

As the frequency with which Classroom Response Systems (CRSs) are used is increasing, it becomes more and more important to define the affordances and limitations of these tools. Currently existing literature is largely either anecdotal or focuses on comparing CRS and non-CRS environments that are unequal in other aspects as well. In addition, the literature primarily describes situations in which the CRS is used to provide an individual as opposed to a group response. This article points to the need for a concerted research effort, one that rigorously explores conditions of use across diverse settings and pedagogies.
---
paper_title: The C3 Framework: Evaluating Classroom Response System Interactions in University Classrooms

paper_content:

The larger the classroom, the more likely is it that communications consist of a one-way flow from the instructor to students. Classroom Response Systems (CRSs) are frequently hailed as technologies capable of improving communications by opening the space for dialogic engagement; yet, a causal relationship is not documented in the literature. The data reported on here stem from a mixed methodology study and provide insights into motivations for CRS use and enacted CRS use across disciplines, as well as student and instructor perceptions of the tool’s effects on teaching and learning. From these data emerged a framework of interaction (the C3 Framework) that situates CRS use from both the instructors’ and learners’ perspectives. The framework consists of an interdependent relationship between Concerns, Centeredness, and Control of discourse. Although this study took place in university classrooms, the C3 Framework presented here applies across educational settings.
---
paper_title: Dynamic Game Difficulty Scaling Using Adaptive Behavior-Based AI

paper_content:

Games are played by a wide variety of audiences. Different individuals will play with different gaming styles and employ different strategic approaches. This often involves interacting with nonplayer characters that are controlled by the game AI. From a developer's standpoint, it is important to design a game AI that is able to satisfy the variety of players that will interact with the game. Thus, an adaptive game AI that can scale the difficulty of the game according to the proficiency of the player has greater potential to customize a personalized and entertaining game experience compared to a static game AI. In particular, dynamic game difficulty scaling refers to the use of an adaptive game AI that performs game adaptations in real time during the game session. This paper presents two adaptive algorithms that use ideas from reinforcement learning and evolutionary computation to improve player satisfaction by scaling the difficulty of the game AI while the game is being played. The effects of varying the learning and mutation rates are examined and a general rule of thumb for the parameters is proposed. The proposed algorithms are demonstrated to be capable of matching its opponents in terms of mean scores and winning percentages. Both algorithms are able to generalize well to a variety of opponents.
---
paper_title: The Guide to Computer Simulations and Games

paper_content:

The first computer simulation book for anyone designing or building a gameAnswering the growing demand for a book catered for those who design, develop, or use simulations and games this book teaches you exactly what you need to know in order to understand the simulations you build or use all without having to earn another degree. Organized into three parts, this informative book first defines computer simulations and describes how they are different from live-action and paper-based simulations.The second section builds upon the previous, with coverage of the technical details of simulations, a detailed description of how models are built, and an explanation of how those models are translated into simulations. Finally, the last section develops four examples that walk you through the process from model to finished and functional simulation, all of which are created using freely available software and all of which can be downloaded.Targets anyone interested in learning about the inner workings of a simulation or game, but may not necessarily be a programmer or scientistOffers technical details on what simulations are and how they are built without overwhelming you with intricate jargonBreaks down simulation vs. modeling and traditional vs. computer simulationsExamines verification and validation and discusses simulation toolsWhether you need to learn how simulations work or it's something you've always been curious about but couldn't find the right resource, look no further. The Guide to Computer Simulations and Games is the ideal book for getting a solid understanding of this fascinating subject.
---
paper_title: CancerSpace: A Simulation-Based Game for Improving Cancer-Screening Rates

paper_content:

Can playing a video game help save lives? It might seem unlikely, especially if the first image that comes to mind at the word "gamer" is that of an overstimulated, underdeveloped teenager who desperately needs a suntan and a girlfriend. However, like many of today's gamers, video games are moving out of their parents' basement and into the real world. Video games are maturing and facing adaptation to serious applications. Such serious games are seeing use in a variety of fields, from the military to corporate management, and are finally being employed in healthcare. One of the biggest, most challenging areas is modeling simulations for medical training, particularly for managing chronic illness and providing system-level population-based care.
---
paper_title: Serious Games for Immersive Cultural Training: Creating a Living World

paper_content:

Living worlds offer a nonlinear, unscripted process for experiencing and safely learning the cognitive complexity and nuance of culture through emergent high-fidelity simulation. The 3D asymmetric domain analysis and training model uses visual, auditory, behavioral, and cultural models for immersive cultural training using the living-world construct.
---
paper_title: Assessing the Thinking Curriculum: New Tools for Educational Reform

paper_content:

In America, educational reform and testing are intimately linked. Test scores signal the need for reform, as evidenced by the attention paid to declining scores on college entrance exams and standardized tests, to Americans’ weak performances on international comparisons, and to the percentages of students failing certain kinds of items on our national assessments. Tests are also widely viewed as instruments for educational improvement. Calls for better performance by American schools are almost always accompanied by increases in the amounts of testing done in the schools. New tests, or more active scrutiny of tests already in place, are frequently prescribed, both as a source of information for a concerned public and as a form of “quality control” and an incentive to better performance by educators and students. This link between testing and efforts at educational reform is not new—it has been a feature of efforts to improve American schools since at least the end of the nineteenth century (D. P. Resnick 1982). In each new round of reform, testing theory and practice have been refined and elaborated. Tests are so ubiquitous in this country’s educational life, however, and the test instruments we use are often so technically elegant, that it is difficult to imagine proceeding in a different way. Complaints about testing and tests, from those who claim that tests block opportunities for certain social groups and those who point to the limited range of human competence assessed by the tests, bubble up whenever the amount and visibility of testing increase. These complaints sometimes lead to modifications of tests, but there is rarely sustained or widespread consideration of the possibility that the very idea of using test technology as it has developed over the past century may be inimical to the real goals of educational reform.
---
paper_title: Assessing Preschool Children's Pretend Play: Preliminary Validation of the Affect in Play Scale-Preschool Version

paper_content:

Research Findings: A description of the development and preliminary validation of the Affect in Play Scale-Preschool version (APS-P) is presented by demonstrating associations among preschool children's play, creativity, and daily behavior using multiple methodologies. Thirty-three preschool-age children completed a standardized 5-minute play task and a measure of creativity. Teachers answered questions about children's daily play behavior, affective expression, social competence, and adjustment. Good interrater reliability and internal consistency were demonstrated for the APS-P. Affect expression and fantasy in 5 minutes of play were related to theoretically and empirically relevant constructs such as daily play behavior, creativity, and social competence. Practice or Policy: Results support the use of the APS-P as a potentially valuable tool for assessing developments and changes in both affective and cognitive dimensions of preschool children's play.
---
paper_title: Design and evaluation of a Serious Game for immersive cultural training

paper_content:

3D Virtual Worlds have the potential to expand the knowledge of a foreign culture by presenting the information in a visual context. In this paper we describe the design and evaluation of ICURA, a Serious Game that provides an opportunity to learn more about Japanese culture and etiquette. ICURA incorporates different game design and learning principles to effectively communicate knowledge. The main goal is to combine aspects of learning and fun in an immersive 3D Environment to make the communication of knowledge an entertaining experience. We evaluated the game through a user study with 20 participants. The comparison of pre- and post-test results highlights the positive learning effect of ICURA.
---
paper_title: How to build serious games

paper_content:

Like a discovery game that requires players to earn the right to take on new challenges, Immune Attack compels its players to learn the rules of the immune system before it reveals deeper biological insight.
---
paper_title: Adaptive Experience Engine for Serious Games

paper_content:

Designing games that support knowledge and skill acquisition has become a promising frontier of education techniques, since games are able to capture the user concentration for long periods and can present users with realistic and compelling challenges. In this scenario, there is a need for scientific and engineering methods to build games not only as more realistic simulations of the physical world but as means to provide effective learning experiences. Abstracting state of the art serious games' (SGs) features, we propose a new design methodology for the sand box serious games (SBSGs) class, decoupling content from the delivery strategy during the gameplay. This methodology aims at making design more efficient and standardized in order to meet the growing demand for interactive learning. The methodology consists in modeling an SBSG as a hierarchy of tasks (e.g., missions) and specifies the requirements for a runtime scheduling policy that maximizes learning objectives in a full entertainment context. The policy is learned by an experience engine (EE) based on computational intelligence. In this approach, the domain-expert author focuses on the creation and semantic annotation of tasks. Tasks are put in a repository and can then be exploited by game designers who define the expected learning curve and other requirements about education and entertainment for the game. The task sequencing that aims at matching such specifications with the real user profile is then presented to the EE. The EE can operate also in absence of the specification of the learning curve, continuously adapting the game flow without aiming at the achievement of target knowledge levels predefined by the author. We have implemented an EE module based on genetic computation and reinforcement learning (RL) atop of a state-of-the-art game engine. Test results show that the EE is able to define in real-time missions that meet the requirements expressed by the author.
---
paper_title: Designing entertaining educational games using procedural rhetoric: a case study

paper_content:

In the paper we describe the design and development of a video game about sustainable energy use that effectively unites fun with learning. We also present results from an initial study of the educational impact of the game. Many educational games do not properly translate knowledge, facts, and lessons into the language of games. This results in games that are often neither engaging nor educational. Our approach differs by using game mechanics to express the educational content. The design combines the fantasy elements and game play conventions of the real-time strategy (RTS) genre with numbers, resources and situations based on research about real-world energy production and use. The result is a game in which the player learns about energy use simply by trying to overcome the game's challenges. We demonstrate that effective and engaging learning games can be developed as long as sound game design principles are used. Results from a combined quantitative/qualitative study show that players enjoyed the game, learned new things and became more interested in the topic of energy use. This paper will highlight key aspects of the design that we believe contributed towards making the game fun as well as educational. The game also presents a model for translating real-world topics into game mechanics using the language of procedural rhetoric. The real world is ripe with problems and situations that could inspire interesting game mechanics and provide new creative ideas for educational and traditional game designers.
---
paper_title: Retrospective on educational testing and assessment in the 20th century

paper_content:

Over the last 100 years, the ever-increasing demand for testing as a measure of educational reform has created a very profitable market for the US testing industry. We follow the growth of this market since the 1900s in two different, but related, ways. First, we discuss some of the technical developments that have encouraged the use of standardized testing in general and contributed to the growth of the commercial testing industry. Second, we attempt to quantify the expansion of the testing marketplace during the 20th century by tracking several indirect indices of growth over time. We conclude that although technical innovations may have contributed to the growth of the US testing marketplace, they do not necessarily lead to better tests or better outcomes for those who use them. There is a need to more carefully monitor the effects of these tests on teaching and learning in general, particularly when the tests are used in high-stakes contexts.
---
paper_title: Educational Validity of Business Gaming Simulation: A Research Methodology Framework

paper_content:

Many past educational validity studies of business gaming simulation, and more specifically total enterprise simulation, have been inconclusive. Studies have focused on the weaknesses of business gaming simulation; which is often regarded as an educational medium that has limitations regarding learning effectiveness. However, no attempts have been made to provide guidelines for assessing educational validity in terms of building, implementing, and validating business gaming simulation. Accordingly, this study has combined literature on learning, simulation design, and research methods to formulate a methodology to assess the educational validity of total enterprise simulation; the concepts of which can be applied more broadly to business gaming simulation. The authors propose that the framework introduced within this article can provide a foundation for future educational validity studies that can assist simulation designers to implement valuable and powerful simulation learning media in the future.
---
paper_title: Evaluating business simulation software: approach, tools and pedagogy

paper_content:

Purpose – The purpose of this paper is to identify a business simulation appropriate for MEng Engineering students. The selection was based on the following factors; exploring methods for evaluating potential software and enhancing the learner experience.Design/methodology/approach – An interdisciplinary project team was formed to try and resolve the pedagogic, technical and business aspects that would need to be addressed in order to implement such software within the programme. Tools included a questionnaire to assess the potential enhancement of employability skills and a usability questionnaire on ease of use. These were supplemented with discourse on technical and pedagogic issues.Findings – After the initial scoping study, the findings indicated that two business simulation software packages had potential. These were “Marketplace – Venture Strategy” and “SimVenture”. Marketplace proved to be the most suitable in terms of the pedagogic and technical requirements.Research limitations/implications – Th...
---
paper_title: A task annotation model for Sandbox Serious Games

paper_content:

SGs represent a great potential for education since they can join an instructional value with the appealing language of digital natives. Searching for methodologies to support effective authoring of Serious Games (SGs), we have abstracted a model consisting of a 3D geo-referenced Virtual World (VW) where knowledge is implemented also through tasks that are disseminated across the environment. The model is designed to support authors to focus on defining the tasks' content and annotation (e.g. relevance to various topics, difficulty, position in the VW, typology of supported learning styles, interrelation with other tasks and events, etc.), without being asked to do game scripting, which typically involves hardcoding events, actions, locations and contents of a specific game. Task annotation allows reusing tasks also in different games. Moreover, different game designers can exploit repositories of tasks and build educational games specifying a VW, and its missions as sequences of tasks that meet learning strategy criteria that can be expressed through the proposed framework. This model opens new perspectives to pedagogical experts and game designers to develop runtime game engines able to provide players with adaptive contents. The paper presents the user, task and learning strategy models in detail and an example of application.
---
paper_title: A systematic literature review of empirical evidence on computer games and serious games

paper_content:

This paper examines the literature on computer games and serious games in regard to the potential positive impacts of gaming on users aged 14 years or above, especially with respect to learning, skill enhancement and engagement. Search terms identified 129 papers reporting empirical evidence about the impacts and outcomes of computer games and serious games with respect to learning and engagement and a multidimensional approach to categorizing games was developed. The findings revealed that playing computer games is linked to a range of perceptual, cognitive, behavioural, affective and motivational impacts and outcomes. The most frequently occurring outcomes and impacts were knowledge acquisition/content understanding and affective and motivational outcomes. The range of indicators and measures used in the included papers are discussed, together with methodological limitations and recommendations for further work in this area.
---
paper_title: Designing Effective Serious Games: Opportunities and Challenges for Research

paper_content:

Serious Games represent an acknowledged potential for instruction, because they are able to strongly motivate learners. They can also provide immersive environments where advanced users can practice knowledge and skills, also exploiting multimodal interaction. They can combine the effectiveness of computer processing and data storage, with high levels of attractiveness. Our work has investigated the state of the art research on SGs, starting from the cognitive aspects, that are necessary in order to root technological development and applications in sound theoretical foundations. The paper discusses some key aspects about SG design and exploitation: choice of components-off-the-shelf or from-scratch design, tools and methodologies for development or adaptation, intelligent tutoring, virtual coaches and affective learning, living worlds, game mechanics, Human-Computer Interaction. ::: While several SGs have been developed, still the literature stresses a lack of significant, extensive user tests. Further research is necessary to investigate in greater detail the real effectiveness of the various types of SGs. The paper proposes several research questions - that range from requirements elicitation to design and from deployment to use and evaluation - to be answered in order to avoid technology pushing and drive technological research according to the requirements of the end-users and stakeholders. ::: We believe that deepening the analysis about these issues is key to strengthen the foundations of SG research, for which we identify four major directions: definition of metrics and learning progress evaluation tools; methodologies and tools for designing games from various topics and for various users; computing and communication architectures; technologies that can enhance the overall system performance.
---

Title: Assessment in and of Serious Games: An Overview

Section 1: Introduction
Description 1: Introduce the concept of serious games, their dual purpose of entertainment and education, and the need for their assessment.

Section 2: General Context
Description 2: Discuss the educational potential of serious games, the existing literature, and empirical studies on their effectiveness.

Section 3: Assessing a Serious Game's Effectiveness
Description 3: Explain the different methods for assessing the educational impact of serious games, including summative and formative assessments.

Section 4: Indirect Measures of Learning
Description 4: Explore the indirect indicators of learning such as engagement and flow, and how they can be measured.

Section 5: Audio/Visual Technologies to Support Assessment
Description 5: Describe how modern audio/visual technologies can facilitate in-process and teacher assessments by recording gameplay and player interactions.

Section 6: Assessing Entertainment
Description 6: Discuss how to measure the entertainment value of serious games, including quantitative and qualitative approaches.

Section 7: Techniques and Tools for Student Performance Assessment
Description 7: Review various technology-assisted methods and tools for student performance assessment, such as e-Assessments and online testing platforms.

Section 8: In-Game Assessment
Description 8: Focus on the methods of assessing learning within the game itself, including the use of in-game metrics and stealth assessment techniques.

Section 9: Conclusions and Directions for Future Research
Description 9: Summarize key findings and discuss future research directions for improving the assessment of serious games.