AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring
---
paper_title: Collaborative dialogue patterns in naturalistic one‐to‐one tutoring

paper_content:

Naturalistic one-to-one tutoring is more effective than traditional classroom teaching methods, but there have been few attempts to examine the features of normal tutoring that might explain its advantage. This project explored dialogue patterns in two samples of naturalistic tutoring with normal unskilled tutors (as opposed to expert tutors): graduate students tutoring undergraduates in research methods and high school students tutoring 7th graders in algebra. We analysed the extent to which those tutoring protocols manifested components that have been emphasized in contemporary pedagogical theories and intelligent tutoring systems: active student learning, sophisticated pedagogical strategies, specific examples and cases, collaborative problem solving and question answering, deep explanatory reasoning, convergance toward shared meanings, feedback, error diagnosis and remediation, and affect. The most prominent components consisted of collaborative problem solving, question answering, and explanation in the context of specific examples. We identify frequent dialogue patterns that characterize these collaborative processes.
---
paper_title: The Skills of Document Use: From Text Comprehension to Web-Based Learning

paper_content:

Contents: Foreword. Preface. Introduction. The Psychology of Text Comprehension: Current Theories and Issues. The Role of Text Organizers in Expert Comprehension. Comprehending Multiple Documents. Question Answering and Document Search. Using Hypertext Systems. Learning From Multiple Online Sources of Information. General Conclusions.
---
paper_title: Parsing the Turing Test: Philosophical and Methodological Issues in the Quest for the Thinking Computer

paper_content:

Parsing the Turing Test is a landmark exploration of both the philosophical and methodological issues surrounding the search for true artificial intelligence. Will computers and robots ever think and communicate the way humans do? When a computer crosses the threshold into self-consciousness, will it immediately jump into the Internet and create a World Mind? Will intelligent computers someday recognize the rather doubtful intelligence of human beings? Distinguished psychologists, computer scientists, philosophers, and programmers from around the world debate these weighty issues and, in effect, the future of the human race in this important volume.
---
paper_title: Focus on Formative Feedback

paper_content:

This article reviews the corpus of research on feedback, with a focus on formative feedback—defined as information communicated to the learner that is intended to modify his or her thinking or behavior to improve learning. According to researchers, formative feedback should be nonevaluative, supportive, timely, and specific. Formative feedback is usually presented as information to a learner in response to some action on the learner’s part. It comes in a variety of types (e.g., verification of response accuracy, explanation of the correct answer, hints, worked examples) and can be administered at various times during the learning process (e.g., immediately following an answer, after some time has elapsed). Finally, several variables have been shown to interact with formative feedback’s success at promoting learning (e.g., individual characteristics of the learner and aspects of the task). All of these issues are discussed. This review concludes with guidelines for generating formative feedback.
---
paper_title: Cognitive Tutor: Applied research in mathematics education

paper_content:

For 25 years, we have been working to build cognitive models of mathematics, which have become a basis for middle- and high-school curricula. We discuss the theoretical background of this approach and evidence that the resulting curricula are more effective than other approaches to instruction. We also discuss how embedding a well specified theory in our instructional software allows us to dynamically evaluate the effectiveness of our instruction at a more detailed level than was previously possible. The current widespread use of the software is allowing us to test hypotheses across large numbers of students. We believe that this will lead to new approaches both to understanding mathematical cognition and to improving instruction.
---
paper_title: Intelligent Tutoring Systems with Conversational Dialogue

paper_content:

Many of the intelligent tutoring systems that have been developed during the last 20 years have proven to be quite successful, particularly in the domains of mathematics, science, and technology. They produce significant learning gains beyond classroom environments. They are capable of engaging most students' attention and interest for hours. We have been working on a new generation of intelligent tutoring systems that hold mixed-initiative conversational dialogues with the learner. The tutoring systems present challenging problems and questions to the learner, the learner types in answers in English, and there is a lengthy multiturn dialogue as complete solutions or answers evolve. This article presents the tutoring systems that we have been developing. AutoTutor is a conversational agent, with a talking head, that helps college students learn about computer literacy. andes, atlas, and why2 help adults learn about physics. Instead of being mere information-delivery systems, our systems help students actively construct knowledge through conversations.
---
paper_title: Simulating human tutor dialog moves in AutoTutor

paper_content:

This purpose of this paper is to show how prevalent features of successful human tutoring interactions can be integrated into a pedagogical agent, AutoTutor. AutoTutor is a fully automated computer tutor that responds to learner input by simulating the dialog moves of effective, normal human tutors. AutoTutor’s delivery of dialog moves is organized within a 5step framework that is unique to normal human tutoring interactions. We assessed AutoTutor’s performance as an effective tutor and conversational partner during tutoring sessions with virtual students of varying ability levels. Results from three evaluation cycles indicate the following: (1) AutoTutor is capable of delivering pedagogically effective dialog moves that mimic the dialog move choices of human tutors, and (2) AutoTutor is a reasonably effective conversational partner.
---
paper_title: Does active learning work? A review of the research

paper_content:

This study examines the evidence for the effectiveness of active learning. It defines the common forms of active learning most relevant for engineering faculty and critically examines the core element of each method. It is found that there is broad but uneven support for the core elements of active, collaborative, cooperative and problem-based learning.
---
paper_title: A Time For Telling

paper_content:

Suggestions for improving text understanding often prescribe activating prior knowledge, a prescription that may be problematic if students do not have the relevant prior knowledge to begin with. In this article, we describe research about a method for developing prior knowledge that prepares students to learn from a text or lecture. We propose that analyzing contrasting cases can help learners generate the differentiated knowledge structures that enable them to understand a text deeply. Noticing the distinctions between contrasting cases creates a "time for telling"; learners are prepared to be told the significance of the distinctions they have discovered. In 3 classroom studies, college students analyzed contrasting cases that consisted of simplified experimental designs and data from classic psychology experiments. They then received a lecture or text on the psychological phenomena highlighted in the experiments. Approximately 1 week later, the students predicted outcomes for a hypothetical experiment that could be interpreted in light of the concepts they had studied. Generating the distinctions between contrasting cases and then reading a text or hearing a lecture led to more accurate predictions than the control treatments of (a) reading about the distinctions between the cases and hearing alecture, (b) summarizing a relevant text and hearing a lecture, and (c) analyzing the contrasting cases twice without receiving a lecture. We argue that analyzing the contrasting cases increased students' abilities to discern specific features that differentiated classes of psychological phenomena, much as a botanist can distinguish subspecies of a given flower. This differentiated knowledge prepared the students to understand deeply an explanation of the relevant psychological principles when it was presented to them. These results can inform constructivist models of instruction as they apply to classroom activities and learning from verbal materials. In particular, the results indicate that there is a place for lectures and readings in the classroom if students have sufficiently differentiated domain knowledge to use the expository materials in a generative manner.
---
paper_title: Why/AutoTutor: A Test of Learning Gains from a Physics Tutor with Natural Language Dialog

paper_content:

Author(s): Graesser, A.C.; Jackson, G.T.; Matthews, E.C.; Mitchell, H.H.; Olney, A.; Ventura, M.; Chipman, P.; Franceschetti, D.; Hu, X.; Louwerse, M.M.; Person, N.K.; Tutoring Research Group
---
paper_title: Why do only some events cause learning during human tutoring

paper_content:

Developers of intelligent tutoring systems would like to know what human tutors do and which activities are responsible for their success in tutoring. We address these questions by comparing episodes where tutoring does and does not cause learning. Approximately 125 hr of tutorial dialog between expert human tutors and physics students are analyzed to see what features of the dialog are associated with learning. Successful learning appears to require that the student reach an impasse. When students were not at an impasse, learning was uncommon regardless of the tutorial explanations employed. On the other hand, once students were at an impasse, tutorial explanations were sometimes associated with learning. Moreover, for different types of knowledge, different types of tutorial explanations were associated with learning different types of knowledge.
---
paper_title: Self-regulated Learning with MetaTutor: Advancing the Science of Learning with MetaCognitive Tools

paper_content:

The key to understanding complex learning with advanced learning technologies (e.g., hypermedia) lies in our ability to comprehend the temporal deployment of students’ cognitive, metacognitive, motivational, and affective processes. Our chapter will focus on critically analyzing the use of mixed-method approaches to analyze the complex nature of self-regulated learning (SRL) during hypermedia learning. We will use examples from our own research (e.g., Azevedo 2008, Recent innovations in educational technology that facilitate student learning (pp. 127–156); Azevedo & Witherspoon, in press, Handbook of metacognition in education) and that of others (e.g., Biswas et al., 2005; Schwartz et al., in press; Winne & Nesbitt, in press, Handbook of metacognition in education) to present and discuss the strengths and weaknesses in using mixed methods to capture, model, trace, and infer the unfolding SRL processes during learning with nonlinear, multirepresentational computerized environments. The chapter will focus on the methods, and quantitative and qualitative analyses used to converge product data (e.g., learning outcomes), process data (e.g., think-aloud data), and log-file data collected during learning, develop coding schemes to categorize and infer the deployment of SRL processes, and the use of computational tools to examine learners’ behaviors and navigation paths. Lastly, we will present a theoretical model that integrates the various topics presented in this chapter that will guide future research and educational practices for fostering students’ SRL with hypermedia environments.
---
paper_title: Coh-Metrix: Analysis of text on cohesion and language

paper_content:

Advances in computational linguistics and discourse processing have made it possible to automate many language- and text-processing mechanisms. We have developed a computer tool called Coh-Metrix, which analyzes texts on over 200 measures of cohesion, language, and readability. Its modules use lexicons, part-of-speech classifiers, syntactic parsers, templates, corpora, latent semantic analysis, and other components that are widely used in computational linguistics. After the user enters an English text, Coh-Metrix returns measures requested by the user. In addition, a facility allows the user to store the results of these analyses in data files (such as Text, Excel, and SPSS). Standard text readability formulas scale texts on difficulty by relying on word length and sentence length, whereas Coh-Metrix is sensitive to cohesion relations, world knowledge, and language and discourse characteristics.
---
paper_title: Operation ARA: A Computerized Learning Game that Teaches Critical Thinking and Scientific Reasoning.

paper_content:

Abstract Operation ARA (Acquiring Research Acumen) is a computerized learning game that teaches critical thinking and scientific reasoning. It is a valuable learning tool that utilizes principles from the science of learning and serious computer games. Students learn the skills of scientific reasoning by engaging in interactive dialogs with avatars. They are tutored by avatars with tutoring sessions that vary depending on how well students have responded to questions about the material they are learning. Students also play a jeopardy-like game against a feisty avatar to identify flaws in research and then generate their own questions to determine the quality of different types of research. The research examples are taken from psychology, biology, and chemistry to help students transfer the thinking skills across domains of knowledge. Early results show encouraging learning gains.
---
paper_title: Operation ARIES!: A Serious Game for Teaching Scientific Inquiry

paper_content:

Operation ARIES! is a serious game that teaches critical thinking about scientific inquiry. The player must help to identify aliens on Earth who are intentionally publishing bad research. The game combines aspects of video games and intelligent tutors in which the player holds conversations with animated agents using natural language. The player first takes a training course with a virtual trainee, followed by a module in which the player identifies flaws in research cases. In the third and final module, the player interviews suspected alien scientists on their research. Operation ARIES! is designed for high school seniors and adults.
---
paper_title: Learning, thinking, and emoting with discourse technologies.

paper_content:

This is an unusual moment in the history of psychology because of landmark advances in digital information technologies, computational linguistics, and other fields that use the computer to analyze language, discourse, and behavior. The technologies developed from this interdisciplinary fusion are helping students learn and think in ways that are sensitive to their cognitive and emotional states. Recent projects have developed computer technologies that help us understand the nature of conversational discourse and text comprehension in addition to improving learning. AutoTutor and other systems with conversational agents (i.e., talking heads) help students learn by holding conversations in natural language. One version of AutoTutor is sensitive to the emotions of students in addition to their cognitive states. Coh-Metrix analyzes texts on multiple levels of language and discourse, such as text genre, cohesion, syntax, and word characteristics. Coh-Metrix can assist students, teachers, principals, and policymakers when they make decisions on the right text to assign to the right student at the right time. Computers are not perfect conversation partners and comprehenders of text, but the current systems are undeniably useful. (PsycINFO Database Record (c) 2011 APA, all rights reserved).
---
paper_title: When does feedback facilitate learning of words

paper_content:

The question of what form of feedback best promotes associative learning and retention is of obvious practical import. However, the literature on feedback is confusing, with some researchers suggesting that although feedback may enhance performance during learning, it does so at the expense of later retention. To shed further light on this issue, subjects (N = 258) learned Luganda–English word pairs. After 2 initial exposures to the materials, subjects were tested on each item several times, with the presence and type of feedback varying between subjects. Subjects were given a final test on the same items 1 week later. Supplying the correct answer after an incorrect response not only improved performance during the initial learning session—it also increased final retention by 494%. On the other hand, feedback after correct responses made little difference either immediately or at a delay, regardless of whether the subject was confident in the response. Practical and theoretical implications are discussed.
---
paper_title: Vygotsky, Tutoring and Learning

paper_content:

Abstract This paper starts with a selective examination of research into individualised tutoring. We articulate theoretical and empirical evidence to support the conclusion that a set of common principles governs such tutoring. These principles apply to the provision of tutorial support for preschool aged children through to adult learners in a number of domains. In support of these claims, we bring together and relate research and theory from both face to‐face teaching and computer‐based tutoring situations. However, we also try to articulate some of the limits of our current knowledge. We suggest that the principles identified apply mainly to procedural learning in well‐structured domains. We examine the idea that other learning goals, such as the development of skills in self‐regulation, need to be derived from a different model of the learning process.
---
paper_title: Human use regulatory affairs advisor (HURAA): Learning about research ethics with intelligent learning modules

paper_content:

The Human Use Regulatory Affairs Advisor (HURAA) is a Web-based facility that provides help and training on the ethical use of human subjects in research, based on documents and regulations in United States federal agencies. HURAA has a number of standard features of conventional Web facilities and computer-based training, such as hypertext, multimedia, help modules, glossaries, archives, links to other sites, and page-turning didactic instruction. HURAA also has these intelligent features: (1) an animated conversational agent that serves as a navigational guide for the Web facility, (2) lessons with case-based and explanation-based reasoning, (3) document retrieval through natural language queries, and (4) a context-sensitive Frequently Asked Questions segment, calledPoint & Query. This article describes the functional learning components of HURAA, specifies its computational architecture, and summarizes empirical tests of the facility on learners.
---
paper_title: The Andes Physics Tutoring System: An Experiment in Freedom

paper_content:

The Andes physics tutoring system is an experiment in student freedom. It allows students to solve a physics problem in virtually any legal way. This means that Andes must recognize an extremely large number of possible steps occurring in an extraordinarily large number of possible orders. Such freedom raises several research questions. (1) How can Andes solve the technical challenge of understanding student’s behavior in such a wide-open context? (2) How can Andes give pedagogically useful help and guidance? In particular, how can it guide students who are floundering without curtailing the freedom of students who are not floundering? (3) Will Andes be effective in getting students in real classrooms to learn physics? (4) What does it take to scale up Andes and disseminate it widely? The Andes project, which began in the mid 1990’s, has achieved workable solutions to the first three goals: Andes can understand student behavior; It provides pedagogical help similar to that of human experts; Most importantly, Andes causes large, reliable learning gains compared to control classes taught with convention, paper-based instruction. This chapter summarizes the first three results and discusses our progress on the fourth goal, scale-up.
---
paper_title: When Are Tutorial Dialogues More Effective Than Reading?

paper_content:

It is often assumed that engaging in a one-on-one dialogue with a tutor is more effective than listening to a lecture or reading a text. Although earlier experiments have not always supported this hypothesis, this may be due in part to allowing the tutors to cover different content than the noninteractive instruction. In 7 experiments, we tested the interaction hypothesis under the constraint that (a) all students covered the same content during instruction, (b) the task domain was qualitative physics, (c) the instruction was in natural language as opposed to mathematical or other formal languages, and (d) the instruction conformed with a widely observed pattern in human tutoring: Graesser, Person, and Magliano’s 5-step frame. In the experiments, we compared 2 kinds of human tutoring (spoken and computer mediated) with 2 kinds of natural-language-based computer tutoring (Why2-Atlas and Why2-AutoTutor) and 3 control conditions that involved studying texts. The results depended on whether the students’ preparation matched the content of the instruction. When novices (students who had not taken college physics) studied content that was written for intermediates (students who had taken college physics), then tutorial dialogue was reliably more beneficial than less interactive instruction, with large effect sizes. When novices studied material written for novices or intermediates studied material written for intermediates, then tutorial dialogue was not reliably more effective than the text-based control conditions.
---
paper_title: Autotutor: A tutor with dialogue in natural language

paper_content:

AutoTutor is a learning environment that tutors students by holding a conversation in natural language. AutoTutor has been developed for Newtonian qualitative physics and computer literacy. Its design was inspired by explanation-based constructivist theories of learning, intelligent tutoring systems that adaptively respond to student knowledge, and empirical research on dialogue patterns in tutorial discourse. AutoTutor presents challenging problems (formulated as questions) from a curriculum script and then engages in mixed initiative dialogue that guides the student in building an answer. It provides the student with positive, neutral, or negative feedback on the student’s typed responses, pumps the student for more information, prompts the student to fill in missing words, gives hints, fills in missing information with assertions, identifies and corrects erroneous ideas, answers the student’s questions, and summarizes answers. AutoTutor has produced learning gains of approximately .70 sigma for deep levels of comprehension.
---
paper_title: Collaborative dialogue patterns in naturalistic one‐to‐one tutoring

paper_content:

Naturalistic one-to-one tutoring is more effective than traditional classroom teaching methods, but there have been few attempts to examine the features of normal tutoring that might explain its advantage. This project explored dialogue patterns in two samples of naturalistic tutoring with normal unskilled tutors (as opposed to expert tutors): graduate students tutoring undergraduates in research methods and high school students tutoring 7th graders in algebra. We analysed the extent to which those tutoring protocols manifested components that have been emphasized in contemporary pedagogical theories and intelligent tutoring systems: active student learning, sophisticated pedagogical strategies, specific examples and cases, collaborative problem solving and question answering, deep explanatory reasoning, convergance toward shared meanings, feedback, error diagnosis and remediation, and affect. The most prominent components consisted of collaborative problem solving, question answering, and explanation in the context of specific examples. We identify frequent dialogue patterns that characterize these collaborative processes.
---
paper_title: Intelligent Tutoring Systems with Conversational Dialogue

paper_content:

Many of the intelligent tutoring systems that have been developed during the last 20 years have proven to be quite successful, particularly in the domains of mathematics, science, and technology. They produce significant learning gains beyond classroom environments. They are capable of engaging most students' attention and interest for hours. We have been working on a new generation of intelligent tutoring systems that hold mixed-initiative conversational dialogues with the learner. The tutoring systems present challenging problems and questions to the learner, the learner types in answers in English, and there is a lengthy multiturn dialogue as complete solutions or answers evolve. This article presents the tutoring systems that we have been developing. AutoTutor is a conversational agent, with a talking head, that helps college students learn about computer literacy. andes, atlas, and why2 help adults learn about physics. Instead of being mere information-delivery systems, our systems help students actively construct knowledge through conversations.
---
paper_title: Learning, thinking, and emoting with discourse technologies.

paper_content:

This is an unusual moment in the history of psychology because of landmark advances in digital information technologies, computational linguistics, and other fields that use the computer to analyze language, discourse, and behavior. The technologies developed from this interdisciplinary fusion are helping students learn and think in ways that are sensitive to their cognitive and emotional states. Recent projects have developed computer technologies that help us understand the nature of conversational discourse and text comprehension in addition to improving learning. AutoTutor and other systems with conversational agents (i.e., talking heads) help students learn by holding conversations in natural language. One version of AutoTutor is sensitive to the emotions of students in addition to their cognitive states. Coh-Metrix analyzes texts on multiple levels of language and discourse, such as text genre, cohesion, syntax, and word characteristics. Coh-Metrix can assist students, teachers, principals, and policymakers when they make decisions on the right text to assign to the right student at the right time. Computers are not perfect conversation partners and comprehenders of text, but the current systems are undeniably useful. (PsycINFO Database Record (c) 2011 APA, all rights reserved).
---
paper_title: Indexing by Latent Semantic Analysis

paper_content:

A new method for automatic indexing and retrieval is described. The approach is to take advantage of implicit higher-order structure in the association of terms with documents (“semantic structure”) in order to improve the detection of relevant documents on the basis of terms found in queries. The particular technique used is singular-value decomposition, in which a large term by document matrix is decomposed into a set of ca. 100 orthogonal factors from which the original matrix can be approximated by linear combination. Documents are represented by ca. 100 item vectors of factor weights. Queries are represented as pseudo-document vectors formed from weighted combinations of terms, and documents with supra-threshold cosine values are returned. initial tests find this completely automatic method for retrieval to be promising.
---
paper_title: Effective Tutoring Techniques: A Comparison of Human Tutors and Intelligent Tutoring Systems

paper_content:

There has been much debate about instructional strategies for computerized learning environments. Many of the arguments designed to choose between the various philosophies have appealed, at least implicitly, to the behavior of effective human teachers. In this article, we compare the guidance and support offered by human tutors with that offered by intelligent tutoring systems. First, we review research on human tutoring strategies in various domains. Then we investigate the capabilities of a widely used technique for providing feedback, model tracing. Finally, we contrast the types of guidance and support provided by human tutors with those in intelligent tutoring systems, by examining the process of recovering from impasses encountered during problem solving. In general, the support offered by human tutors is more flexible and more subtle than that offered by model tracing tutors, but the two are more similar than sometimes argued.
---
paper_title: Question asking during tutoring

paper_content:

Whereas it is well documented that student question asking is infrequent in classroom environments, there is little research on questioning processes during tutoring. The present study investigated the questions asked in tutoring sessions on research methods (college students) and algebra (7th graders). Student questions were approximately 240 times as frequent in tutoring settings as classroom settings, whereas tutor questions were only slightly more frequent than teacher questions. Questions were classified by (a) degree of specification, (b) content, and (c) question-generation mechanism to analyze their quality. Student achievement was positively correlated with the quality of student questions after students had some experience with tutoring, but the frequency of questions was not correlated with achievement. Students partially self-regulated their learning by identifying knowledge deficits and asking questions to repair them, but they need training to improve these skills. We identified some ways th...
---
paper_title: Theoretical Foundations for Intelligent Tutoring Systems

paper_content:

This paper considers the case for formalising aspects of intelligent tutoring systems in order to derive more reliable implementations, as opposed to the present use of informal theories to build experimental systems which are then studied empirically. Some recent work in theoretical AI is suggested as a possible source for the elements of a 'theory of ITS'. I n t r o d u c t i o n The engineering of any complex device (such as an ITS) gradually relies less on empirical experimentation and more on mathematical or scientific theory. As yet, there is no significant 'theory of ITS': all of the recent ITS texts (e.g. Wenger, 1987; Mandl and Lesgold, 1988; Polson and Richardson, 1988) are entirely discursive and attempt no kind of formalisation of their content. The aim of this paper is to suggest that it is not premature for ITS research to begin an attempt to complement a short-term emphasis on pragmatic aspects (Kearsley, 1989) by seeking theoretical foundations for its implementations. Most AI researchers regard ITSs as peripheral applications of AI, an understandable opinion in view of the virtual absence of ITS papers from the major AI journals and conferences. But Clancey (1986) has argued that work on ITSs is not a "mere matter of putting well-known AI methods into practice" but is (or should be) "broadening the meaning of AI research". Historically, ITS research began within AI, but AI researchers have retreated from the ITS arena as they have come to appreciate the need for more fundamental work on mental models, language understanding, knowledge representation, etc., leaving others to move into an intrinsically multi-disciplinary field. However, if there is ever to be a formal theory of (aspects of) ITS then it will be derived from elements of AI. Moreover, recent AI research begins to indicate what those elements might be.
---
paper_title: Intelligent tutoring systems: an overview

paper_content:

This is a non-expert overview of Intelligent Tutoring Systems (ITSs), a way in which Artificial Intelligence (AI) techniques are being applied to education. It introduces ITSs and the motivation for them. It looks at its history: its evolution from Computer-Assisted Instruction (CAI). After looking at the structure of a ‘typical’ ITS, the paper further examines and discusses some other architectures. Several classic ITSs are reviewed, mainly due to their historical significance or because they best demonstrate some of the principles of intelligent tutoring. A reasonably representative list of ITSs is also provided in order to provide a better appreciation of this vibrant field as well as reveal the scope of existing tutors. The paper concludes, perhaps more appropriately, with some of the author's viewpoints on a couple of controversial issues in the intelligent tutoring domain.
---
paper_title: The Deep-Level-Reasoning-Question Effect: The Role of Dialogue and Deep-Level-Reasoning Questions During Vicarious Learning

paper_content:

We investigated the impact of dialogue and deep-level-reasoning questions on vicarious learning in 2 studies with undergraduates. In Experiment 1, participants learned material by interacting with AutoTutor or by viewing 1 of 4 vicarious learning conditions: a noninteractive recorded version of the AutoTutor dialogues, a dialogue with a deep-level-reasoning question preceding each sentence, a dialogue with a deep-level-reasoning question preceding half of the sentences, or a monologue. Learners in the condition where a deep-level-reasoning question preceded each sentence significantly outperformed those in the other 4 conditions. Experiment 2 included the same interactive and noninteractive recorded condition, along with 2 vicarious learning conditions involving deep-level-reasoning questions. Both deep-level-reasoning-question conditions significantly outperformed the other conditions. These findings provide evidence that deep-level-reasoning questions improve vicarious learning.
---
paper_title: Scaffolding deep comprehension strategies through Point&Query, AutoTutor, and iSTART

paper_content:

It is well-documented that most students do not have adequate proficiencies in inquiry and metacognition, particularly at deeper levels of comprehension that require explanatory reasoning. The proficiencies are not routinely provided by teachers and normal tutors so it is worthwhile to turn to computer-based learning environments. This article describes some of our recent computer systems that were designed to facilitate explanation-centered learning through strategies of inquiry and metacognition while students learn science and technology content. Point&Query augments hypertext, hypermedia, and other learning environments with question-answer facilities that are under the learner control. AutoTutor and iSTART use animated conversational agents to scaffold strategies of inquiry, metacognition, and explanation construction. AutoTutor coaches students in generating answers to questions that require explanations (e.g., why, what-if, how) by holding a mixed-initiative dialogue in natural language. iSTART mod...
---
paper_title: Operation ARIES!: A Serious Game for Teaching Scientific Inquiry

paper_content:

Operation ARIES! is a serious game that teaches critical thinking about scientific inquiry. The player must help to identify aliens on Earth who are intentionally publishing bad research. The game combines aspects of video games and intelligent tutors in which the player holds conversations with animated agents using natural language. The player first takes a training course with a virtual trainee, followed by a module in which the player identifies flaws in research cases. In the third and final module, the player interviews suspected alien scientists on their research. Operation ARIES! is designed for high school seniors and adults.
---
paper_title: Intelligent Tutoring Systems with Conversational Dialogue

paper_content:

Many of the intelligent tutoring systems that have been developed during the last 20 years have proven to be quite successful, particularly in the domains of mathematics, science, and technology. They produce significant learning gains beyond classroom environments. They are capable of engaging most students' attention and interest for hours. We have been working on a new generation of intelligent tutoring systems that hold mixed-initiative conversational dialogues with the learner. The tutoring systems present challenging problems and questions to the learner, the learner types in answers in English, and there is a lengthy multiturn dialogue as complete solutions or answers evolve. This article presents the tutoring systems that we have been developing. AutoTutor is a conversational agent, with a talking head, that helps college students learn about computer literacy. andes, atlas, and why2 help adults learn about physics. Instead of being mere information-delivery systems, our systems help students actively construct knowledge through conversations.
---
paper_title: Why/AutoTutor: A Test of Learning Gains from a Physics Tutor with Natural Language Dialog

paper_content:

Author(s): Graesser, A.C.; Jackson, G.T.; Matthews, E.C.; Mitchell, H.H.; Olney, A.; Ventura, M.; Chipman, P.; Franceschetti, D.; Hu, X.; Louwerse, M.M.; Person, N.K.; Tutoring Research Group
---
paper_title: Improving the efficiency of dialogue in tutoring

paper_content:

Abstract The current studies investigated the efficient use of dialogue in intelligent tutoring systems that use natural language interaction. Such dialogues can be relatively time-consuming. This work addresses the question of how much dialogue is needed to produce significant learning gains. In Experiment 1, a full dialogue condition and a read-only control condition were compared with a mixed dialogue condition in which students engaged in full dialogue for half the problems followed by problems requiring only a limited engagement. We found that the mixed dialogue condition produced results as impressive as the full dialogue condition and took less time. Experiment 2 replicated these findings and further examined issues of time engaged in learning, quality of instruction, and learning gains. Overall, these results show that dialogue-based intelligent tutoring systems could be designed in a more efficient manner to maximize learning and minimize the cost of time-on-task.
---
paper_title: Coh-Metrix: Analysis of text on cohesion and language

paper_content:

Advances in computational linguistics and discourse processing have made it possible to automate many language- and text-processing mechanisms. We have developed a computer tool called Coh-Metrix, which analyzes texts on over 200 measures of cohesion, language, and readability. Its modules use lexicons, part-of-speech classifiers, syntactic parsers, templates, corpora, latent semantic analysis, and other components that are widely used in computational linguistics. After the user enters an English text, Coh-Metrix returns measures requested by the user. In addition, a facility allows the user to store the results of these analyses in data files (such as Text, Excel, and SPSS). Standard text readability formulas scale texts on difficulty by relying on word length and sentence length, whereas Coh-Metrix is sensitive to cohesion relations, world knowledge, and language and discourse characteristics.
---
paper_title: The goal structure of a socratic tutor

paper_content:

We describe the current version of the Why System, a script-based socratic tutor which uses tutoring strategies formulated as production rules. The current system is capable of carrying on a dialogue about the factors influencing rainfall by presenting different cases to the student, asking for predictions, probing for relevent factors, entrapping the student when he has not identified all necessary factors, and presenting counterexamples. The current system is incomplete because it lacks a goal structure to guide the tutorial sessions. We outline a more complete theory of the goal structure of Socratic tutors based on analysis of human tutorial dialogues. There are two top level goals: (1) refinement of the student's causal model and (2) refinement of the student's predictive abilities. The subgoals are diagnosis of bugs in the student's knowledge and correction of the bugs. This goal-driven control mechanism governs the selection of examples and teaching strategies used by the tutor.
---
paper_title: Teaching Students to Generate Questions: A Review of the Intervention Studies

paper_content:

This is a review of intervention studies in which students have been taught to generate questions as a means of improving their comprehension. Overall, teaching students the cognitive strategy of generating questions about the material they had read resulted in gains in comprehension, as measured by tests given at the end of the intervention. All tests were based on new material. The overall median effect size was 0.36 (64th percentile) when standardized tests were used and 0.86 (81st percentile) when experimenter-developed comprehension tests were used. The traditional skill-based instructional approach and the reciprocal teaching approach yielded similar results.
---
paper_title: AutoTutor : A simulation of a human tutor Action

paper_content:

AutoTutor is a computer tutor that simulates the discourse patterns and pedagogical strategies of a typical human tutor. AutoTutor is designed to assist college students in learning the fundamentals of hardware, operating systems, and the Internet in an introductory computer literacy course. Most tutors in school systems are not highly trained in tutoring techniques and have only a modest expertise on the tutoring topic, but they are surprisingly effective in producing learning gains in students. We have dissected the discourse and pedagogical strategies these unskilled tutors exhibit by analyzing approximately 100 hours of naturalistic tutoring sessions. These mechanisms are implemented in AutoTutor. AutoTutor presents questions and problems from a curriculum script, attempts to comprehend learner contributions that are entered by keyboard, formulates dialog moves that are sensitive to the learner's contributions (such as short feedback, pumps, prompts, elaborations, corrections, and hints), and delivers the dialog moves with a talking head. AutoTutor has seven modules: a curriculum script, language extraction, speech act classification, latent semantic analysis, topic selection, dialog move generation, and a talking head.
---
paper_title: Question asking during tutoring

paper_content:

Whereas it is well documented that student question asking is infrequent in classroom environments, there is little research on questioning processes during tutoring. The present study investigated the questions asked in tutoring sessions on research methods (college students) and algebra (7th graders). Student questions were approximately 240 times as frequent in tutoring settings as classroom settings, whereas tutor questions were only slightly more frequent than teacher questions. Questions were classified by (a) degree of specification, (b) content, and (c) question-generation mechanism to analyze their quality. Student achievement was positively correlated with the quality of student questions after students had some experience with tutoring, but the frequency of questions was not correlated with achievement. Students partially self-regulated their learning by identifying knowledge deficits and asking questions to repair them, but they need training to improve these skills. We identified some ways th...
---
paper_title: The Andes Physics Tutoring System: An Experiment in Freedom

paper_content:

The Andes physics tutoring system is an experiment in student freedom. It allows students to solve a physics problem in virtually any legal way. This means that Andes must recognize an extremely large number of possible steps occurring in an extraordinarily large number of possible orders. Such freedom raises several research questions. (1) How can Andes solve the technical challenge of understanding student’s behavior in such a wide-open context? (2) How can Andes give pedagogically useful help and guidance? In particular, how can it guide students who are floundering without curtailing the freedom of students who are not floundering? (3) Will Andes be effective in getting students in real classrooms to learn physics? (4) What does it take to scale up Andes and disseminate it widely? The Andes project, which began in the mid 1990’s, has achieved workable solutions to the first three goals: Andes can understand student behavior; It provides pedagogical help similar to that of human experts; Most importantly, Andes causes large, reliable learning gains compared to control classes taught with convention, paper-based instruction. This chapter summarizes the first three results and discusses our progress on the fourth goal, scale-up.
---
paper_title: When Are Tutorial Dialogues More Effective Than Reading?

paper_content:

It is often assumed that engaging in a one-on-one dialogue with a tutor is more effective than listening to a lecture or reading a text. Although earlier experiments have not always supported this hypothesis, this may be due in part to allowing the tutors to cover different content than the noninteractive instruction. In 7 experiments, we tested the interaction hypothesis under the constraint that (a) all students covered the same content during instruction, (b) the task domain was qualitative physics, (c) the instruction was in natural language as opposed to mathematical or other formal languages, and (d) the instruction conformed with a widely observed pattern in human tutoring: Graesser, Person, and Magliano’s 5-step frame. In the experiments, we compared 2 kinds of human tutoring (spoken and computer mediated) with 2 kinds of natural-language-based computer tutoring (Why2-Atlas and Why2-AutoTutor) and 3 control conditions that involved studying texts. The results depended on whether the students’ preparation matched the content of the instruction. When novices (students who had not taken college physics) studied content that was written for intermediates (students who had taken college physics), then tutorial dialogue was reliably more beneficial than less interactive instruction, with large effect sizes. When novices studied material written for novices or intermediates studied material written for intermediates, then tutorial dialogue was not reliably more effective than the text-based control conditions.
---
paper_title: A Natural Language Tutorial Dialogue System for Physics

paper_content:

Abstract : We describe the WHY2-ATLAS intelligent tutoring system for qualitative physics that interacts with students via natural language dialogue. We focus on the issue of analyzing and responding to multi-sentential explanations. We explore approaches for achieving a deeper understanding of these explanations and dialogue management approaches and strategies for providing appropriate feedback on them.
---
paper_title: Expert vs. non-expert tutoring: Dialogue moves, interaction patterns and multi-utterance turns

paper_content:

Studies of one-on-one tutoring have found that expert tutoring is more effective than non-expert tutoring, but the reasons for its effectiveness are relatively unexplored. Since tutoring involves deep natural language interactions between tutor and student, we explore the differences between an expert and non-expert tutors through the analysis of individual dialogue moves, tutorial interaction patterns and multi-utterance turns. Our results are a first step showing what behaviors constitute expertise and provide a basis for modeling effective tutorial language in intelligent tutoring systems.
---
paper_title: Collaborative lecturing by human and computer tutors

paper_content:

We implemented and evaluated a collaborative lecture module in an ITS that models the pedagogical and motivational tactics of expert human tutors Inspired by the lecture delivery styles of the expert tutors, the collaborative lectures of the ITS were conversational and interactive, instead of a polished one-way information delivery from tutor to student We hypothesized that the enhanced interactivity of the expert tutor lectures were linked to efforts to promote student engagement This hypothesis was tested in an experiment that compared the collaborative lecture module (dialogue) to less interactive alternatives such as monologues and vicarious dialogues The results indicated that students in the collaborative lecture condition reported more arousal (a key component of engagement) than the controls and that arousal was positively correlated with learning gains We discuss the implications of our findings for ITSs that aspire to model expert human tutors.
---
paper_title: Mining Collaborative Patterns in Tutorial Dialogues

paper_content:

We present a method to automatically detect collaborative patterns of student and tutor dialogue moves. The method identifies significant two-step excitatory transitions between dialogue moves, integrates the transitions into a directed graph representation, and generates and tests data-driven hypotheses from the directed graph. The method was applied to a large corpus of student-tutor dialogue moves from expert tutoring sessions. An examination of the subset of the corpus consisting of tutor lectures revealed collaborative patterns consistent with information-transmission, information-elicitation, off topic-conversation, and student initiated questions. Sequences of dialogue moves within each of these patterns were also identified. Comparisons of the method to other approaches and applications towards the computational modeling of expert human tutors are discussed.
---
paper_title: Guru: a computer tutor that models expert human tutors

paper_content:

We present Guru, an intelligent tutoring system for high school biology that has conversations with students, gestures and points to virtual instructional materials, and presents exercises for extended practice. Guru's instructional strategies are modeled after expert tutors and focus on brief interactive lectures followed by rounds of scaffolding as well as summarizing, concept mapping, and Cloze tasks. This paper describes the Guru session and presents learning outcomes from an in-school study comparing Guru, human tutoring, and classroom instruction. Results indicated significant learning gains for students in the Guru and human tutoring conditions compared to classroom controls.
---
paper_title: Short and long term benefits of enjoyment and learning within a serious game

paper_content:

Intelligent Tutoring Systems (ITSs) have been used for decades to teach students domain content or strategies. ITSs often struggle to maintain students' interest and sustain a productive practice environment over time. ITS designers have begun integrating game components as an attempt to engage learners and maintain motivation during prolonged interactions. Two studies were conducted to investigate enjoyment and performance at short-term (90 minutes) and long-term (3 weeks) timescales. The short-term study (n=34) found that students in a non-game practice condition performed significantly better and wrote more than the game-based practice. However, the long-term study (n=9) found that when students were in the game-based environment they produced longer contributions than when in the non-game version. Both studies revealed trends that the game-based system was slightly more enjoyable, though the differences were not significant. The different trends across studies indicate that games may contribute to an initial decrease in performance, but that students are able to close this gap over time.
---
paper_title: Writing pal: Feasibility of an intelligent writing strategy tutor in the high school classroom

paper_content:

The Writing Pal (W-Pal) is a novel intelligent tutoring system (ITS) that offers writing strategy instruction, game-based practice, essay writing practice, and formative feedback to developing writers. Compared to more tractable and constrained learning domains for ITS, writing is an ill-defined domain because the features of effective writing are difficult to quantify and individual writers can employ diverse strategies to achieve similar goals. The development of an ITS in an ill-defined domain presents particular challenges regarding comprehensive instruction, modularized content, extended practice, and formative feedback. In this article, we describe how the development of W-Pal has uniquely addressed these concerns and present the results of a study assessing the feasibility of this system in high school English classrooms. This study included 2 teachers and their 141 10th grade English class students who utilized W-Pal over a 6-month period during the academic year. Log-file analyses showed that students used all aspects of W-Pal, but activity and engagement was uneven throughout the year and decreased over time. Essay scores improved over time and surveys indicated that students perceived the lessons, games, and feedback as beneficial. However, specific aspects of the learning environment were critiqued as annoying, challenging, or lacking specificity. Overall, the results suggest that the system was generally well-received by the students but also offer insights for the development of ITSs in ill-defined domains.
---
paper_title: Changes in Reading Strategies as a Function of Reading Training: A Comparison of Live and Computerized Training

paper_content:

The purpose of this study was to compare the relative effectiveness of live (SERT) and computer-based (iSTART) reading strategy training. Prior to and after training, participants read scientific texts and self-explained after each sentence. They also answered comprehension questions. Students showed improvement in the quality of their self-explanations and in the performance on the comprehension questionsasa function of both live and computer-based training. However, there were some differences in response to iSTART training as a function of reading skill. Specifically, less skilled readers improved their performance on text-based questions, but not bridging questions. The opposite was found for skilled readers. These results indicate that computer-based, reading-skills training is effective, but different readers may improve at different levels of comprehension.
---
paper_title: The Evolution of an Automated Reading Strategy Tutor: From the Classroom to a Game-Enhanced Automated System

paper_content:

The implementation of effective pedagogical software is difficult to achieve. In this chapter we describe one possible solution to this problem, the evolutionary development of an Intelligent Tutoring System (ITS). This development process typically involves establishing training practices, developing automated instruction, and then amending motivational elements. While this development cycle can take years for completion because each step requires an iterative process of both execution and evaluation, it also has a greater chance of success. We illustrate such a cycle in this chapter in the evolution of an intelligent tutoring and gaming environment [i.e., interactive Strategy Trainer for Active Reading and Thinking-Motivationally Enhanced (iSTART-ME)] from an ITS (i.e., iSTART), which was originally conceived and tested as a human-delivered intervention (i.e., SERT).
---
paper_title: Improving Adolescent Students' Reading Comprehension with Istart:

paper_content:

This study examines the benefits of reading strategy training on adolescent readers' comprehension of science text. Training was provided via an automated reading strategy trainer called the Interactive Strategy Trainer for Active Reading and Thinking (iSTART), which is an interactive reading strategy trainer that utilizes animated agents to provide reading strategy instruction. Half of the participants were provided with iSTART while the others (control) were given a brief demonstration of how to self-explain text. All of the students then self-explained a text about heart disease and answered text-based and bridging-inference questions. Both iSTART training and prior knowledge of reading strategies significantly contributed to the quality of self-explanations and comprehension. Adolescents with less prior knowledge about reading strategies performed significantly better on text-based questions if they received iSTART training. Conversely, for high-strategy knowledge students, iSTART improved comprehensi...
---
paper_title: The wisdom of practice: Lessons learned from the study of highly effective tutors.

paper_content:

Publisher Summary This chapter describes the wisdom of practice that explains the lessons learned from the study of highly effective tutors. This chapter presents that in the 21st century, tutoring remains the ideal of education. The tutorial is inherently individualized. This individualization, in turn, permits the tutor to elicit from each student a much higher level of on-task attention and effort. It is, in addition, a virtual prerequisite for the high levels of both immediacy and interactivity that also characterize the tutorial process. Thus, in an individual tutorial, both knowledge of results and other forms of feedback and instruction are received by students. Highly effective or “expert” tutors are then identified on the basis of their actual degree of observable success, across a number of different tutees, in promoting student learning and motivation. The tutoring sessions conducted by the highly effective tutors are analyzed from a number of perspectives, and are contrasted with tutoring sessions conducted by less experienced or by equally experienced but objectively less successful tutors. The tutors share a generally Socratic approach, in the sense that tutors seek to draw as much as possible from the student and to impose as little as possible of themselves on the student. Finally, the goal of the analyses is to begin to identify the goals, strategies, and specific techniques that might contribute to the success of an individual tutorial.
---
paper_title: Inducing and Tracking Confusion with Contradictions during Complex Learning

paper_content:

Cognitive disequilibrium and its affiliated affective state of confusion have been found to positively correlate with learning, presumably due to the effortful cognitive activities that accompany their experience. Although confusion naturally occurs in several learning contexts, we hypothesize that it can be induced and scaffolded to increase learning opportunities. We addressed the possibility of confusion induction in a study where learners engaged in trialogues on research methods concepts with animated tutor and student agents. Confusion was induced by staging disagreements and contradictions between the animated agents, and then inviting the human learners to provide their opinions. Self-reports of confusion indicated that the contradictions were successful at inducing confusion in the minds of the learners. A second, more objective, method of tracking learners' confusion consisted of analyzing learners' performance on forced-choice questions that were embedded after contradictions. This measure was also found to be revealing of learners' underlying confusion. The contradictions alone did not result in enhanced learning gains. However, when confusion had been successfully induced, learners who were presented with contradictions did show improved learning compared to a no-contradiction control. Theoretical and applied implications along with possible future directions are discussed.
---
paper_title: Self-regulated Learning with MetaTutor: Advancing the Science of Learning with MetaCognitive Tools

paper_content:

The key to understanding complex learning with advanced learning technologies (e.g., hypermedia) lies in our ability to comprehend the temporal deployment of students’ cognitive, metacognitive, motivational, and affective processes. Our chapter will focus on critically analyzing the use of mixed-method approaches to analyze the complex nature of self-regulated learning (SRL) during hypermedia learning. We will use examples from our own research (e.g., Azevedo 2008, Recent innovations in educational technology that facilitate student learning (pp. 127–156); Azevedo & Witherspoon, in press, Handbook of metacognition in education) and that of others (e.g., Biswas et al., 2005; Schwartz et al., in press; Winne & Nesbitt, in press, Handbook of metacognition in education) to present and discuss the strengths and weaknesses in using mixed methods to capture, model, trace, and infer the unfolding SRL processes during learning with nonlinear, multirepresentational computerized environments. The chapter will focus on the methods, and quantitative and qualitative analyses used to converge product data (e.g., learning outcomes), process data (e.g., think-aloud data), and log-file data collected during learning, develop coding schemes to categorize and infer the deployment of SRL processes, and the use of computational tools to examine learners’ behaviors and navigation paths. Lastly, we will present a theoretical model that integrates the various topics presented in this chapter that will guide future research and educational practices for fostering students’ SRL with hypermedia environments.
---
paper_title: Improving students ’ help-seeking skills using metacognitive feedback in an intelligent tutoring system

paper_content:

The present research investigated whether immediate metacognitive feedback on students’ help-seeking errors can help students acquire better help-seeking skills. The Help Tutor, an intelligent tutor agent for help seeking, was integrated into a commercial tutoring system for geometry, the Geometry Cognitive Tutor. Study 1, with 58 students, found that the real-time assessment of students’ help-seeking behavior correlated with other independent measures of help seeking, and that the Help Tutor improved students’ help-seeking behavior while learning Geometry with the Geometry Cognitive Tutor. Study 2, with 67 students, evaluated more elaborated support that included, in addition to the Help Tutor, also help-seeking instruction and support for self-assessment. The study replicated the effect found in Study 1. It was also found that the improved help-seeking skills transferred to learning new domain-level content during the month following the intervention, while the helpseeking support was no longer in effect. Implications for metacognitive tutoring are discussed. 2010 Elsevier Ltd. All rights reserved.
---
paper_title: Operation ARA: A Computerized Learning Game that Teaches Critical Thinking and Scientific Reasoning.

paper_content:

Abstract Operation ARA (Acquiring Research Acumen) is a computerized learning game that teaches critical thinking and scientific reasoning. It is a valuable learning tool that utilizes principles from the science of learning and serious computer games. Students learn the skills of scientific reasoning by engaging in interactive dialogs with avatars. They are tutored by avatars with tutoring sessions that vary depending on how well students have responded to questions about the material they are learning. Students also play a jeopardy-like game against a feisty avatar to identify flaws in research and then generate their own questions to determine the quality of different types of research. The research examples are taken from psychology, biology, and chemistry to help students transfer the thinking skills across domains of knowledge. Early results show encouraging learning gains.
---
paper_title: Skill Diaries: Improve Student Learning in an Intelligent Tutoring System with Periodic Self-Assessment

paper_content:

According to Self-Regulated Learning theories, self-assessment by students can facilitate in-depth reflection and help direct effective self-regulated learning. Yet, not much work has investigated the relation between students’ self-assessment and learning outcomes in Intelligent Tutoring Systems (ITSs). This paper investigates this relation with classrooms using the Geometry Cognitive Tutor. We designed a paper-based skill diary that helps students take advantage of the tutor’s Open Learner Model to self-assess their problem-solving skills periodically, and investigated whether it can support students’ self-assessment and learning. In an experiment with 122 high school students, students in the experimental group were prompted periodically to fill out the skill diaries, whereas the control group answered general questions that did not involve active self-assessment. The experimental group performed better on the post-test, and the skill diaries helped lower-performing students to significantly improve their learning outcomes and self-assessment accuracy. This work is among the first empirical studies that successfully establish the beneficial role of self-assessment in students’ learning of problem-solving tasks in ITSs.
---
paper_title: The effectiveness of pedagogical agents' prompting and feedback in facilitating co-adapted learning with metatutor

paper_content:

Co-adapted learning involves complex, dynamically unfolding interactions between human and artificial pedagogical agents (PAs) during learning with intelligent systems. In general, these interactions lead to effective learning when (1) learners correctly monitor and regulate their cognitive and metacognitive processes in response to internal (e.g., accurate metacognitive judgments followed by the selection of effective learning strategies) and external (e.g., response to agents' prompting and feedback) conditions, and (2) pedagogical agents can adequately and correctly detect, track, model, and foster learners' self-regulatory processes. In this study, we tested the effectiveness of PAs' prompting and feedback on learners' self-regulated learning about the human circulatory system with MetaTutor, an adaptive, multi-agent learning environment. Sixty-nine (N=69) undergraduates learned about the topic with MetaTutor, during a 2-hour session under one of three conditions: prompt and feedback (PF), prompt-only (PO), and no prompt (NP) condition. The PF condition received timely prompts from several pedagogical agents to deploy various SRL processes and received immediate directive feedback concerning the deployment of the processes. The PO condition received the same timely prompts, without feedback. Finally, the NP condition learned without assistance from the agents. Results indicate that those in the PF condition had significantly higher learning efficiency scores than those in both the PO and control conditions. In addition, log-file data provided evidence of the effectiveness of the PA's timely scaffolding and feedback in facilitating learners' (in the PF condition) metacognitive monitoring and regulation during learning.
---
paper_title: SEEK Web tutor: fostering a critical stance while exploring the causes of volcanic eruption

paper_content:

We investigated the impact of a Web tutor on college students’ critical stance and learning while exploring Web pages on science. Critical stance is an aspect of self-regulated learning that emphasizes the need to evaluate the truth and relevance of information as the learner engages in systematic inquiry to answer challenging questions. The Web tutor is called SEEK, an acronym for Source, Evidence, Explanation, and Knowledge. The SEEK Tutor was designed to promote a critical stance through several facilities in a computer environment: spoken hints on a mock Google™ search page, on-line ratings on the reliability of particular Web sites, and a structured note-taking facility that prompted them to reflect on the quality of particular Web sites. We conducted two experiments that trained students how to take a critical stance and that tracked their behavior while exploring Web pages on plate tectonics to research the causes of the volcanic eruption of Mt. St. Helens. The SEEK Tutor did improve critical stance, as manifested in essays on the causes of the volcanic eruption, and did yield learning gains for some categories of information (compared with comparison conditions). However, many measures were unaffected by either the presence of the SEEK Tutor or by prior training on critical stance. We anticipate that robust improvements on critical stance and learning will require more training and/or some expert feedback and interactive scaffolding of critical stance in the context of specific examples.
---
paper_title: The Deep-Level-Reasoning-Question Effect: The Role of Dialogue and Deep-Level-Reasoning Questions During Vicarious Learning

paper_content:

We investigated the impact of dialogue and deep-level-reasoning questions on vicarious learning in 2 studies with undergraduates. In Experiment 1, participants learned material by interacting with AutoTutor or by viewing 1 of 4 vicarious learning conditions: a noninteractive recorded version of the AutoTutor dialogues, a dialogue with a deep-level-reasoning question preceding each sentence, a dialogue with a deep-level-reasoning question preceding half of the sentences, or a monologue. Learners in the condition where a deep-level-reasoning question preceded each sentence significantly outperformed those in the other 4 conditions. Experiment 2 included the same interactive and noninteractive recorded condition, along with 2 vicarious learning conditions involving deep-level-reasoning questions. Both deep-level-reasoning-question conditions significantly outperformed the other conditions. These findings provide evidence that deep-level-reasoning questions improve vicarious learning.
---
paper_title: Achieving affective impact: Visual emotive communication in lifelike pedagogical agents

paper_content:

Lifelike animated agents for knowledge-based learning environments can provide timely, customized advice to support learners' problem-solving activities. By drawing on a rich repertoire of emotive behaviors to exhibit contextually appropriate facial expressions and emotive gestures, these agents could exploit the visual channel to more effectively communicate with learners. To address these issues, this article proposes the emotive-kinesthetic behavior sequencing framework for dynamically sequencing lifelike pedagogical agents' full-body emotive expression. By exploiting a rich behavior space populated with emotive behaviors and structured by pedagogical speech act categories, a behavior sequencing engine operates in realtime to select and assemble contextually appropriate expressive behaviors. This framework has been implemented in a lifelike pedagogical agent, C OSMO, who exhibits full-body emotive behaviors in response to learners' problem-solving activities.
---
paper_title: Operation ARA: A Computerized Learning Game that Teaches Critical Thinking and Scientific Reasoning.

paper_content:

Abstract Operation ARA (Acquiring Research Acumen) is a computerized learning game that teaches critical thinking and scientific reasoning. It is a valuable learning tool that utilizes principles from the science of learning and serious computer games. Students learn the skills of scientific reasoning by engaging in interactive dialogs with avatars. They are tutored by avatars with tutoring sessions that vary depending on how well students have responded to questions about the material they are learning. Students also play a jeopardy-like game against a feisty avatar to identify flaws in research and then generate their own questions to determine the quality of different types of research. The research examples are taken from psychology, biology, and chemistry to help students transfer the thinking skills across domains of knowledge. Early results show encouraging learning gains.
---
paper_title: Operation ARIES!: A Serious Game for Teaching Scientific Inquiry

paper_content:

Operation ARIES! is a serious game that teaches critical thinking about scientific inquiry. The player must help to identify aliens on Earth who are intentionally publishing bad research. The game combines aspects of video games and intelligent tutors in which the player holds conversations with animated agents using natural language. The player first takes a training course with a virtual trainee, followed by a module in which the player identifies flaws in research cases. In the third and final module, the player interviews suspected alien scientists on their research. Operation ARIES! is designed for high school seniors and adults.
---
paper_title: Steve: an animated pedagogical agent for procedural training in virtual environments

paper_content:

Virtual reality can broaden the types of interaction between students and computer tutors. As in conventional simulation-based training, the computer can watch students practice tasks, responding to questions and offering advice. However, immersive virtual environments also allow the computer tutor to inhabit the virtual world with the student. Unlike previous, disembodied computer tutors, such a "pedagogical agent" can "physically" collaborate with students, enabling new types of interaction. To illustrate the possibilities, this paper describes Steve, a pedagogical agent for virtual environments that helps students learn procedural tasks. After providing an overview of Steve's capabilities, the paper focuses on the benefits and challenges of graphically representing Steve in the virtual environment.
---
paper_title: Building Intelligent Interactive Tutors: Student-Centered Strategies for Revolutionizing e-Learning

paper_content:

Computers have transformed every facet of our culture, most dramatically communication, transportation, finance, science, and the economy. Yet their impact has not been generally felt in education due to lack of hardware, teacher training, and sophisticated software. Another reason is that current instructional software is neither truly responsive to student needs nor flexible enough to emulate teaching. The more instructional software can reason about its own teaching process, know what it is teaching, and which method to use for teaching, the greater is its impact on education. ::: ::: Building Intelligent Interactive Tutors discusses educational systems that assess a student's knowledge and are adaptive to a student's learning needs. Dr. Woolf taps into 20 years of research on intelligent tutors to bring designers and developers a broad range of issues and methods that produce the best intelligent learning environments possible, whether for classroom or life-long learning. The book describes multidisciplinary approaches to using computers for teaching, reports on research, development, and real-world experiences, and discusses intelligent tutors, web-based learning systems, adaptive learning systems, intelligent agents and intelligent multimedia. ::: ::: *Combines both theory and practice to offer most in-depth and up-to-date treatment of intelligent tutoring systems available ::: *Presents powerful drivers of virtual teaching systems, including cognitive science, artificial intelligence, and the Internet ::: *Features algorithmic material that enables programmers and researchers to design building components and intelligent systems
---
paper_title: Guru: a computer tutor that models expert human tutors

paper_content:

We present Guru, an intelligent tutoring system for high school biology that has conversations with students, gestures and points to virtual instructional materials, and presents exercises for extended practice. Guru's instructional strategies are modeled after expert tutors and focus on brief interactive lectures followed by rounds of scaffolding as well as summarizing, concept mapping, and Cloze tasks. This paper describes the Guru session and presents learning outcomes from an in-school study comparing Guru, human tutoring, and classroom instruction. Results indicated significant learning gains for students in the Guru and human tutoring conditions compared to classroom controls.
---
paper_title: AutoTutor : A simulation of a human tutor Action

paper_content:

AutoTutor is a computer tutor that simulates the discourse patterns and pedagogical strategies of a typical human tutor. AutoTutor is designed to assist college students in learning the fundamentals of hardware, operating systems, and the Internet in an introductory computer literacy course. Most tutors in school systems are not highly trained in tutoring techniques and have only a modest expertise on the tutoring topic, but they are surprisingly effective in producing learning gains in students. We have dissected the discourse and pedagogical strategies these unskilled tutors exhibit by analyzing approximately 100 hours of naturalistic tutoring sessions. These mechanisms are implemented in AutoTutor. AutoTutor presents questions and problems from a curriculum script, attempts to comprehend learner contributions that are entered by keyboard, formulates dialog moves that are sensitive to the learner's contributions (such as short feedback, pumps, prompts, elaborations, corrections, and hints), and delivers the dialog moves with a talking head. AutoTutor has seven modules: a curriculum script, language extraction, speech act classification, latent semantic analysis, topic selection, dialog move generation, and a talking head.
---
paper_title: Teachable Agents: Combining Insights from Learning Theory and Computer Science

paper_content:

We discuss computer environments that invite students to learn by instructing “teachable agents” (TA’s) who venture forth and attempt to solve problems that require knowledge of disciplines such as mathematics, science or history. If the agents have been taught properly they solve the problems they confront; otherwise they need to be further educated. The TA’s have both a “knowledge dimension” and a “personality dimension” (e.g., some may be impetuous, not listen or collaborate well, need many examples to understand, etc.). This helps students focus on academic content plus the characteristics of “difficult agents” that interfere with learning. The paper briefly discusses learning by teaching, learning by programming, and relevant classroom research. This background helps identify key principles underlying teachable agent learning environments. The rest of the paper discusses a framework for instantiating these principles into a general teachable agent environment.
---
paper_title: Software agents: An overview

paper_content:

Agent software is a rapidly developing area of research. However, the overuse of the word ‘agent’ has tended to mask the fact that, in reality, there is a truly heterogeneous body of research being carried out under this banner. This overview paper presents a typology of agents. Next, it places agents in context, defines them and then goes on, inter alia, to overview critically the rationales, hypotheses, goals, challenges and state-of-the-art demonstrators of the various agent types in our typology. Hence, it attempts to make explicit much of what is usually implicit in the agents literature. It also proceeds to overview some other general issues which pertain to all the types of agents in the typology. This paper largely reviews software agents, and it also contains some strong opinions that are not necessarily widely accepted by the agent community.
---
paper_title: Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments

paper_content:

Recent years have witnessed the birth of a new paradigm for learning environments: animated pedagogical agents. These lifelike autonomous characters cohabit learning environments with students to create rich, face-to-face learning interactions. This opens up exciting new possibilities; for example, agents can demonstrate complex tasks, employ locomotion and gesture to focus students' attention on the most salient aspect of the task at hand, and convey emotional responses to the tutorial situation. Animated pedagogical agents offer great promise for broadening the bandwidth of tutorial communication and increasing learning environments' ability to engage and motivate students. This article sets forth the motivations behind animated pedagogical agents, describes the key capabilities they offer, and discusses the technical issues they raise. The discussion is illustrated with descriptions of a number of animated agents that represent the current state of the art.
---
paper_title: Factors that Influence the Perception of Feedback Delivered by a Pedagogical Agent

paper_content:

Three experiments were conducted to assess the relative importance of speech parameters and facial expressions in the delivery of feedback by a pedagogical agent. In Experiment 1, we manipulated linguistic form (i.e., positive, negative, or neutral terms), rate, pitch, pause, and emphasis. In Experiment 2, we manipulated eye size, mouth curve, brow height, and brow curve. In a third study, both speech parameters and facial expressions were manipulated. In all three experiments, the participants were asked to indicate how positive or negative the agent's feedback seemed to be. Across the studies, the variables collectively accounted for a significant amount of the variance. More specifically, the linguistic expressions and mouth curve emerged as significant predictors of the participants' ratings. This suggests that these two features should be implemented by developers wishing to provide appropriate feedback in their pedagogical agents.
---
paper_title: The Deep-Level-Reasoning-Question Effect: The Role of Dialogue and Deep-Level-Reasoning Questions During Vicarious Learning

paper_content:

We investigated the impact of dialogue and deep-level-reasoning questions on vicarious learning in 2 studies with undergraduates. In Experiment 1, participants learned material by interacting with AutoTutor or by viewing 1 of 4 vicarious learning conditions: a noninteractive recorded version of the AutoTutor dialogues, a dialogue with a deep-level-reasoning question preceding each sentence, a dialogue with a deep-level-reasoning question preceding half of the sentences, or a monologue. Learners in the condition where a deep-level-reasoning question preceded each sentence significantly outperformed those in the other 4 conditions. Experiment 2 included the same interactive and noninteractive recorded condition, along with 2 vicarious learning conditions involving deep-level-reasoning questions. Both deep-level-reasoning-question conditions significantly outperformed the other conditions. These findings provide evidence that deep-level-reasoning questions improve vicarious learning.
---
paper_title: Promoting vicarious learning of physics using deep questions with explanations

paper_content:

Two experiments explored the role of vicarious ''self'' explanations in facilitating student learning gains during computer-presented instruction. In Exp. 1, college students with low or high knowledge on Newton's laws were tested in four conditions: (a) monologue (M), (b) questions (Q), (c) explanation (E), and (d) question + explanation (Q + E). Those with low pre-experimental knowledge levels showed marginally significant yet consistently greater gains than those with high levels and condition Q + E outperformed the other three (M, Q, E). Among those with high knowledge, the Q + E presentations actually inhibited learning. In Exp. 2, high school physics students in standard and honors classes were studied during their introduction to Newton's laws. Brief (12 min) computer videos that introduced key Newtonian concepts preceded teacher presentations in seven daily sessions. Both standard and honors students who received Q + E presentations prior to regular classroom activities learned more in daily sessions than those who received either M or Q presentations. It was concluded that when key concepts are introduced in the context of deep questions along with explanations new learning was facilitated both in vicarious environments and in subsequent standard classroom activities.
---
paper_title: Improving the efficiency of dialogue in tutoring

paper_content:

Abstract The current studies investigated the efficient use of dialogue in intelligent tutoring systems that use natural language interaction. Such dialogues can be relatively time-consuming. This work addresses the question of how much dialogue is needed to produce significant learning gains. In Experiment 1, a full dialogue condition and a read-only control condition were compared with a mixed dialogue condition in which students engaged in full dialogue for half the problems followed by problems requiring only a limited engagement. We found that the mixed dialogue condition produced results as impressive as the full dialogue condition and took less time. Experiment 2 replicated these findings and further examined issues of time engaged in learning, quality of instruction, and learning gains. Overall, these results show that dialogue-based intelligent tutoring systems could be designed in a more efficient manner to maximize learning and minimize the cost of time-on-task.
---
paper_title: Coh-Metrix: Analysis of text on cohesion and language

paper_content:

Advances in computational linguistics and discourse processing have made it possible to automate many language- and text-processing mechanisms. We have developed a computer tool called Coh-Metrix, which analyzes texts on over 200 measures of cohesion, language, and readability. Its modules use lexicons, part-of-speech classifiers, syntactic parsers, templates, corpora, latent semantic analysis, and other components that are widely used in computational linguistics. After the user enters an English text, Coh-Metrix returns measures requested by the user. In addition, a facility allows the user to store the results of these analyses in data files (such as Text, Excel, and SPSS). Standard text readability formulas scale texts on difficulty by relying on word length and sentence length, whereas Coh-Metrix is sensitive to cohesion relations, world knowledge, and language and discourse characteristics.
---
paper_title: Teaching Students to Generate Questions: A Review of the Intervention Studies

paper_content:

This is a review of intervention studies in which students have been taught to generate questions as a means of improving their comprehension. Overall, teaching students the cognitive strategy of generating questions about the material they had read resulted in gains in comprehension, as measured by tests given at the end of the intervention. All tests were based on new material. The overall median effect size was 0.36 (64th percentile) when standardized tests were used and 0.86 (81st percentile) when experimenter-developed comprehension tests were used. The traditional skill-based instructional approach and the reciprocal teaching approach yielded similar results.
---
paper_title: Inducing and Tracking Confusion with Contradictions during Complex Learning

paper_content:

Cognitive disequilibrium and its affiliated affective state of confusion have been found to positively correlate with learning, presumably due to the effortful cognitive activities that accompany their experience. Although confusion naturally occurs in several learning contexts, we hypothesize that it can be induced and scaffolded to increase learning opportunities. We addressed the possibility of confusion induction in a study where learners engaged in trialogues on research methods concepts with animated tutor and student agents. Confusion was induced by staging disagreements and contradictions between the animated agents, and then inviting the human learners to provide their opinions. Self-reports of confusion indicated that the contradictions were successful at inducing confusion in the minds of the learners. A second, more objective, method of tracking learners' confusion consisted of analyzing learners' performance on forced-choice questions that were embedded after contradictions. This measure was also found to be revealing of learners' underlying confusion. The contradictions alone did not result in enhanced learning gains. However, when confusion had been successfully induced, learners who were presented with contradictions did show improved learning compared to a no-contradiction control. Theoretical and applied implications along with possible future directions are discussed.
---
paper_title: Exploring the deep-level reasoning questions effect during vicarious learning among eighth to eleventh graders in the domains of computer literacy and Newtonian physics

paper_content:

This paper tested the deep-level reasoning questions effect in the domains of computer literacy between eighth and tenth graders and Newtonian physics for ninth and eleventh graders. This effect claims that learning is facilitated when the materials are organized around questions that invite deep-reasoning. The literature indicates that vicarious learners in college student populations show greater pretest to posttest learning gains when presented with deep-level reasoning questions before each content sentence, than when deep-level questions are omitted, or when learners interact with an intelligent tutoring system. This effect holds for vicarious learners across grade levels and domains.
---
paper_title: Animated Presentation of Pictorial and Concept Map Media in Biology

paper_content:

Intelligent tutoring systems are beginning to include more varied forms of media, but little is known about how to choose the appropriate media and whether or not it should be animated. This study used a 2 animated/static x 2 picture/concept map factorial design in order to evaluate the effect of animation and media type on conceptual knowledge, relational knowledge, and free recall. Learners on Mechanical Turk N = 208 were exposed to one of four conditions in which they viewed a modified Khan Academy video on cell parts. We found that animation induced higher learning gains when it comes to relational knowledge. For conceptual knowledge, animated concept maps outperformed animated pictures while static pictures produced slightly more learning than static concept maps. Our results indicate that using animations to slowly build complexity in visual displays is particularly important when the displays have a rich structure as in concept maps.
---
paper_title: Exploring Science: The Cognition and Development of Discovery Processes

paper_content:

Einstein said that "the whole of science is nothing more than a refinement of everyday thinking." David Klahr suggests that we now know enough about cognition -- and hence about everyday thinking -- to advance our understanding of scientific thinking. In this book he sets out to describe the cognitive and developmental processes that have enabled scientists to make the discoveries that comprise the body of information we call "scientific knowledge." Over the past decade Klahr and his colleagues have conducted extensive laboratory experiments in which they create discovery contexts, computer-based environments, to evoke the kind of thinking characteristic of scientific discovery in the "real world." In attempting to solve the problems posed by the discovery tasks, experiment participants (from preschoolers through university students, as well as laypersons) use many of the same higher-order cognitive processes used by practicing scientists. Through this work Klahr integrates two disparate approaches -- the content-based approach and the process-based approach -- to present a comprehensive model of the psychology of scientific discovery.
---
paper_title: Designing Learning by Teaching Agents: The Betty's Brain System

paper_content:

The idea that teaching others is a powerful way to learn is intuitively compelling and supported in the research literature. We have developed computer-based, domain-independent Teachable Agents that students can teach using a visual representation. The students query their agent to monitor their learning and problem solving behavior. This motivates the students to learn more so they can teach their agent to perform better. This paper presents a teachable agent called Betty's Brain that combines learning by teaching with self-regulated learning feedback to promote deep learning and understanding in science domains. A study conducted in a 5th grade science classroom compared three versions of the system: a version where the students were taught by an agent, a baseline learning by teaching version, and a learning by teaching version where students received feedback on self-regulated learning strategies and some domain content. In the other two systems, students received feedback primarily on domain content. Our results indicate that all three groups showed learning gains during a main study where students learnt about river ecosystems, but the two learning by teaching groups performed better than the group that was taught. These differences persisted in the transfer study, but the gap between the baseline learning by teaching and self-regulated learning group decreased. However, there are indications that self-regulated learning feedback better prepared students to learn in new domains, even when they no longer had access to the self-regulation environment.
---
paper_title: Circsim-tutor: an intelligent tutoring system for circulatory physiology

paper_content:

The aim of this research is to develop an intelligent tutoring system (ITS) which teaches students the causal relationships between the components of the circulatory physiology system and the complex behavior of the negative feedback system that stabilizes blood pressure. This system will accept natural language input from students and generate limited natural language explanations. It contains rules that identify the student's errors and build a “bug-based” student model. It uses tutoring rules to plan each response based on its model of the student and the dialog history so that it can tailor the dialog to fit the student's learning needs. The tutoring rule interpreter manages the dialog and determines strategy and tactics to achieve its educational goals.
---
paper_title: Unsupervised Construction Of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources

paper_content:

We investigate unsupervised techniques for acquiring monolingual sentence-level paraphrases from a corpus of temporally and topically clustered news articles collected from thousands of web-based news sources. Two techniques are employed: (1) simple string edit distance, and (2) a heuristic strategy that pairs initial (presumably summary) sentences from different news stories in the same cluster. We evaluate both datasets using a word alignment algorithm and a metric borrowed from machine translation. Results show that edit distance data is cleaner and more easily-aligned than the heuristic data, with an overall alignment error rate (AER) of 11.58% on a similarly-extracted test set. On test data extracted by the heuristic strategy, however, performance of the two training sets is similar, with AERs of 13.2% and 14.7% respectively. Analysis of 100 pairs of sentences from each set reveals that the edit distance data lacks many of the complex lexical and syntactic alternations that characterize monolingual paraphrase. The summary sentences, while less readily alignable, retain more of the non-trivial alternations that are of greatest interest learning paraphrase relationships.
---
paper_title: Identification of Sentence-to-Sentence Relations Using a Textual Entailer

paper_content:

We show in this article how an approach developed for the task of recognizing textual entailment relations can be extended to identify paraphrase and elaboration relations. Entailment is a unidirectional relation between two sentences in which one sentence logically infers the other. There seems to be a close relation between entailment and two other sentence-to-sentence relations: elaboration and paraphrase. This close relation is discussed to theoretically justify the newly derived approaches. The proposed approaches use lexical, syntactic, and shallow negation handling. The proposed approaches offer significantly better results than several baselines. When compared to other paraphrase and elaboration approaches they produce similar or better results. We report results on several data sets: the Microsoft Research Paraphrase corpus, a benchmark for evaluating approaches to paraphrase identification, and a data set collected from high-school students' interactions with an intelligent tutoring system iSTART, which includes both paraphrase and elaboration utterances.
---
paper_title: Coh-Metrix: Analysis of text on cohesion and language

paper_content:

Advances in computational linguistics and discourse processing have made it possible to automate many language- and text-processing mechanisms. We have developed a computer tool called Coh-Metrix, which analyzes texts on over 200 measures of cohesion, language, and readability. Its modules use lexicons, part-of-speech classifiers, syntactic parsers, templates, corpora, latent semantic analysis, and other components that are widely used in computational linguistics. After the user enters an English text, Coh-Metrix returns measures requested by the user. In addition, a facility allows the user to store the results of these analyses in data files (such as Text, Excel, and SPSS). Standard text readability formulas scale texts on difficulty by relying on word length and sentence length, whereas Coh-Metrix is sensitive to cohesion relations, world knowledge, and language and discourse characteristics.
---
paper_title: Growing Semantic Grammars

paper_content:

A critical path in the development of natural language understanding (NLU) modules lies in the difficulty of defining a mapping from words to semantics: Usually it takes in the order of years of highly-skilled labor to develop a semantic mapping, e.g., in the form of a semantic grammar, that is comprehensive enough for a given domain. Yet, due to the very nature of human language, such mapping invariably fail to achieve full coverage on unseen data. Acknowledging the impossibility of stating a priori all the surface forms by which a concept can be expressed, we present GSG: an empathic computer system for the rapid deployment of NLU front-ends and their dynamic customization by non-expert end-users. Given a new domain for which an NLU front-end is to be developed, two stages are involved. In the authoring stage, GSG aids the developer in the construction of a simple domain model and a kernel analysis grammar. Then, in the run-time stage, GSG provides the end-user with an interactive environment in which the kernel grammar is dynamically extended. Three learning methods are employed in the acquisition of semantic mappings from unseen data: (i) parser predictions, (ii) hidden understanding model, and (iii) end-user paraphrases. A baseline version of GSG has been implemented and preliminary experiments show promising results.
---
paper_title: Discourse cohesion in text and tutorial dialogue

paper_content:

Discourse cohesion is presumably an important facilitator of comprehension when individuals read texts and hold conversations. This study investigated components of cohesion and language in different types of discourse about Newtonian physics: A textbook, textoids written by experimental psychologists, naturalistic tutorial dialogue between expert human tutors and college students, and AutoTutor tutorial dialogue between a computer tutor and students (AutoTutor is an animated pedagogical agent that helps students learn about physics by holding conversations in natural language). We analyzed the four types of discourse with Coh-Metrix, a software tool that measures discourse on different components of cohesion, language, and readability. The cohesion indices included co-reference, syntactic and semantic similarity, causal cohesion, incidence of cohesion signals (e.g., connectives, logical operators), and many other measures. Cohesion data were quite similar for the two forms of discourse in expository monologue (textbooks and textoids) and for the two types of tutorial dialogue (i.e., students interacting with human tutors and AutoTutor), but very different between the discourse of expository monologue and tutorial dialogue. Coh-Metrix was also able to detect subtle differences in the language and discourse of AutoTutor versus human tutoring.
---
paper_title: SEMILAR: A Semantic Similarity Toolkit for Assessing Students' Natural Language Inputs

paper_content:

We present in this demo SEMILAR, a SEMantic similarity toolkit. SEMILAR includes offers in one software environment several broad categories of semantic similarity methods: vectorial methods including Latent Semantic Analysis, probabilistic methods such as Latent Dirichlet Allocation, greedy lexical matching methods, optimal lexico-syntactic matching methods based on word-to-word similarities and syntactic dependencies with negation handling, kernel based methods, and some others. We will demonstrate during this demo presentation the efficacy of using SEMILAR to investigate and tune assessment algorithms for evaluating students’ natural language input based on data from the DeepTutor computer tutor.
---
paper_title: AutoTutor : A simulation of a human tutor Action

paper_content:

AutoTutor is a computer tutor that simulates the discourse patterns and pedagogical strategies of a typical human tutor. AutoTutor is designed to assist college students in learning the fundamentals of hardware, operating systems, and the Internet in an introductory computer literacy course. Most tutors in school systems are not highly trained in tutoring techniques and have only a modest expertise on the tutoring topic, but they are surprisingly effective in producing learning gains in students. We have dissected the discourse and pedagogical strategies these unskilled tutors exhibit by analyzing approximately 100 hours of naturalistic tutoring sessions. These mechanisms are implemented in AutoTutor. AutoTutor presents questions and problems from a curriculum script, attempts to comprehend learner contributions that are entered by keyboard, formulates dialog moves that are sensitive to the learner's contributions (such as short feedback, pumps, prompts, elaborations, corrections, and hints), and delivers the dialog moves with a talking head. AutoTutor has seven modules: a curriculum script, language extraction, speech act classification, latent semantic analysis, topic selection, dialog move generation, and a talking head.
---
paper_title: Using Latent Semantic Analysis to Evaluate the Contributions of Students in AutoTutor

paper_content:

AutoTutor is a fully automated computer tutor that assists students in learning about hardware, operating systems, and the Internet in an introductory computer literacy course. AutoTutor presents questions and problems from a curriculum script, attempts to comprehend learner contributions that are entered by keyboard, formulates dialog moves that are sensitive to the learner's contributions (such as prompts, elaborations, corrections, and hints), and delivers the dialog moves with a talking head. Latent Semantic Analysis (LSA) is a major component of the mechanism that evaluates the quality of student contributions in the tutorial dialog. LSA's evaluations of college students' answers to deep reasoning questions are equivalent to the evaluations provided by intermediate experts of computer literacy, but not as high as more accomplished experts in computer science. LSA is capable of discriminating different classes of student ability (good, vague, erroneous or mute students) and in tracking the quality of ...
---
paper_title: Deeper Natural Language Processing for Evaluating Student Answers in Intelligent Tutoring Systems

paper_content:

This paper addresses the problem of evaluating students' answers in intelligent tutoring environments with mixed-initiative dialogue by modelling it as a textual entailment problem. The problem of meaning representation and inference is a pervasive challenge in any integrated intelligent system handling communication. For intelligent tutorial dialogue systems, we show that entailment cases can be detected at various dialog turns during a tutoring session. We report the performance of a lexico-syntactic approach on a set of entailment cases that were collected from a previous study we conducted with AutoTutor.
---
paper_title: Opening the Door to Non-programmers: Authoring Intelligent Tutor Behavior by Demonstration

paper_content:

Intelligent tutoring systems are quite difficult and time intensive to develop. In this paper, we describe a method and set of software tools that ease the process of cognitive task analysis and tutor development by allowing the author to demonstrate, instead of programming, the behavior of an intelligent tutor. We focus on the subset of our tools that allow authors to create Pseudo Tutors that exhibit the behavior of intelligent tutors without requiring AI programming. Authors build user interfaces by direct manipulation and then use a Behavior Recorder tool to demonstrate alternative correct and incorrect actions. The resulting behavior graph is annotated with instructional messages and knowledge labels. We present some preliminary evidence of the effectiveness of this approach, both in terms of reduced development time and learning outcome. Pseudo Tutors have now been built for economics, analytic logic, mathematics, and language learning. Our data supports an estimate of about 25:1 ratio of development time to instruction time for Pseudo Tutors, which compares favorably to the 200:1 estimate for Intelligent Tutors, though we acknowledge and discuss limitations of such estimates.
---
paper_title: A framework of synthesizing tutoring conversation capability with web-based distance education courseware

paper_content:

Whereas existing learning environments on the Web lack high level interactivity, we have developed a human tutor-like tutorial conversation system for the Web that enhances educational courseware through mixed-initiative dialog with natural language processing. The conversational tutoring agent is composed of an animated tutor, a Latent Semantic Analysis (LSA) module, a database with curriculum scripts, and a dialog manager. As in the case of human tutors, the meaning of learner's contributions in natural language are compared with the content of expected answers to questions or problems specified in curriculum scripts. LSA is used to evaluate the conceptual matches between learner input and tutor expectations, whereas the dialog manager determines how the tutor adaptively responds to the learner by selecting content from the curriculum script. The integration of available courseware with the tutorial dialog system guarantees the reusability of existing Web tutorials with minimal effort in the modification of the curriculum script and LSA module. This development thereby simplifies the change into more valuable Web based training courseware.
---
paper_title: The development and analysis of tutorial dialogues in AutoTutor Lite

paper_content:

The goal of intelligent tutoring systems (ITS) that interact in natural language is to emulate the benefits that a well-trained human tutor provides to students, by interpreting student answers and appropriately responding in order to encourage elaboration. BRCA Gist is an ITS developed using AutoTutor Lite, a Web-based version of AutoTutor. Fuzzy-trace theory theoretically motivated the development of BRCA Gist, which engages people in tutorial dialogues to teach them about genetic breast cancer risk. We describe an empirical method to create tutorial dialogues and fine-tune the calibration of BRCA Gist’s semantic processing engine without a team of computer scientists. We created five interactive dialogues centered on pedagogic questions such as “What should someone do if she receives a positive result for genetic risk of breast cancer?” This method involved an iterative refinement process of repeated testing with different texts and successively making adjustments to the tutor’s expectations and settings in order to improve performance. The goal of this method was to enable BRCA Gist to interpret and respond to answers in a manner that best facilitated learning. We developed a method to analyze the efficacy of the tutor’s dialogues. We found that BRCA Gist’s assessment of participants’ answers was highly correlated with the quality of the answers found by trained human judges using a reliable rubric. The dialogue quality between users and BRCA Gist predicted performance on a breast cancer risk knowledge test completed after exposure to the tutor. The appropriateness of BRCA Gist’s feedback also predicted the quality of answers and breast cancer risk knowledge test scores.
---
paper_title: The Assessment of Knowledge, in Theory and in Practice

paper_content:

Adaptations from a book and many scholarly articles are presented. It reviews the main ideas of a novel theory for the assessment of a student's knowledge in a topic and gives details on a practical implementation in the form of a software system available on the Internet. The basic concept of the theory is the 'knowledge state,' which is the complete set of problems that an individual is capable of solving in a particular topic, such as elementary algebra. The task of the assessor consists in uncovering the particular state of the student being assessed. Even though the number of knowledge states for a topic may exceed several hundred thousand, these large numbers are well within the capacity of current home or school computers. The result of an assessment consists in two short lists of problems, which may be labelled: 'what the student can do' and 'what the student is ready to learn'. In the most important applications of the theory, these two lists specify the exact knowledge state of the individual being assessed.
---
paper_title: GnuTutor: An open source intelligent tutoring system

paper_content:

A single crystal is grown in accordance with a Czochralski method such that the time for passing through a temperature zone of 1150-1080 DEG C. is 20 minutes or less, or such that the length of a portion of the single crystal corresponding to the temperature zone of 1150-1080 DEG C. in the temperature distribution is 2.0 cm or less. Alternatively, the single crystal is grown such that the time for passing through a temperature zone of 1250-1200 DEG C. is 20 minutes or less, or such that the length of a portion of the single crystal corresponding to the temperature zone of 1250-1200 DEG C. in the temperature distribution is 2.0 cm or less. This method decreases both the density and size of so-called grown-in defects such as FPD (100 defects/cm2 or less), LSTD, and COP (10 defects/cm2 or less) to thereby enable efficient production of a single crystal having an excellent good chip yield (80% or greater) in terms of oxide dielectric breakdown voltage characteristics.
---
paper_title: A Motivationally Supportive Affect-Sensitive AutoTutor

paper_content:

This chapter describes a fully automated affect-sensitive Intelligent Tutoring System (ITS) called the Affective AutoTutor. AutoTutor is an ITS that helps students learn topics in Newtonian physics, computer literacy, and critical thinking via natural language dialogues that simulate the dialogue patterns observed in human–human tutoring. AutoTutor uses state-of-the-art natural language understanding mechanisms to model learners’ cognitive states and plan its dialogue moves in a manner that is sensitive to these states. While the original AutoTutor is sensitive to learners’ cognitive states, the affect-sensitive tutor is responsive to their affective states as well. This Affective tutor automatically detects learners’ boredom, confusion, and frustration by monitoring conversational cues, gross body language, and facial features. The sensed affective states guide the tutor’s responses in a manner that helps learners regulate their negative emotions. The tutor also synthesizes affect via the verbal content of its responses and the facial expressions and speech of an embodied pedagogical agent. An experiment comparing the affect-sensitive and nonaffective tutors indicated that the affective tutor improved learning for low-domain knowledge learners, particularly at deeper levels of comprehension.
---
paper_title: Content Matters: An Investigation of Feedback Categories within an ITS

paper_content:

The primary goal of this study was to investigate the role of feedback in an intelligent tutoring system (ITS) with natural language dialogue. One core component of tutorial dialogue is feedback, which carries the primary burden of informing students of their performance. AutoTutor is an ITS with tutorial dialogue that was developed at the University of Memphis. This article addresses the effectiveness of two types of feedback (content & progress) while college students interact with AutoTutor on conceptual physics. Content feedback provides qualitative information about the domain content and its accuracy as it is covered in a tutoring session. Progress feedback is a quantitative assessment of the student's advancement through the material being covered (i.e., how far the student has come and how much farther they have to go). A factorial design was used that manipulated the presence or absence of both feedback categories (content & progress). Each student interacted with one of four different versions of AutoTutor that varied the type of feedback. Data analyses showed significant effects of feedback on learning and motivational measures, supporting the notion that “content matters” and the adage “no pain, no gain.”
---
paper_title: Operation ARA: A Computerized Learning Game that Teaches Critical Thinking and Scientific Reasoning.

paper_content:

Abstract Operation ARA (Acquiring Research Acumen) is a computerized learning game that teaches critical thinking and scientific reasoning. It is a valuable learning tool that utilizes principles from the science of learning and serious computer games. Students learn the skills of scientific reasoning by engaging in interactive dialogs with avatars. They are tutored by avatars with tutoring sessions that vary depending on how well students have responded to questions about the material they are learning. Students also play a jeopardy-like game against a feisty avatar to identify flaws in research and then generate their own questions to determine the quality of different types of research. The research examples are taken from psychology, biology, and chemistry to help students transfer the thinking skills across domains of knowledge. Early results show encouraging learning gains.
---
paper_title: Operation ARIES!: A Serious Game for Teaching Scientific Inquiry

paper_content:

Operation ARIES! is a serious game that teaches critical thinking about scientific inquiry. The player must help to identify aliens on Earth who are intentionally publishing bad research. The game combines aspects of video games and intelligent tutors in which the player holds conversations with animated agents using natural language. The player first takes a training course with a virtual trainee, followed by a module in which the player identifies flaws in research cases. In the third and final module, the player interviews suspected alien scientists on their research. Operation ARIES! is designed for high school seniors and adults.
---
paper_title: AutoTutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back

paper_content:

We present AutoTutor and Affective AutoTutor as examples of innovative 21st century interactive intelligent systems that promote learning and engagement. AutoTutor is an intelligent tutoring system that helps students compose explanations of difficult concepts in Newtonian physics and enhances computer literacy and critical thinking by interacting with them in natural language with adaptive dialog moves similar to those of human tutors. AutoTutor constructs a cognitive model of students' knowledge levels by analyzing the text of their typed or spoken responses to its questions. The model is used to dynamically tailor the interaction toward individual students' zones of proximal development. Affective AutoTutor takes the individualized instruction and human-like interactivity to a new level by automatically detecting and responding to students' emotional states in addition to their cognitive states. Over 20 controlled experiments comparing AutoTutor with ecological and experimental controls such reading a textbook have consistently yielded learning improvements of approximately one letter grade after brief 30--60-minute interactions. Furthermore, Affective AutoTutor shows even more dramatic improvements in learning than the original AutoTutor system, particularly for struggling students with low domain knowledge. In addition to providing a detailed description of the implementation and evaluation of AutoTutor and Affective AutoTutor, we also discuss new and exciting technologies motivated by AutoTutor such as AutoTutor-Lite, Operation ARIES, GuruTutor, DeepTutor, MetaTutor, and AutoMentor. We conclude this article with our vision for future work on interactive and engaging intelligent tutoring systems.
---
paper_title: Guru: a computer tutor that models expert human tutors

paper_content:

We present Guru, an intelligent tutoring system for high school biology that has conversations with students, gestures and points to virtual instructional materials, and presents exercises for extended practice. Guru's instructional strategies are modeled after expert tutors and focus on brief interactive lectures followed by rounds of scaffolding as well as summarizing, concept mapping, and Cloze tasks. This paper describes the Guru session and presents learning outcomes from an in-school study comparing Guru, human tutoring, and classroom instruction. Results indicated significant learning gains for students in the Guru and human tutoring conditions compared to classroom controls.
---
paper_title: When Is Tutorial Dialogue More Effective Than Step-Based Tutoring?

paper_content:

It is often assumed that one-on-one dialogue with a tutor, which involves micro-steps, is more effective than conventional step-based tutoring. Although earlier research often has not supported this hypothesis, it may be because tutors often are not good at making micro-step decisions. In this paper, we compare a micro-step based NL-tutoring system that employs induced pedagogical policies, Cordillera, to a well-evaluated step-based ITS, Andes. Our overall conclusion is that the pairing of effective policies with a micro-step based system does significantly outperform a step-based system; however, there is no significant difference in the absence of effective policies. Moreover, while micro-step tutoring is more time-consuming, the findings still hold for five out of six learning performance measures when time on task is factored out.
---
paper_title: The effect of motivational learning companions on low achieving students and students with disabilities

paper_content:

We report the results of a randomized controlled evaluation of the effectiveness of pedagogical agents as providers of affective feedback. These digital learning companions were embedded in an intelligent tutoring system for mathematics, and were used by approximately one hundred students in two public high schools. Students in the control group did not receive the learning companions. Results indicate that low-achieving students—one third of whom have learning disabilities—had higher affective needs than their higher-achieving peers; they initially considered math problem-solving more frustrating, less exciting, and felt more anxious when solving math problems. However, after they interacted with affective pedagogical agents, low-achieving students improved their affective outcomes, e.g., reported reduced frustration and anxiety.
---
paper_title: The politeness effect: Pedagogical agents and learning outcomes

paper_content:

Pedagogical agent research seeks to exploit Reeves and Nass's media equation theory, which holds that users respond to interactive media as if they were social actors. Investigations have tended to focus on the media used to realize the pedagogical agent, e.g., the use of animated talking heads and voices, and the results have been mixed. This paper focuses instead on the manner in which a pedagogical agent communicates with learners, i.e., on the extent to which it exhibits social intelligence. A model of socially intelligent tutorial dialog was developed based on politeness theory, and implemented in an agent interface within an online learning system called virtual factory teaching system. A series of Wizard-of-Oz studies was conducted in which subjects either received polite tutorial feedback that promotes learner face and mitigates face threat, or received direct feedback that disregards learner face. The polite version yielded better learning outcomes, and the effect was amplified in learners who expressed a preference for indirect feedback, who had less computer experience, and who lacked engineering backgrounds. These results confirm the hypothesis that learners tend to respond to pedagogical agents as social actors, and suggest that research should focus less on the media in which agents are realized, and place more emphasis on the agent's social intelligence.
---
paper_title: Exploring the deep-level reasoning questions effect during vicarious learning among eighth to eleventh graders in the domains of computer literacy and Newtonian physics

paper_content:

This paper tested the deep-level reasoning questions effect in the domains of computer literacy between eighth and tenth graders and Newtonian physics for ninth and eleventh graders. This effect claims that learning is facilitated when the materials are organized around questions that invite deep-reasoning. The literature indicates that vicarious learners in college student populations show greater pretest to posttest learning gains when presented with deep-level reasoning questions before each content sentence, than when deep-level questions are omitted, or when learners interact with an intelligent tutoring system. This effect holds for vicarious learners across grade levels and domains.
---
paper_title: When Are Tutorial Dialogues More Effective Than Reading?

paper_content:

It is often assumed that engaging in a one-on-one dialogue with a tutor is more effective than listening to a lecture or reading a text. Although earlier experiments have not always supported this hypothesis, this may be due in part to allowing the tutors to cover different content than the noninteractive instruction. In 7 experiments, we tested the interaction hypothesis under the constraint that (a) all students covered the same content during instruction, (b) the task domain was qualitative physics, (c) the instruction was in natural language as opposed to mathematical or other formal languages, and (d) the instruction conformed with a widely observed pattern in human tutoring: Graesser, Person, and Magliano’s 5-step frame. In the experiments, we compared 2 kinds of human tutoring (spoken and computer mediated) with 2 kinds of natural-language-based computer tutoring (Why2-Atlas and Why2-AutoTutor) and 3 control conditions that involved studying texts. The results depended on whether the students’ preparation matched the content of the instruction. When novices (students who had not taken college physics) studied content that was written for intermediates (students who had taken college physics), then tutorial dialogue was reliably more beneficial than less interactive instruction, with large effect sizes. When novices studied material written for novices or intermediates studied material written for intermediates, then tutorial dialogue was not reliably more effective than the text-based control conditions.
---

Title: AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring

Section 1: The Rationale for AutoTutor: Tutoring Through Discourse
Description 1: Describe the rationale and benefits of using discourse in intelligent tutoring systems, emphasizing the ways AutoTutor leverages natural language to enhance learning activities.

Section 2: Human-Inspired Tutoring Strategies
Description 2: Discuss the foundation of AutoTutor's design in human tutoring protocols, focusing on the strategies inspired by human tutors which have been incorporated into AutoTutor.

Section 3: AutoTutor's 5-Step Tutoring Frame for Collaborative Reasoning
Description 3: Explain the 5-step tutoring frame observed in non-expert tutors and how this framework structured the original AutoTutor's dialog moves and interaction patterns.

Section 4: Deep Reasoning Questions
Description 4: Elaborate on the role of deep reasoning questions within AutoTutor dialogs, including how these questions are framed and their impact on student learning outcomes.

Section 5: Expert Strategies and Collaborative Lecturing
Description 5: Summarize the differences between expert and non-expert tutoring strategies and how these informed the design of systems like Guru, which employs collaborative lecturing.

Section 6: Learning Progressions
Description 6: Outline the concept and implementation of learning progressions in DeepTutor, focusing on the use of macro-adaptivity to track student understanding through structured knowledge pathways.

Section 7: Reading Comprehension Strategy Tutoring
Description 7: Review the adaptations of AutoTutor for reading comprehension, such as iSTART and Writing-Pal, highlighting specific strategies and how they differ from content-focused tutoring.

Section 8: Beyond Domain Information: Affect
Description 8: Examine the research on incorporating affective measures into AutoTutor, particularly how detecting and responding to emotions like frustration and engagement can influence learning.

Section 9: Learning about Thinking: Metacognition
Description 9: Discuss the role of AutoTutor descendants like SEEK and MetaTutor in teaching metacognitive and self-regulated learning skills, including their effectiveness and challenges.

Section 10: Pedagogical Agents and Complementary Media
Description 10: Describe the features and roles of animated pedagogical agents used in AutoTutor systems, as well as the integration of complementary media like simulations and concept maps for enriched tutoring experiences.

Section 11: AutoTutor Technologies: Semantic Analysis, Authoring, and Delivery
Description 11: Detail the technologies behind AutoTutor, including advancements in semantic analysis, the development of authoring tools, and various delivery platforms to enhance accessibility and scalability.

Section 12: Discussion: Key Findings from AutoTutor
Description 12: Summarize key findings from research on AutoTutor, focusing on learning gains, the effectiveness of different features, and insights into optimizing tutor designs for various learner needs.

Section 13: Future Directions
Description 13: Provide an overview of the future directions for AutoTutor research, including potential new roles for pedagogical agents, integration with existing educational platforms, and strategies for widespread adoption.