DOI: 10.1017/S000000000000000 Printed in the United Kingdom A Review of Case-Based Reasoning in Cognition-action Continuum: a step towards Bridging Symbolic and Non-symbolic Artificial Intelligence
---
paper_title: Efficacy of motor imagery in post-stroke rehabilitation: a systematic review

paper_content:

BackgroundEvaluation of how Motor Imagery and conventional therapy (physiotherapy or occupational therapy) compare to conventional therapy only in their effects on clinically relevant outcomes during rehabilitation of persons with stroke.DesignSystematic review of the literatureMethodsWe conducted an electronic database search in seven databases in August 2005 and also hand-searched the bibliographies of studies that we selected for the review.Two reviewers independently screened and selected all randomized controlled trials that compare the effects of conventional therapy plus Motor Imagery to those of only conventional therapy on stroke patients.The outcome measurements were: Fugl-Meyer Stroke Assessment upper extremity score (66 points) and Action Research Arm Test upper extremity score (57 points).Due to the high variability in the outcomes, we could not pool the data statistically.ResultsWe identified four randomized controlled trials from Asia and North America. The quality of the included studies was poor to moderate. Two different Motor imagery techniques were used (three studies used audiotapes and one study had occupational therapists apply the intervention). Two studies found significant effects of Motor Imagery in the Fugl-Meyer Stroke Assessment: Differences between groups amounted to 11.0 (1.0 to 21.0) and 3.2 (-4 to 10.3) respectively and in the Action Research Arm Test 6.1 (-6.2 to 18.4) and 15.8 (0.5 to 31.0) respectively. One study did not find a significant effect in the Fugl-Meyer Stroke Assessment and Color trail Test (p = 0.28) but in the task-related outcomes (p > 0.001).ConclusionCurrent evidence suggests that Motor imagery provides additional benefits to conventional physiotherapy or occupational therapy. However, larger and methodologically sounder studies should be conducted to assess the benefits of Motor imagery.
---
paper_title: Human cortical representations for reaching : mirror neurons for execution , observation , and imagery

paper_content:

Abstract We used functional magnetic resonance imaging (fMRI) to map the cortical representations of executed reaching, observed reaching, and imagined reaching in humans. Whereas previous studies have mostly examined hand actions related to grasping, hand–object interactions, or local finger movements, here we were interested in reaching only (i.e. the transport phase of the hand to a particular location in space), without grasping. We hypothesized that mirror neuron areas specific to reaching-related representations would be active in all three conditions. An overlap between executed, observed, and imagined reaching activations was found in dorsal premotor cortex as well as in the superior parietal lobe and the intraparietal sulcus, in accord with our hypothesis. Activations for observed reaching were more dorsal than activations typically reported in the literature for observation of hand–object interactions (grasping). Our results suggest that the mirror neuron system is specific to the type of hand action performed, and that these fronto-parietal activations are a putative human homologue of the neural circuits underlying reaching in macaques. The parietal activations reported here for executed, imagined, and observed reaching are also consistent with previous functional imaging studies on planned reaching and delayed pointing movements, and extend the proposed localization of human reach-related brain areas to observation as well as imagery of reaching.
---
paper_title: Intelligence without Representation

paper_content:

Brooks, R.A., Intelligence without representation, Artificial Intelligence 47 (1991) 139159. Artificial intelligence research has foundered on the issue of representation. When intelligence is approached in an incremental manner, with strict reliance on interfacing to the real world through perception and action, reliance on representation disappears. In this paper we outline our approach to incrementally building complete intelligent Creatures. The fundamental decomposition of the intelligent system is not into independent information processing units which must interface with each other via representations. Instead, the intelligent system is decomposed into independent and parallel activity producers which all interface directly to the world through perception and action, rather than interface to each other particularly much. The notions of central and peripheral systems evaporateeverything is both central and peripheral. Based on these principles we have built a very successful series of mobile robots which operate without supervision as Creatures in standard office environments.
---
paper_title: A review of contemporary ideomotor theory.

paper_content:

A framework for action planning, called ideomotor theory, suggests that actions are represented by their perceivable effects. Thus, any activation of the effect image, either endogenously or exogenously, will trigger the corresponding action. We review contemporary studies relating to ideomotor theory in which researchers have investigated various manipulations of action effects and how those effects acquire discriminative control over the actions. Evidence indicates that the knowledge about the relation between response and effect is still a critical component even when other factors, such as stimulus-response or response-response relations, are controlled. When consistent tone effects are provided after responses are made, performance in serial-reaction tasks is better than when the effects are random. Methodology in which acquisition and test stages are used with choice-reaction tasks shows that an action is automatically associated with its effect bilaterally and that anticipation of the effect facilitates action. Ideomotor phenomena include stimulus-response compatibility, in which the perceptual feature of the stimulus activates its corresponding action code when the stimulus itself resembles the effect codes. For this reason, other stimulus-driven action facilitation such as ideomotor action and imitation are treated as ideomotor phenomena and are reviewed. Ideomotor theory also implies that ongoing action affects perception of concurrent events, a topic which we review briefly. Issues concerning ideomotor theory are identified and evaluated. We categorize the range of ideomotor explanations into several groups by whether intermediate steps are assumed to complete sensorimotor transformation or not and by whether a general theoretical framework or a more restricted one is provided by the account.
---
paper_title: From Anomaly Reports to Cases

paper_content:

Creating case representations in unsupervised textual case-based reasoning applications is a challenging task because class knowledge is not available to aid selection of discriminatory features or to evaluate alternative system design configurations. Representation is considered as part of the development of a tool, called CAM , which supports an anomaly report processing task for the European Space Agency. Novel feature selection/extraction techniques are created which consider word co-occurrence patterns to calculate similarity between words. These are used together with existing techniques to create 5 different case representations. A new evaluation technique is introduced to compare these representations empirically, without the need for expensive, domain expert analysis. Alignment between the problem and solution space is measured at a local level and profiles of these local alignments used to evaluate the competenceof the system design.
---
paper_title: The Symbol Grounding Problem

paper_content:

A black powder flamespread tester is disclosed in which powder to be qualified is packed in a sample tube provided with a plurality of sight holes. Light sensors are disposed adjacent each sight hole and emit signals upon flame front passage of the sight hole. The sample tube is removably secured to an ignition head which contains an electrically actuated pyrotechnic squib. The squib discharges gases into a transition chamber from which the gases pass through orifices in an orifice plate, through a void volume into the black powder bed, which ignites the black powder. The ignition head is removably secured to a tester chamber in which the sample holder is received. Light sensors are housed in a separate fin which is removably received in an aperture in the tester chamber wall. After the squib gas ignites the powder bed, the output signal of each light sensor increases to saturation as the flame front passes its respective sight hole. The output signal from each light sensor is conditioned and summed to form one output signal. A plot of the summed output signal versus time is prepared, which shows the time of passage of the flame front through the black powder from each light sensor to the next. The flamespread rate is determined by dividing the known distance between light sensors by the time of passage of the flame front between respective light sensors.
---
paper_title: Adaptation Versus Retrieval Trade-Off Revisited: an Analysis of Boundary Conditions

paper_content:

In this paper we revisit the trade-off between adaptation and retrieval effort traditionally held as a principle in case-based reasoning. This principle states that the time needed for adaptation reduces with the time spent searching for an adequate case to be retrieved. In particular, if very little time is spent in retrieval, the adaptation effort will be high. Correspondingly, if the retrieval effort is high, the adaption effort is low. We analyzed this principle in two boundary conditions: (1) when very bad and (2) when highly capable adaptation procedures are used. We conclude that in the first boundary condition the adaptation-retrieval trade-off does not necessarily exist. We also claim that the second does not hold for a class of planning domains frequently used in the literature. To validate this claim, we performed experiments on two domains of this type.
---
paper_title: Case-Based Plan Adaptation: An Analysis and Review

paper_content:

Case-based planning (CBP) is a problem-solving method that uses a library of cases, where a case associates a past problem and goal description with a plan that solves the problem by achieving the goal. Given a new problem, CBP systems retrieve one or more cases that solve similar problems and adapt the retrieved cases' plans to achieve the new goal. Case retrieval involves intelligent search of the case library. Plan adaptation can include steps copied from the retrieved case plans and steps derived by other means, such as first-principles planning. Important relationships exist between the technology that measures similarity during case retrieval and the technology that performs effective adaptation during case reuse. However, we concentrate here on the adaptation process itself.
---
paper_title: Catastrophic Forgetting in Connectionist Networks: Causes, Consequences and Solutions

paper_content:

All natural cognitive systems, and, in particular, our own, gradually forget previously learned information. Consequently, plausible models of human cognition should exhibit similar patterns of gradual forgetting old information as new information is acquired. Only rarely (see Box 3) does new learning in natural cognitive systems completely disrupt or erase previously learned information. In other words, natural cognitive systems do not, in general, forget catastrophically. Unfortunately, however, this is precisely what occurs under certain circumstances in distributed connectionist networks. It turns out that the very features that give these networks their much-touted abilities to generalize, to function in the presence of degraded input, etc., are the root cause of catastrophic forgetting. The challenge is how to keep the advantages of distributed connectionist networks while avoiding the problem of catastrophic forgetting. In this article, we examine the causes, consequences and numerous solutions to the problem of catastrophic forgetting in neural networks. We consider how the brain might have overcome this problem and explore the consequences of this solution.
---
paper_title: MULTI-PLAN RETRIEVAL AND ADAPTATION IN AN EXPERIENCE-BASED AGENT

paper_content:

To address these issues, we propose a theory of experience-based agency which specifies how an agent with the ability to richly represent and store its experiences could remember those experiences with a context-sensitive, asynchronous memory, incorporate the relevant portions of those experiences into its reasoning on demand w ith integration mechanisms, and direct memory and reasoning through the use of a utility-based control mechanism. We have implemented this theory in the NICOLE multistrategy reasoning system and are currently using it to explore the problem of merging multiple past planning experiences. N ICOLE’S control system allows memory and reasoning to proceed in parallel; the asyn chronous and context-sensitive nature of that memory system allows NICOLE to return a “best guess” retrie val immediately and then to update that retrieval whenever new cues become available. But solving the problem of merging planning experiences requires more than just memory and control. We need mechanisms to integrate new retrieved plans into the planner’s current reasoning context whenever they are found; to en sure that those new retr ieved plans are useful, we need ways to use the planner’s current context to generate new cues that can help guide the memory system’s search. To solve these subproblems, we have developed the Multi-Plan Adaptor (MPA) algorithm, a novel method for merging partial-order plans in the context of casebased least-commitment planning. MPA allows the merging of arbitrary numbers of plans at any point during the adaptation process; it achieves this by dynamically extracting relevant case subparts and splicing those subpart s into partially completed plan s. MPA can also help guide retrieval by extracting intermediate goal statements from partial plans. In this chapter, we briefly review the properties of the real world that present challenges for the design of intelligent agents, examining in particular the need to combine past planning experiences. We then review our theory of experience-based agency and its implementation in the NICOLE system. We present the MPA algorithm, illustrate how it supports the merging of plans at any point during the adaptation pro cess, and describe its foundations in leastcommitment case-based planning. We then discuss how MPA can be integrated into various
---
paper_title: Case-Based Reuse of Software Examplets

paper_content:

We present a software tool for examplet reuse. We define examplets to be goal-directed snippets of source code, often written for tutorial purposes, that show how to use program library facilities to achieve some task. Our tool allows users to specify both their goal (in free text) and their 'situation' (the source code on which they are working). The system combines text retrieval and spreading activation through a semantic net representation of the source code.
---
paper_title: Motor Schema — Based Mobile Robot Navigation

paper_content:

Motor schemas serve as the basic unit of behavior specifica tion for the navigation of a mobile robot. They are multiple concurrent processes that operate in conjunction with asso ciated perceptual schemas and contribute independently to the overall concerted action of the vehicle. The motivation be hind the use of schemas for this domain is drawn from neuro- scientific, psychological, and robotic sources. A variant of the potential field method is used to produce the appropriate velocity and steering commands for the robot. Simulation re sults and actual mobile robot experiments demonstrate the feasibility of this approach.
---
paper_title: Case-Based Planning for Medical Diagnosis

paper_content:

In this paper we describe a case-based planner (BOLERO) developed for learning the procedure of a diagnosis in a medical expert system. A diagnostic plan is build according to the most recent information known about a patient. BOLERO has been tested in a real application of pneumonia diagnosis. Results show that BOLERO is able to acquire enough strategic knowledge to perform a diagnostic procedure with a high degree of success.
---
paper_title: Continuous Case-Based Reasoning

paper_content:

Abstract Case-based reasoning systems have traditionally been used to perform high-level reasoning in problem domains that can be adequately described using discrete, symbolic representations. However, many real-world problem domains, such as autonomous robotic navigation, are better characterized using continuous representations. Such problem domains also require continuous performance, such as on-line sensorimotor interaction with the environment, and continuous adaptation and learning during the performance task. This article introduces a new method for continuous case-based reasoning , and discusses its application to the dynamic selection, modification, and acquisition of robot behaviors in an autonomous navigation system, SINS (self-improving navigation system). The computer program and the underlying method are systematically evaluated through statistical analysis of results from several empirical studies. The article concludes with a general discussion of case-based reasoning issues addressed by this research.
---
paper_title: A Process Model of Cased-Based Reasoning in Problem Solving

paper_content:

Much of the problem solving done by both novices and experts uses "case-based" reasoning, or reasoning by analogy to previous similar cases. We explore the ways in which case-based reasoning can help in problem solving. According to our model, transfer of knowledge between cases is guided largely by the problem solving process itself. Our model shows the interactions between problem solving processes and memory for experience. Our computer program, called the MEDIATOR, illustrates case-based reasoning in interpreting and resolving common sense disputes.
---
paper_title: Indexing, Elaboration and Refinement: Incremental Learning of Explanatory Cases

paper_content:

This article describes how a reasoner can improve its understanding of an incompletely understood domain through the application of what it already knows to novel problems in that domain. Case-based reasoning is the process of using past experiences stored in the reasoner's memory to understand novel situations or solve novel problems. However, this process assumes that past experiences are well understood and provide good “lessons” to be used for future situations. This assumption is usually false when one is learning about a novel domain, since situations encountered previously in this domain might not have been understood completely. Furthermore, the reasoner may not even have a case that adequately deals with the new situation, or may not be able to access the case using existing indices. We present a theory of incremental learning based on the revision of previously existing case knowledge in response to experiences in such situations. The theory has been implemented in a case-based story understanding program that can (a) learn a new case in situations where no case already exists, (b) learn how to index the case in memory, and (c) incrementally refine its understanding of the case by using it to reason about new situations, thus evolving a better understanding of its domain through experience. This research complements work in case-based reasoning by providing mechanisms by which a case library can be automatically built for use by a case-based reasoning program.
---
paper_title: Case-Based Planning : Viewing Planning as a Memory Task

paper_content:

Much of the research at the Yale AI Lab relates to the investigation of memory. Learning and reminding form the basis of case-based reasoning in general and case-based planning (CBP) in particular. By remembering past successes and failures in planning, a planner may reduce much of its work to retrieving previously used plans and applying the to the situation at hand.
---
paper_title: Planning from Second Principles

paper_content:

Abstract Planning from second principles by reusing and modifying plans is one way of improving the efficiency of planning systems. In this paper, we study it in the general framework of deductive planning and develop a logical formalization of planning from second principles, which relies on a systematic decomposition of the planning process. Deductive inference processes with clearly defined semantics formalize each of the subtasks a second principles planner has to address. Plan modification, which comprises matching and adaptation tasks, is based on a deductive approach yielding provably correct modified plans. Description logics are introduced as query languages to plan libraries, which leads to a novel and efficient solution to the indexing problem in case-based reasoning. Apart from sequential plans, this approach enables a planner to reuse and modify complex plans containing control structures like conditionals and loops.
---
paper_title: Integrating CBR components within a Case-Based Planner

paper_content:

Multimodal reasoning systems can improve the effectiv.eness of reasoning by integrating multiple reasoning methods, each selectively applied to the tasks for which it is best-suited. One integration approach is to bring CBR into other systems, by developing case-based intelligent components (Riesbeck 1996) that collaborate with other reasoning systems, monitoring their successes and failures and suggesting solutions when prior experiences are relevant. Another approach is to bring other reasoning processes into a CBR system’s own architecture, to facilitate subprocesses of CBR such as case adaptation and similarity assessment. This paper describes a project combining both approaches: It discusses motivations and methods for a case-based components approach to integrating multiple reasoning modes, styles, and levels within a case-based reasoning system. The fundamental principle is for the system to use case-based components to learn by monitoring, capturing, and exploiting traces of multiple types of prior reasoning within the CBR system. The paper considers the benefits of this approach for improving CBR and its potential applicability to integrations in other contexts.
---
paper_title: Feature Weighting by Explaining Case-Based Planning Episodes

paper_content:

We present a similarity criterion based on feature weighting. Feature weights are recomputed dynamically according to the performance of cases during planning episodes. We will also present a novel algorithm to analyze and explain the performance of the retrieved cases and to determine the features whose weights need to be recomputed. Experiments show that the integration of our similarity criterion in a feature weighting model and our analysis algorithm improves the adaptability of the retrieved cases over a period of multiple problem solving episodes.
---
paper_title: Intelligence without Representation

paper_content:

Brooks, R.A., Intelligence without representation, Artificial Intelligence 47 (1991) 139159. Artificial intelligence research has foundered on the issue of representation. When intelligence is approached in an incremental manner, with strict reliance on interfacing to the real world through perception and action, reliance on representation disappears. In this paper we outline our approach to incrementally building complete intelligent Creatures. The fundamental decomposition of the intelligent system is not into independent information processing units which must interface with each other via representations. Instead, the intelligent system is decomposed into independent and parallel activity producers which all interface directly to the world through perception and action, rather than interface to each other particularly much. The notions of central and peripheral systems evaporateeverything is both central and peripheral. Based on these principles we have built a very successful series of mobile robots which operate without supervision as Creatures in standard office environments.
---
paper_title: Case-based reactive navigation: a method for on-line selection and adaptation of reactive robotic control parameters.

paper_content:

We present a new line of research investigating on-line adaptive reactive control mechanisms for autonomous intelligent agents. We discuss a case-based method for dynamic selection and modification of behavior assemblages for a navigational system. The case-based reasoning module is designed as an addition to a traditional reactive control system, and provides more flexible performance in novel environments without extensive high level reasoning that would otherwise slow the system down. The method is implemented in the ACBARR (case-based reactive robotic) system and evaluated through empirical simulation of the system on several different environments, including "box canyon" environments known to be problematic for reactive control systems in general.
---
paper_title: Learning behavioral parameterization using spatio-temporal case-based reasoning

paper_content:

This paper presents an approach to learning an optimal behavioral parameterization in the framework of a case-based reasoning methodology for autonomous navigation tasks. It is based on our previous work on a behavior-based robotic system that also employed spatio-temporal case-based reasoning in the selection of behavioral parameters but was not capable of learning new parameterizations. The present method extends the case-based reasoning module by making it capable of learning new and optimizing the existing cases where each case is a set of behavioral parameters. The learning process can either be a separate training process or be part of the mission execution. In either case, the robot learns an optimal parameterization of its behavior for different environments it encounters. The goal of this research is not only to automatically optimize the performance of the robot but also to avoid the manual configuration of behavioral parameters and the initial configuration of a case library, both of which require the user to possess good knowledge of robot behavior and the performance of numerous experiments. The presented method was integrated within a hybrid robot architecture and evaluated in extensive computer simulations, showing a significant increase in the performance over a nonadaptive system and a performance comparable to a non-learning CBR system that uses a hand-coded case library.
---
paper_title: Continuous Case-Based Reasoning

paper_content:

Abstract Case-based reasoning systems have traditionally been used to perform high-level reasoning in problem domains that can be adequately described using discrete, symbolic representations. However, many real-world problem domains, such as autonomous robotic navigation, are better characterized using continuous representations. Such problem domains also require continuous performance, such as on-line sensorimotor interaction with the environment, and continuous adaptation and learning during the performance task. This article introduces a new method for continuous case-based reasoning , and discusses its application to the dynamic selection, modification, and acquisition of robot behaviors in an autonomous navigation system, SINS (self-improving navigation system). The computer program and the underlying method are systematically evaluated through statistical analysis of results from several empirical studies. The article concludes with a general discussion of case-based reasoning issues addressed by this research.
---
paper_title: Spatio-temporal case-based reasoning for behavioral selection

paper_content:

Presents the application of a case-based reasoning approach to the selection and modification of behavioral assemblage parameters. The goal of this research is to achieve an optimal parameterization of robotic behaviors in run-time. This increases robot performance and makes a manual configuration of parameters unnecessary. The case-based reasoning module selects a set of parameters for an active behavioral assemblage in real-time. This set of parameters fits the environment better than hand-coded ones, and its performance is monitored providing feedback for a possible reselection of the parameters. The paper places a significant emphasis on the technical details of the case-based reasoning module and how it is integrated within a schema-based reactive navigation system. The paper also presents the results and evaluation of the system in both in simulation and real world robotic experiments.
---
paper_title: Intelligence without Representation

paper_content:

Brooks, R.A., Intelligence without representation, Artificial Intelligence 47 (1991) 139159. Artificial intelligence research has foundered on the issue of representation. When intelligence is approached in an incremental manner, with strict reliance on interfacing to the real world through perception and action, reliance on representation disappears. In this paper we outline our approach to incrementally building complete intelligent Creatures. The fundamental decomposition of the intelligent system is not into independent information processing units which must interface with each other via representations. Instead, the intelligent system is decomposed into independent and parallel activity producers which all interface directly to the world through perception and action, rather than interface to each other particularly much. The notions of central and peripheral systems evaporateeverything is both central and peripheral. Based on these principles we have built a very successful series of mobile robots which operate without supervision as Creatures in standard office environments.
---
paper_title: Case-based reactive navigation: a method for on-line selection and adaptation of reactive robotic control parameters.

paper_content:

We present a new line of research investigating on-line adaptive reactive control mechanisms for autonomous intelligent agents. We discuss a case-based method for dynamic selection and modification of behavior assemblages for a navigational system. The case-based reasoning module is designed as an addition to a traditional reactive control system, and provides more flexible performance in novel environments without extensive high level reasoning that would otherwise slow the system down. The method is implemented in the ACBARR (case-based reactive robotic) system and evaluated through empirical simulation of the system on several different environments, including "box canyon" environments known to be problematic for reactive control systems in general.
---
paper_title: Adaptation-guided retrieval: Questioning the similarity assumption in reasoning

paper_content:

Abstract One of the major assumptions in Artificial Intelligence is that similar experiences can guide future reasoning, problem solving and learning; what we will call, the similarity assumption. The similarity assumption is used in problem solving and reasoning systems when target problems are dealt with by resorting to a previous situation with common conceptual features. In this article, we question this assumption in the context of case-based reasoning (CBR). In CBR, the similarity assumption plays a central role when new problems are solved, by retrieving similar cases and adapting their solutions. The success of any CBR system is contingent on the retrieval of a case that can be successfully reused to solve the target problem. We show that it is often unwarranted to assume that the most similar case is also the most appropriate from a reuse perspective. We argue that similarity must be augmented by deeper, adaptation knowledge about whether a case can be easily modified to fit a target problem. We implement this idea in a new technique, called adaptation-guided retrieval (AGR), which provides a direct link between retrieval similarity and adaptation needs. This technique uses specially formulated adaptation knowledge, which, during retrieval, facilitates the computation of a precise measure of a case's adaptation requirements. In closing, we assess the broader implications of AGR and argue that it is just one of a growing number of methods that seek to overcome the limitations of the traditional similarity assumption in an effort to deliver more sophisticated and scalable reasoning systems.
---
paper_title: Retrieving and Reusing Game Plays for Robot Soccer

paper_content:

The problem of defining robot behaviors to completely address a large and complex set of situations is very challenging. We present an approach for robot's action selection in the robot soccer domain using Case-Based Reasoning techniques. A case represents a snapshot of the game at time t and the actions the robot should perform in that situation. We basically focus our work on the retrieval and reuse steps of the system, presenting the similarity functions and a planning process to adapt the current problem to a case. We present first results of the performance of the system under simulation and the analysis of the parameters used in the approach.
---
paper_title: A Case-Based Approach for Coordinated Action Selection in Robot Soccer

paper_content:

Designing coordinated robot behaviors in uncertain, dynamic, real-time, adversarial environments, such as in robot soccer, is very challenging. In this work we present a case-based reasoning approach for cooperative action selection, which relies on the storage, retrieval, and adaptation of example cases. We focus on cases of coordinated attacking passes between robots in the presence of the defending opponent robots. We present the case representation explicitly distinguishing between controllable and uncontrollable indexing features, corresponding to the positions of the team members and opponent robots, respectively. We use the symmetric properties of the domain to automatically augment the case library. We introduce a retrieval technique that weights the similarity of a situation in terms of the continuous ball positional features, the uncontrollable features, and the cost of moving the robots from the current situation to match the case controllable features. The case adaptation includes a best match between the positions of the robots in the past case and in the new situation. The robots are assigned an adapted position to which they move to maximize the match to the retrieved case. Case retrieval and reuse are achieved within the distributed team of robots through communication and sharing of own internal states and actions. We evaluate our approach, both in simulation and with real robots, in laboratory scenarios with two attacking robots versus two defending robots as well as versus a defender and a goalie. We show that we achieve the desired coordinated passing behavior, and also outperform a reactive action selection approach.
---
paper_title: Adaptation guided retrieval based on formal concept analysis

paper_content:

In previous papers [5, 4] we have proved the usefulness of Formal Concept Analysis (FCA) as an inductive technique that elicits knowledge embedded in a case library. The dependency knowledge implicitly contained in the case base is captured during the FCA process in the form of dependence rules among the attributes describing the cases. A substitution-based adaptation process is proposed that profits from these dependence rules since substituting an attribute may require to substitute dependant attributes. Dependence rules will guide an interactive query formulation process which favors retrieving cases where successful adaptations can be accomplished. In this paper we exemplify the use of FCA to help query formulation in an application to generate Spanish poetry versions of texts provided by the user.
---
paper_title: SOAR: an architecture for general intelligence

paper_content:

Abstract The ultimate goal of work in cognitive architecture is to provide the foundation for a system capable of general intelligent behavior. That is, the goal is to provide the underlying structure that would enable a system to perform the full range of cognitive tasks, employ the full range of problem solving methods and representations appropriate for the tasks, and learn about all aspects of the tasks and its performance on them. In this article we present SOAR, an implemented proposal for such an architecture. We describe its organizational principles, the system as currently implemented, and demonstrations of its capabilities.
---
paper_title: Case-Based Plan Recognition in Computer Games

paper_content:

In this paper we explore the use of case-based plan recognition to predict a player's actions in a computer game. The game we work with is the classic Space Invaders game and we show that case-based plan recognition can produce good prediction accuracy in real-time, working with a fairly simple game representation. Our evaluation suggests that a personalized plan library will produce better prediction accuracy but, for Space Invaders, good accuracy can be produced using a plan library derived from the game play of another player.
---
paper_title: Fast and robust incremental action prediction for interactive agents

paper_content:

The ability for a given agent to adapt on-line to better interact with another agent is a difficult and important problem. This problem becomes even more difficult when the agent to interact with is a human, because humans learn quickly and behave nondeterministically. In this paper, we present a novel method whereby an agent can incrementally learn to predict the actions of another agent (even a human), and thereby can learn to better interact with that agent. We take a case-based approach, where the behavior of the other agent is learned in the form of state‐action pairs. We generalize these cases either through continuous k-nearest neighbor, or a modified bounded minimax search. Through our case studies, our technique is empirically shown to require little storage, learn very quickly, and be fast and robust in practice. It can accurately predict actions several steps into the future. Our case studies include interactive virtual environments involving mixtures of synthetic agents and humans, with cooperative and/or competitive relationships.
---
paper_title: Online action adaptation in interactive computer games

paper_content:

Nonplayer characters (NPCs) in today's computer games lack the ability to adapt to situations that were not envisaged by the artificial intelligence (AI) programmer. This lack of adaptation produces lifeless characters that are prone to repetitive and predictable behavior. In this article, we present our work towards the development of an online learning and adaptation architecture for NPCs in first-person shooter (FPS) computer games. Our architecture builds upon incremental case-based approaches to modelling an observed entity, and makes a number of novel contributions. In particular, we develop a dual state representation to enhance case matching, and use adaptive k-d tree-based techniques to improve case storage and retrieval. The dual state representation allows more game features to be represented in the system, which enables observed behavior to be more accurately recorded and actions predicted. The system is applied to the Unreal Tournament using the GameBots API and evaluated in a number of different game scenarios. Our results show that the adaptation system can accurately predict a human player's actions and that our dual state representation enhances prediction. We also demonstrate that an adaptive k-d tree-based technique can be used online to maintain a balanced tree of observed cases.
---
paper_title: Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game

paper_content:

While several researchers have applied case-based reasoning techniques to games, only Ponsen and Spronck (2004) have addressed the challenging problem of learning to win real-time games. Focusing on Wargus, they report good results for a genetic algorithm that searches in plan space, and for a weighting algorithm (dynamic scripting) that biases subplan retrieval. However, both approaches assume a static opponent, and were not designed to transfer their learned knowledge to opponents with substantially different strategies. We introduce a plan retrieval algorithm that, by using three key sources of domain knowledge, removes the assumption of a static opponent. Our experiments show that its implementation in the Case-based Tactician (CaT) significantly outperforms the best among a set of genetically evolved plans when tested against random Wargus opponents. CaT communicates with Wargus through TIELT, a testbed for integrating and evaluating decision systems with simulators. This is the first application of TIELT. We describe this application, our lessons learned, and our motivations for future work.
---
paper_title: On-Line Case-Based Plan Adaptation for Real-Time Strategy Games

paper_content:

Traditional artificial intelligence techniques do not perform well in applications such as real-time strategy games because of the extensive search spaces which need to be explored. In addition, this exploration must be carried out on-line during performance time; it cannot be precomputed. We have developed on-line case-based planning techniques that are effective in such domains. In this paper, we extend our earlier work using ideas from traditional planning to inform the real-time adaptation of plans. In our framework, when a plan is retrieved, a plan dependency graph is inferred to capture the relations between actions in the plan. The plan is then adapted in real-time using its plan dependency graph. This allows the system to create and adapt plans in an efficient and effective manner while performing the task. The approach is evaluated using WARGUS, a well-known real-time strategy game.
---
paper_title: The mirror system, imitation, and the evolution of language

paper_content:

A dance class in Santa Fe, Sept. 25, 1999: The percussion is insistent. Dancers move in rows from the back of the hall towards the drummers at the front. From time to time, the mistress of the dance breaks the flow, and twice repeats a sequence of energetic dance moves. The dancers then move forward again, repeating her moves, more or less. Some do it well, others not so well. Imitation involves, in part, seeing the instructor's dance as set of familiar movements of shoulders, arms, hands, belly and legs. Many constituents are variants of familiar actions, rather than familiar actions themselves. Thus one must not only observe actions and their composition, but also novelties in the constituents and their variations. One must also perceive the overlapping and sequencing of all these moves and then remember the “coordinated control program” so constructed Probably, memory and perception are intertwined. As the dancers perform they both act out the recalled coordinated control program and tune it. By observing other dancers and synchronizing with their neighbors and the insistent percussion of the drummers, they achieve a collective representation that tunes their own, possibly departing from the instructor's original. At the same time, some dancers seem more or less skilled – some will omit a movement, or simplify it, others may replace it with their imagined equivalent. (One example: the instructor alternates touching her breast and moving her arm outwards. Most dancers move their arms in and out with no particular target.) Other changes are matters of motor rather than perceptual or mnemonic skill – not everyone can lean back as far as the instructor without losing balance. These are the ingredients of imitation.
---
paper_title: Forward models: Supervised learning with a distal teacher

paper_content:

Internal models of the environment have an important role to play in adaptive systems, in general, and are of particular importance for the supervised learning paradigm. In this article we demonstrate that certain classical problems associated with the notion of the “teacher” in supervised learning can be solved by judicious use of learned internal models as components of the adaptive system. In particular, we show how supervised learning algorithms can be utilized in cases in which an unknown dynamical system intervenes between actions and desired outcomes. Our approach applies to any supervised learning algorithm that is capable of learning in multilayer networks.
---
paper_title: DRAMA, a Connectionist Architecture for Control and Learning in Autonomous Robots

paper_content:

Adaptation to their environment is a fundamental capability for living agents, from which autonomous robots could also benefit. This work proposes a connectionist architecture, DRAMA, for dynamic control and learning of autonomous robots. DRAMA stands for dynamical recurrent associative memory architecture. It is a time-delay recurrent neural network, using Hebbian update rules. It allows learning of spatio-temporal regularities and time series in discrete sequences of inputs, in the face of an important amount of noise. The first part of this paper gives the mathematical description of the architecture and analyses theoretically and through numerical simulations its performance. The second part of this paper reports on the implementation of DRAMA in simulated and physical robotic experiments. Training and rehearsal of the DRAMA architecture is computationally fast and inexpensive, which makes the model particularly suitable for controlling ecomputationally-challengedi robots. In the experiments, we use a basic hardware system with very limited computational capability and show that our robot can carry out real time computation and on-line learning of relatively complex cognitive tasks. In these experiments, two autonomous robots wander randomly in a fixed environment, collecting information about its elements. By mutually associating information of their sensors and actuators, they learn about physical regularities underlying their experience of varying stimuli. The agents learn also from their mutual interactions. We use a teacher-learner scenario, based on mutual following of the two agents, to enable transmission of a vocabulary from one robot to the other.
---
paper_title: Hearing sounds, understanding actions: action representation in mirror neurons

paper_content:

Many object-related actions can be recognized by their sound. We found neurons in monkey premotor cortex that discharge when the animal performs a specific action and when it hears the related sound. Most of the neurons also discharge when the monkey observes the same action. These audiovisual mirror neurons code actions independently of whether these actions are performed, heard, or seen. This discovery in the monkey homolog of Broca9s area might shed light on the origin of language: audiovisual mirror neurons code abstract contents—the meaning of actions—and have the auditory access typical of human language to these contents.
---
paper_title: Mirror neurons and the simulation theory of mind-reading

paper_content:

A new class of visuomotor neuron has been recently discovered in the monkey’s premotor cortex: mirror neurons. These neurons respond both when a particular action is performed by the recorded monkey and when the same action, performed by another individual, is observed. Mirror neurons appear to form a cortical system matching observation and execution of goal-related motor actions. Experimental evidence suggests that a similar matching system also exists in humans. What might be the functional role of this matching system? One possible function is to enable an organism to detect certain mental states of observed conspecifics. This function might be part of, or a precursor to, a more general mind-reading ability. Two different accounts of mindreading have been suggested. According to ‘theory theory’, mental states are represented as inferred posits of a naive theory. According to ‘simulation theory’, other people’s mental states are represented by adopting their perspective: by tracking or matching their states with resonant states of one’s own. The activity of mirror neurons, and the fact that observers undergo motor facilitation in the same muscular groups as those utilized by target agents, are findings that accord well with simulation theory but would not be predicted by theory theory.
---
paper_title: Hierarchical attentive multiple models for execution and recognition of actions

paper_content:

According to the motor theories of perception, the motor systems of an observer are actively involved in the perception of actions when these are performed by a demonstrator. In this paper we review our computational architecture, HAMMER (Hierarchical Attentive Multiple Models for Execution and Recognition), where the motor control systems of a robot are organised in a hierarchical, distributed manner, and can be used in the dual role of (a) competitively selecting and executing an action, and (b) perceiving it when performed by a demonstrator. We subsequently demonstrate that such an arrangement can provide a principled method for the top-down control of attention during action perception, resulting in significant performance gains. We assess these performance gains under a variety of resource allocation strategies. c 2006 Elsevier B.V. All rights reserved.
---
paper_title: Imitation, mirror neurons and autism

paper_content:

Various deficits in the cognitive functioning of people with autism have been documented in recent years but these provide only partial explanations for the condition. We focus instead on an imitative disturbance involving difficulties both in copying actions and in inhibiting more stereotyped mimicking, such as echolalia. A candidate for the neural basis of this disturbance may be found in a recently discovered class of neurons in frontal cortex, 'mirror neurons' (MNs). These neurons show activity in relation both to specific actions performed by self and matching actions performed by others, providing a potential bridge between minds. MN systems exist in primates without imitative and 'theory of mind' abilities and we suggest that in order for them to have become utilized to perform social cognitive functions, sophisticated cortical neuronal systems have evolved in which MNs function as key elements. Early developmental failures of MN systems are likely to result in a consequent cascade of developmental impairments characterised by the clinical syndrome of autism. Crown Copyright (C) 2001 Published by Elsevier Science Ltd. All rights reserved.
---
paper_title: EEG evidence for mirror neuron dysfunction in autism spectrum disorders

paper_content:

Abstract Autism spectrum disorders (ASD) are largely characterized by deficits in imitation, pragmatic language, theory of mind, and empathy. Previous research has suggested that a dysfunctional mirror neuron system may explain the pathology observed in ASD. Because EEG oscillations in the mu frequency (8–13 Hz) over sensorimotor cortex are thought to reflect mirror neuron activity, one method for testing the integrity of this system is to measure mu responsiveness to actual and observed movement. It has been established that mu power is reduced (mu suppression) in typically developing individuals both when they perform actions and when they observe others performing actions, reflecting an observation/execution system which may play a critical role in the ability to understand and imitate others' behaviors. This study investigated whether individuals with ASD show a dysfunction in this system, given their behavioral impairments in understanding and responding appropriately to others' behaviors. Mu wave suppression was measured in ten high-functioning individuals with ASD and ten age- and gender-matched control subjects while watching videos of (1) a moving hand, (2) a bouncing ball, and (3) visual noise, or (4) moving their own hand. Control subjects showed significant mu suppression to both self and observed hand movement. The ASD group showed significant mu suppression to self-performed hand movements but not to observed hand movements. These results support the hypothesis of a dysfunctional mirror neuron system in high-functioning individuals with ASD.
---
paper_title: An Embodied Model for Sensorimotor Grounding and Grounding Transfer: Experiments With Epigenetic Robots

paper_content:

The grounding of symbols in computational models of linguistic abilities is one of the fundamental properties of psychologically plausible cognitive models. In this article, we present an embodied model for the grounding of language in action based on epigenetic robots. Epigenetic robotics is one of the new cognitive modeling approaches to modeling autonomous mental development. The robot model is based on an integrative vision of language in which linguistic abilities are strictly dependent on and grounded in other behaviors and skills. It uses simulated robots that learn through imitation the names of basic actions. Robots also learn higher order action concepts through the process of grounding transfer. The simulation demonstrates how new, higher order behavioral abilities can be autonomously built on previously grounded basic action categories following linguistic interaction with human users.
---
paper_title: Explaining facial imitation: a theoretical model

paper_content:

A long-standing puzzle in developmental psychology is how infants imitate gestures they cannot see themselves perform (facial gestures). Two critical issues are: (a) the metric infants use to detect cross-modal equivalences in human acts and (b) the process by which they correct their imitative errors. We address these issues in a detailed model of the mechanisms underlying facial imitation. The model can be extended to encompass other types of imitation. The model capitalizes on three new theoretical concepts. First, organ identification is the means by which infants relate parts of their own bodies to corresponding ones of the adult's. Second, body babbling (infants' movement practice gained through self-generated activity) provides experience mapping movements to the resulting body configurations. Third, organ relations provide the metric by which infant and adult acts are perceived in commensurate terms. In imitating, infants attempt to match the organ relations they see exhibited by the adults with those they feel themselves make. We show how development restructures the meaning and function of early imitation. We argue that important aspects of later social cognition are rooted in the initial cross-modal equivalence between self and other found in newborns. ©1997 John Wiley & Sons, Ltd.
---
paper_title: A unifying computational framework for motor control and social interaction

paper_content:

Recent empirical studies have implicated the use of the motor system during action observation, imitation and social interaction. In this paper, we explore the computational parallels between the processes that occur in motor control and in action observation, imitation, social interaction and theory of mind. In particular, we examine the extent to which motor commands acting on the body can be equated with communicative signals acting on other people and suggest that computational solutions for motor control may have been extended to the domain of social interaction.
---
paper_title: Localization of grasp representations in humans by PET: 1. Observation versus execution

paper_content:

Positron emission tomography (PET) was used to localize brain regions that are active during the observation of grasping movements. Normal, right-handed subjects were tested under three conditions. In the first, they observed grasping movements of common objects performed by the experimenter. In the second, they reached and grasped the same objects. These two conditions were compared with a third condition consisting of object observation. On the basis of monkey data, it was hypothesized that during grasping observation, activations should be present in the region of the superior temporal sulcus (STS) and in inferior area 6. The findings in humans demonstrated that grasp observation significantly activates the cortex of the middle temporal gyrus including that of the adjacent superior temporal sulcus (Brodmann's area 21) and the caudal part of the left inferior frontal gyrus (Brodmann's area 45). The possible functional homologies between these areas and the monkey STS region and frontal area F5 are discussed.
---
paper_title: Premotor cortex and the recognition of motor actions

paper_content:

Abstract In area F5 of the monkey premotor cortex there are neurons that discharge both when the monkey performs an action and when he observes a similar action made by another monkey or by the experimenter. We report here some of the properties of these ‘mirror’ neurons and we propose that their activity ‘represents’ the observed action. We posit, then, that this motor representation is at the basis of the understanding of motor events. Finally, on the basis of some recent data showing that, in man, the observation of motor actions activate the posterior part of inferior frontal gyrus, we suggest that the development of the lateral verbal communication system in man derives from a more ancient communication system based on recognition of hand and face gestures.
---
paper_title: Imitation of facial and manual gestures by human neonates.

paper_content:

Infants between 12 and 21 days of age can imitate both facial and manual gestures; this behavior cannot be explained in terms of either conditioning or innate releasing mechanisms. Such imitation implies that human neonates can equate their own unseen behaviors with gestures they see others perform.
---
paper_title: Is imitation learning the route to humanoid robots?

paper_content:

Summary This review investigates two recent developments in artificial intelligence and neural computation: learning from imitation and the development of humanoid robots. It will be postulated that the study of imitation learning offers a promising route to gain new insights into mechanisms of perceptual motor control that could ultimately lead to the creation of autonomous humanoid robots. Imitation learning focuses on three important issues: efficient motor learning, the connection between action and perception, and modular motor control in form of movement primitives. It will be reviewed how research on representations of, and functional connections between action and perception have contributed to our understanding of motor acts of other beings. The recent discovery that some areas in the primate brain are active during both movement perception and execution has provided a hypothetical neural basis of imitation. Computational approaches to imitation learning will also be described, initially from the perspective of traditional AI and robotics, but also from the perspective of neural network models and statistical learning research. Parallels and differences between biological and computational approaches to imitation will be highlighted and an overview of current projects that actually employ imitation learning for humanoid robots will be given.
---
paper_title: Brain areas selective for both observed and executed movements

paper_content:

When observing a particular movement a subset of movement-selective visual and visuomotor neurons are active in the observer's brain, forming a representation of the observed movement. Similarly, when executing a movement a subset of movement-selective motor and visuomotor neurons are active, forming a representation of the executed movement. In this study we used an fMRI-adaptation protocol to assess cortical response selectivity to observed and executed movements simultaneously. Subjects freely played the rock–paper–scissors game against a videotaped opponent, sometimes repeatedly observing or executing the same movement on subsequent trials. Numerous brain areas exhibited adaptation (repetition suppression) during either repeated observations or repeated executions of the same movement. A subset of areas exhibited an overlap of both effects, containing neurons with selective responses for both executed and observed movements. We describe the function of these movement representation areas in the context of the human mirror system, which is expected to respond selectively to both observed and executed movements.
---
paper_title: Asymmetric fMRI adaptation reveals no evidence for mirror neurons in humans

paper_content:

Neurons in macaque ventral premotor cortex and inferior parietal lobe discharge during both the observation and the execution of motor acts. It has been claimed that these so-called mirror neurons form the basis of action understanding by matching the visual input with the corresponding motor program (direct matching). Functional magnetic resonance imaging (fMRI) adaptation can be used to test the direct matching account of action recognition by determining whether putative mirror neurons show adaptation for repeated motor acts independently of whether they are observed or executed. An unambiguous test of the hypothesis requires that the motor acts be meaningless to ensure that any adaptation effect is directly because of movement recognition/motor execution and not contextually determined inferences. We found adaptation for motor acts that were repeatedly observed or repeatedly executed. We also found adaptation for motor acts that were first observed and then executed, as would be expected if a previously seen act primed the subsequent execution of that act. Crucially, we found no signs of adaptation for motor acts that were first executed and then observed. Failure to find cross-modal adaptation for executed and observed motor acts is not compatible with the core assumption of mirror neuron theory, which holds that action recognition and understanding are based on motor simulation.
---
paper_title: Functional magnetic resonance imaging examination of two modular architectures for switching multiple internal models.

paper_content:

An internal model is a neural mechanism that can mimic the input-output properties of a controlled object such as a tool. Recent research interests have moved on to how multiple internal models are learned and switched under a given context of behavior. Two representative computational models for task switching propose distinct neural mechanisms, thus predicting different brain activity patterns in the switching of internal models. In one model, called the mixture-of-experts architecture, switching is commanded by a single executive called a "gating network," which is different from the internal models. In the other model, called the MOSAIC (MOdular Selection And Identification for Control), the internal models themselves play crucial roles in switching. Consequently, the mixture-of-experts model predicts that neural activities related to switching and internal models can be temporally and spatially segregated, whereas the MOSAIC model predicts that they are closely intermingled. Here, we directly examined the two predictions by analyzing functional magnetic resonance imaging activities during the switching of one common tool (an ordinary computer mouse) and two novel tools: a rotated mouse, the cursor of which appears in a rotated position, and a velocity mouse, the cursor velocity of which is proportional to the mouse position. The switching and internal model activities temporally and spatially overlapped each other in the cerebellum and in the parietal cortex, whereas the overlap was very small in the frontal cortex. These results suggest that switching mechanisms in the frontal cortex can be explained by the mixture-of-experts architecture, whereas those in the cerebellum and the parietal cortex are explained by the MOSAIC model.
---
paper_title: Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorders

paper_content:

To examine mirror neuron abnormalities in autism, high-functioning children with autism and matched controls underwent fMRI while imitating and observing emotional expressions. Although both groups performed the tasks equally well, children with autism showed no mirror neuron activity in the inferior frontal gyrus (pars opercularis). Notably, activity in this area was inversely related to symptom severity in the social domain, suggesting that a dysfunctional 'mirror neuron system' may underlie the social deficits observed in autism.
---
paper_title: Internal Models in the Cerebellum

paper_content:

This review will focus on the possibility that the cerebellum contains an internal model or models of the motor apparatus. Inverse internal models can provide the neural command necessary to achieve some desired trajectory. First, we review the necessity of such a model and the evidence, based on the ocular following response, that inverse models are found within the cerebellar circuitry. Forward internal models predict the consequences of actions and can be used to overcome time delays associated with feedback control. Secondly, we review the evidence that the cerebellum generates predictions using such a forward model. Finally, we review a computational model that includes multiple paired forward and inverse models and show how such an arrangement can be advantageous for motor learning and control.
---
paper_title: Forward Models for Physiological Motor Control.

paper_content:

Based on theoretical and computational studies it has been suggested that the central nervous system (CNS) internally simulates the behaviour of the motor system in planning, control and learning. Such an internal "forward" model is a representation of the motor system that uses the current state of the motor system and motor command to predict the next state. We will outline the uses of such internal models for solving several fundamental computational problems in motor control and then review the evidence for their existence and use by the CNS. Finally we speculate how the location of an internal model within the CNS may be identified. Copyright 1996 Elsevier Science Ltd.
---
paper_title: Conditioned adaptation to prismatic displacement with a tone as the conditional stimulus

paper_content:

Adaptation to prismatic displacement was conditioned to a tone in 72 min of training by employing Taylor’s alternation training technique. The alternation consisted of two training conditions. In one, S was exposed to the prism and tone; in the other, S was exposed to neither. After training, the pointing to a visual target test measured more aftereffects of adaptation when the tone was present than when it was absent. Conditioning was obtained in two testing situations: (1) with the training goggles still worn by S, and (2) with the goggles removed.
---
paper_title: Predicting the consequences of our own actions: The role of sensorimotor context estimation

paper_content:

During self-generated movement it is postulated that an efference copy of the descending motor command, in conjunction with an internal model of both the motor system and environment, enables us to predict the consequences of our own actions (von Helmholtz, 1867; Sperry, 1950; von Holst, 1954; Wolpert, 1997). Such a prediction is evident in the precise anticipatory modulation of grip force seen when one hand pushes on an object gripped in the other hand (Johansson and Westling, 1984; Flanagan and Wing, 1933). Here we show that self-generation is not in itself sufficient for such a prediction. We used two robots to simulate virtual objects held in one hand and acted on by the other. Precise predictive grip force modulation of the restraining hand was highly dependent on the sensory feedback to the hand producing the load. The results show that predictive modulation requires not only that the movement is self-generated, but also that the efference copy and sensory feedback are consistent with a specific context; in this case, the manipulation of a single object. We propose a novel computational mechanism whereby the CNS uses multiple internal models, each corresponding to a different sensorimotor context, to estimate the probability that the motor system is acting within each context.
---
paper_title: A unifying computational framework for motor control and social interaction

paper_content:

Recent empirical studies have implicated the use of the motor system during action observation, imitation and social interaction. In this paper, we explore the computational parallels between the processes that occur in motor control and in action observation, imitation, social interaction and theory of mind. In particular, we examine the extent to which motor commands acting on the body can be equated with communicative signals acting on other people and suggest that computational solutions for motor control may have been extended to the domain of social interaction.
---
paper_title: Adaptive representation of dynamics during learning of a motor task

paper_content:

We investigated how the CNS learns to control movements in different dynamical conditions, and how this learned behavior is represented. In particular, we considered the task of making reaching movements in the presence of externally imposed forces from a mechanical environment. This environment was a force field produced by a robot manipulandum, and the subjects made reaching movements while holding the end-effector of this manipulandum. Since the force field significantly changed the dynamics of the task, subjects9 initial movements in the force field were grossly distorted compared to their movements in free space. However, with practice, hand trajectories in the force field converged to a path very similar to that observed in free space. This indicated that for reaching movements, there was a kinematic plan independent of dynamical conditions. The recovery of performance within the changed mechanical environment is motor adaptation. In order to investigate the mechanism underlying this adaptation, we considered the response to the sudden removal of the field after a training phase. The resulting trajectories, named aftereffects, were approximately mirror images of those that were observed when the subjects were initially exposed to the field. This suggested that the motor controller was gradually composing a model of the force field, a model that the nervous system used to predict and compensate for the forces imposed by the environment. In order to explore the structure of the model, we investigated whether adaptation to a force field, as presented in a small region, led to aftereffects in other regions of the workspace. We found that indeed there were aftereffects in workspace regions where no exposure to the field had taken place; that is, there was transfer beyond the boundary of the training data. This observation rules out the hypothesis that the subject9s model of the force field was constructed as a narrow association between visited states and experienced forces; that is, adaptation was not via composition of a look-up table. In contrast, subjects modeled the force field by a combination of computational elements whose output was broadly tuned across the motor state space. These elements formed a model that extrapolated to outside the training region in a coordinate system similar to that of the joints and muscles rather than end-point forces. This geometric property suggests that the elements of the adaptive process represent dynamics of a motor task in terms of the intrinsic coordinate system of the sensors and actuators.
---
paper_title: Control of mental activities by internal models in the cerebellum

paper_content:

In recent years the role of the cerebellum in the control of non-motor functions has been acknowledged. Here, Ito elaborates on his hypothesis of internal-model control of mental functions and describes recent findings that provide supporting evidence for the theory.
---
paper_title: Is the cerebellum a smith predictor

paper_content:

The motor system may use intemal predictive models of the motor apparatus to achieve better control than would be possible by negative feedback. Several theories have proposed that the cerebellum may form these predictive representations, In this article, we review these theories and try to unify them by reference to an engineering control model known as a Smith Pre dictor. We suggest that the cerebellum forms two types of internal model. One model is a forward predictive model of the motor apparatus (e.g. , limb and muscle), providing a rapid prediction of the sensory consequences of each movement. The second model is of the time delays in the control loop (due to receptor and effec- tor delays, axona) conductances, and cognitive processing de- lays). This model delays a copy of the rapid prediction so that it can be compared in temporal register with actual sensory feedback from the movement. The result of this comparison is used both to correct for errors in perf.ormance and as a training signal to learn the first model We discuss evidence that the cerebellum could form both of these models and suggest that the cerebellum may hold at least two separate Smith Predictors. One, in the lateral cerebellum, would predict the movement outcome in visual, ego- centric, or peripersonaJ coordinates. Another, in the intermediate cerebellum, would predict the consequences in motor coordinates. Generalization of the Smith Predictor theory is discussed in ) ight of cerebellar involvement in nonmotor control systems, including autonomic functions and cognition
---
paper_title: Improving Performance Of Case-Based Classification Using Context-Based Relevance

paper_content:

Classification involves associating instances with particular classes by maximizing intra-class similarities and minimizing inter-class similarities. Thus, the way similarity among instances is measured is crucial for the success of the system. In case-based reasoning, it is assumed that similar problems have similar solutions. The case-based approach to classification is founded on retrieving cases from the case base that are similar to a given problem, and associating the problem with the class containing the most similar cases. Similarity-based retrieval tools can advantageously be used in building flexible retrieval and classification systems. Case-based classification uses previously classified instances to label unknown instances with proper classes. Classification accuracy is affected by the retrieval process – the more relevant the instances used for classification, the greater the accuracy. The paper presents a novel approach to case-based classification. The algorithm is based on a notion of similarity assessment and was developed for supporting flexible retrieval of relevant information. Case similarity is assessed with respect to a given context that defines constraints for matching. Context relaxation and restriction is used for controlling the classification accuracy. The validity of the proposed approach is tested on real-world domains, and the system's performance, in terms of accuracy and scalability, is compared to that of other machine learning algorithms.
---
paper_title: Improving Performance Of Case-Based Classification Using Context-Based Relevance

paper_content:

Classification involves associating instances with particular classes by maximizing intra-class similarities and minimizing inter-class similarities. Thus, the way similarity among instances is measured is crucial for the success of the system. In case-based reasoning, it is assumed that similar problems have similar solutions. The case-based approach to classification is founded on retrieving cases from the case base that are similar to a given problem, and associating the problem with the class containing the most similar cases. Similarity-based retrieval tools can advantageously be used in building flexible retrieval and classification systems. Case-based classification uses previously classified instances to label unknown instances with proper classes. Classification accuracy is affected by the retrieval process – the more relevant the instances used for classification, the greater the accuracy. The paper presents a novel approach to case-based classification. The algorithm is based on a notion of similarity assessment and was developed for supporting flexible retrieval of relevant information. Case similarity is assessed with respect to a given context that defines constraints for matching. Context relaxation and restriction is used for controlling the classification accuracy. The validity of the proposed approach is tested on real-world domains, and the system's performance, in terms of accuracy and scalability, is compared to that of other machine learning algorithms.
---
paper_title: Improving Performance Of Case-Based Classification Using Context-Based Relevance

paper_content:

Classification involves associating instances with particular classes by maximizing intra-class similarities and minimizing inter-class similarities. Thus, the way similarity among instances is measured is crucial for the success of the system. In case-based reasoning, it is assumed that similar problems have similar solutions. The case-based approach to classification is founded on retrieving cases from the case base that are similar to a given problem, and associating the problem with the class containing the most similar cases. Similarity-based retrieval tools can advantageously be used in building flexible retrieval and classification systems. Case-based classification uses previously classified instances to label unknown instances with proper classes. Classification accuracy is affected by the retrieval process – the more relevant the instances used for classification, the greater the accuracy. The paper presents a novel approach to case-based classification. The algorithm is based on a notion of similarity assessment and was developed for supporting flexible retrieval of relevant information. Case similarity is assessed with respect to a given context that defines constraints for matching. Context relaxation and restriction is used for controlling the classification accuracy. The validity of the proposed approach is tested on real-world domains, and the system's performance, in terms of accuracy and scalability, is compared to that of other machine learning algorithms.
---
paper_title: A Case-Based Approach for Coordinated Action Selection in Robot Soccer

paper_content:

Designing coordinated robot behaviors in uncertain, dynamic, real-time, adversarial environments, such as in robot soccer, is very challenging. In this work we present a case-based reasoning approach for cooperative action selection, which relies on the storage, retrieval, and adaptation of example cases. We focus on cases of coordinated attacking passes between robots in the presence of the defending opponent robots. We present the case representation explicitly distinguishing between controllable and uncontrollable indexing features, corresponding to the positions of the team members and opponent robots, respectively. We use the symmetric properties of the domain to automatically augment the case library. We introduce a retrieval technique that weights the similarity of a situation in terms of the continuous ball positional features, the uncontrollable features, and the cost of moving the robots from the current situation to match the case controllable features. The case adaptation includes a best match between the positions of the robots in the past case and in the new situation. The robots are assigned an adapted position to which they move to maximize the match to the retrieved case. Case retrieval and reuse are achieved within the distributed team of robots through communication and sharing of own internal states and actions. We evaluate our approach, both in simulation and with real robots, in laboratory scenarios with two attacking robots versus two defending robots as well as versus a defender and a goalie. We show that we achieve the desired coordinated passing behavior, and also outperform a reactive action selection approach.
---
paper_title: Improving Performance Of Case-Based Classification Using Context-Based Relevance

paper_content:

Classification involves associating instances with particular classes by maximizing intra-class similarities and minimizing inter-class similarities. Thus, the way similarity among instances is measured is crucial for the success of the system. In case-based reasoning, it is assumed that similar problems have similar solutions. The case-based approach to classification is founded on retrieving cases from the case base that are similar to a given problem, and associating the problem with the class containing the most similar cases. Similarity-based retrieval tools can advantageously be used in building flexible retrieval and classification systems. Case-based classification uses previously classified instances to label unknown instances with proper classes. Classification accuracy is affected by the retrieval process – the more relevant the instances used for classification, the greater the accuracy. The paper presents a novel approach to case-based classification. The algorithm is based on a notion of similarity assessment and was developed for supporting flexible retrieval of relevant information. Case similarity is assessed with respect to a given context that defines constraints for matching. Context relaxation and restriction is used for controlling the classification accuracy. The validity of the proposed approach is tested on real-world domains, and the system's performance, in terms of accuracy and scalability, is compared to that of other machine learning algorithms.
---
paper_title: Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication

paper_content:

We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.
---
paper_title: Improving Performance Of Case-Based Classification Using Context-Based Relevance

paper_content:

Classification involves associating instances with particular classes by maximizing intra-class similarities and minimizing inter-class similarities. Thus, the way similarity among instances is measured is crucial for the success of the system. In case-based reasoning, it is assumed that similar problems have similar solutions. The case-based approach to classification is founded on retrieving cases from the case base that are similar to a given problem, and associating the problem with the class containing the most similar cases. Similarity-based retrieval tools can advantageously be used in building flexible retrieval and classification systems. Case-based classification uses previously classified instances to label unknown instances with proper classes. Classification accuracy is affected by the retrieval process – the more relevant the instances used for classification, the greater the accuracy. The paper presents a novel approach to case-based classification. The algorithm is based on a notion of similarity assessment and was developed for supporting flexible retrieval of relevant information. Case similarity is assessed with respect to a given context that defines constraints for matching. Context relaxation and restriction is used for controlling the classification accuracy. The validity of the proposed approach is tested on real-world domains, and the system's performance, in terms of accuracy and scalability, is compared to that of other machine learning algorithms.
---
paper_title: A Case-Based Approach for Coordinated Action Selection in Robot Soccer

paper_content:

Designing coordinated robot behaviors in uncertain, dynamic, real-time, adversarial environments, such as in robot soccer, is very challenging. In this work we present a case-based reasoning approach for cooperative action selection, which relies on the storage, retrieval, and adaptation of example cases. We focus on cases of coordinated attacking passes between robots in the presence of the defending opponent robots. We present the case representation explicitly distinguishing between controllable and uncontrollable indexing features, corresponding to the positions of the team members and opponent robots, respectively. We use the symmetric properties of the domain to automatically augment the case library. We introduce a retrieval technique that weights the similarity of a situation in terms of the continuous ball positional features, the uncontrollable features, and the cost of moving the robots from the current situation to match the case controllable features. The case adaptation includes a best match between the positions of the robots in the past case and in the new situation. The robots are assigned an adapted position to which they move to maximize the match to the retrieved case. Case retrieval and reuse are achieved within the distributed team of robots through communication and sharing of own internal states and actions. We evaluate our approach, both in simulation and with real robots, in laboratory scenarios with two attacking robots versus two defending robots as well as versus a defender and a goalie. We show that we achieve the desired coordinated passing behavior, and also outperform a reactive action selection approach.
---
paper_title: On-Line Case-Based Plan Adaptation for Real-Time Strategy Games

paper_content:

Traditional artificial intelligence techniques do not perform well in applications such as real-time strategy games because of the extensive search spaces which need to be explored. In addition, this exploration must be carried out on-line during performance time; it cannot be precomputed. We have developed on-line case-based planning techniques that are effective in such domains. In this paper, we extend our earlier work using ideas from traditional planning to inform the real-time adaptation of plans. In our framework, when a plan is retrieved, a plan dependency graph is inferred to capture the relations between actions in the plan. The plan is then adapted in real-time using its plan dependency graph. This allows the system to create and adapt plans in an efficient and effective manner while performing the task. The approach is evaluated using WARGUS, a well-known real-time strategy game.
---

Title: A Review of Case-Based Reasoning in Cognition-action Continuum: a step towards Bridging Symbolic and Non-symbolic Artificial Intelligence
Section 1: Introduction
Description 1: This section introduces the background, context, and primary hypothesis of the paper, emphasizing the integration of cognition and action in AI research and the potential of CBR to bridge symbolic and non-symbolic AI.

Section 2: Main characteristics and design tasks of CBR systems
Description 2: This section discusses the essential characteristics of CBR systems and the crucial tasks that system designers need to address, including case content, population, similarity measurement, and case adaptation.

Section 3: Rationale behind the "CBR in cognition-action continuum"
Description 3: This section explains the theoretical justification for applying CBR across the cognition-action continuum and the continuum's impact on action selection and planning.

Section 4: CBR in dynamic and real-time domains
Description 4: This section reviews the application of CBR in dynamic and real-time domains like navigation, robot soccer, and real-time strategy games, highlighting the continuous representations required in these areas.

Section 5: Navigation
Description 5: This section focuses on the application of CBR in robotic navigation tasks, detailing behavior schemas, parameter adaptation, and the use of contextual and sensory data.

Section 6: Robot soccer
Description 6: This section delves into the use of CBR in robot soccer, including position-based similarity measurement, adaptability cost, and management of uncertain information.

Section 7: Real-time strategic games
Description 7: This section analyzes CBR applications in real-time strategy games, covering plan recognition, sequential action prediction, and case-based plan recognition for anticipating opponent behavior.

Section 8: Imitation in psychology, neuroscience and AI
Description 8: This section explores the multidisciplinary field of learning by imitation, emphasizing the significance of this mechanism in natural intelligence and its implementation in artificial systems.

Section 9: The role of representations in motor control: a neuroscience view
Description 9: This section presents a neuroscientific perspective on representation in motor control, discussing internal models, context sensitivity, and the role of multiple controllers in motor planning.

Section 10: A neuroscience architecture of learning motor control by imitation
Description 10: This section describes a neuroscience-inspired architecture for learning motor control through imitation, detailing the components and processes involved in module selection and motor command generation.

Section 11: CBR at actuator level
Description 11: This section reviews CBR applications related to actuator control, focusing on non-symbolic representation, feedback mechanisms, and dynamic context relevance in inverse kinematic tasks.

Section 12: Discussion
Description 12: This section synthesizes the findings from the reviewed CBR systems, discussing common properties, relaxation of representation constraints, online learning, and prediction's role in reducing solution adaptation.