Robot Learning from Demonstration in Robotic Assembly: A Survey
---
paper_title: Modelling reversible execution of robotic assembly

paper_content:

Programming robotic assembly for industrial small-batch production is challenging; hence, it is vital to increase robustness and reduce development effort in order to achieve flexible robotic automation. A human who has made an assembly error will often simply undo the process until the error is undone and then restart the assembly. Conceptually, robots could do the same. This paper introduces a programming model that enables robot assembly programs to be executed in reverse. We investigate the challenges in running robot programs backwards and present a classification of reversibility characteristics. We demonstrate how temporarily switching the direction of program execution can be an efficient error recovery mechanism. Moreover, we demonstrate additional benefits arising from supporting reversibility in an assembly language, such as increased code reuse and automatically derived disassembly sequences. As a default approach to reversibility, we use program inversion and statement-level inversion of commands, but with a novel override option providing alternative sequences for asymmetric reverse actions. To efficiently program for this model, this paper introduces a new domain-specific language, SCP-RASQ (Simple C++ Reversible Assembly SeQuences). In initial experiments, where 200 consecutive assemblies of two industrial cases were performed, 18 of 22 errors were corrected automatically using only the trial-and-error capabilities that come from reverse execution.
---
paper_title: Teaching a Robot the Semantics of Assembly Tasks

paper_content:

We present a three-level cognitive system in a learning by demonstration context. The system allows for learning and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level representation based on so-called “semantic event chains.” We describe details of the representations and quantify the effect of the associated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the overall system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6.
---
paper_title: Robotic assembly solution by human-in-the-loop teaching method based on real-time stiffness modulation

paper_content:

We propose a novel human-in-the-loop approach for teaching robots how to solve assembly tasks in unpredictable and unstructured environments. In the proposed method the human sensorimotor system is integrated into the robot control loop though a teleoperation setup. The approach combines a 3-DoF end-effector force feedback with an interface for modulation of the robot end-effector stiffness. When operating in unpredictable and unstructured environments, modulation of limb impedance is essential in terms of successful task execution, stability and safety. We developed a novel hand-held stiffness control interface that is controlled by the motion of the human finger. A teaching approach was then used to achieve autonomous robot operation. In the experiments, we analysed and solved two part-assembly tasks: sliding a bolt fitting inside a groove and driving a self-tapping screw into a material of unknown properties. We experimentally compared the proposed method to complementary robot learning methods and analysed the potential benefits of direct stiffness modulation in the force-feedback teleoperation.
---
paper_title: A framework for fine robotic assembly

paper_content:

Fine robotic assembly, in which the parts to be assembled are small and fragile and lie in an unstructured environment, is still out of reach of today's industrial robots. The main difficulties arise in the precise localization of the parts in an unstructured environment and the control of contact interactions. Our contribution in this paper is twofold. First, we propose a taxonomy of the manipulation primitives that are specifically involved in fine assembly. Such a taxonomy is crucial for designing a scalable robotic system (both hardware and software) given the complexity of real-world assembly tasks. Second, we present a hardware and software architecture where we have addressed, in an integrated way, a number of issues arising in fine assembly, such as workspace optimization, external wrench compensation, position-based force control, etc. Finally, we show the above taxonomy and architecture in action on a highly dexterous task -- bimanual pin insertion -- which is one of the key steps in our long term project, the autonomous assembly of an IKEA chair.
---
paper_title: Cooperation of human and machines in assembly lines

paper_content:

Abstract Flexibility and changeability of assembly processes require a close cooperation between the worker and the automated assembly system. The interaction between human and robots improves the efficiency of individual complex assembly processes, particularly when a robot serves as an intelligent assistant. The paper gives a survey about forms of human–machine cooperation in assembly and available technologies that support the cooperation. Organizational and economic aspects of cooperative assembly including efficient component supply and logistics are also discussed.
---
paper_title: Integrated assembly and motion planning using regrasp graphs

paper_content:

This paper presents an integrated assembly and motion planning system to recursively find the assembly sequence and motions to assemble two objects with the help of a horizontal surface as the supporting fixture. The system is implemented in both assembly level and motion level. In the assembly level, the system checks all combinations of the assembly sequences and gets a set of candidates. Then, for each candidate assembly sequence, the system incrementally builds regrasp graphs and performs recursive search to find a pick-and-place motion in the motion level to manipulate the base object as well as to assemble the other object to the base. The system integrates the candidate assembly sequences computed in the assembly level incrementally and recursively with graph searching and motion planning in the motion level and plans the assembly sequences and motions integratedly for assembly tasks. Both simulation and real-world experiments are performed to demonstrate the efficacy of the integrated planning system.
---
paper_title: Spatial Programming for Industrial Robots through Task Demonstration

paper_content:

We present an intuitive system for the programming of industrial robots using markerless gesture recognition and mobile augmented reality in terms of programming by demonstration. The approach covers gesture-based task definition and adaption by human demonstration, as well as task evaluation through augmented reality. A 3D motion tracking system and a handheld device establish the basis for the presented spatial programming system. In this publication, we present a prototype toward the programming of an assembly sequence consisting of several pick-and-place tasks. A scene reconstruction provides pose estimation of known objects with the help of the 2D camera of the handheld. Therefore, the programmer is able to define the program through natural bare-hand manipulation of these objects with the help of direct visual feedback in the augmented reality application. The program can be adapted by gestures and transmitted subsequently to an arbitrary industrial robot controller using a unified interface. Finally, we discuss an application of the presented spatial programming approach toward robot-based welding tasks.
---
paper_title: Solving peg-in-hole tasks by human demonstration and exception strategies

paper_content:

Purpose – The purpose of this paper is to propose a new algorithm based on programming by demonstration and exception strategies to solve assembly tasks such as peg-in-hole. Design/methodology/approach – Data describing the demonstrated tasks are obtained by kinesthetic guiding. The demonstrated trajectories are transferred to new robot workspaces using three-dimensional (3D) vision. Noise introduced by vision when transferring the task to a new configuration could cause the execution to fail, but such problems are resolved through exception strategies. Findings – This paper demonstrated that the proposed approach combined with exception strategies outperforms traditional approaches for robot-based assembly. Experimental evaluation was carried out on Cranfield Benchmark, which constitutes a standardized assembly task in robotics. This paper also performed statistical evaluation based on experiments carried out on two different robotic platforms. Practical implications – The developed framework can have an...
---
paper_title: Automatic error recovery in robot assembly operations using reverse execution

paper_content:

Robotic assembly tasks are in general difficult to program and require a high degree of precision. As the complexity of the task increases it becomes increasingly unlikely that tasks can always be executed without errors. Preventing errors beyond a certain point is economically infeasible, in particular for small-batch productions. As an alternative, we propose a system for automatically handling certain classes of errors instead of preventing them. Specifically, we show that many operations can be automatically reversed. Errors can be handled through automatic reverse execution of the control program to a safe point, from which forward execution can be resumed. This paper describes the principles behind automatic reversal of robotic assembly operations, and experimentally demonstrates the use of a domain-specific language that supports automatic error handling through reverse execution. Our contribution represents the first experimental demonstration of reversible computing principles applied to industrial robotics.
---
paper_title: IkeaBot: An autonomous multi-robot coordinated furniture assembly system

paper_content:

We present an automated assembly system that directs the actions of a team of heterogeneous robots in the completion of an assembly task. From an initial user-supplied geometric specification, the system applies reasoning about the geometry of individual parts in order to deduce how they fit together. The task is then automatically transformed to a symbolic description of the assembly-a sort of blueprint. A symbolic planner generates an assembly sequence that can be executed by a team of collaborating robots. Each robot fulfills one of two roles: parts delivery or parts assembly. The latter are equipped with specialized tools to aid in the assembly process. Additionally, the robots engage in coordinated co-manipulation of large, heavy assemblies. We provide details of an example furniture kit assembled by the system.
---
paper_title: Learning of assembly constraints by demonstration and active exploration

paper_content:

Purpose ::: ::: ::: ::: ::: In this paper, the authors aim to propose a method for learning robotic assembly sequences, where precedence constraints and object relative size and location constraints can be learned by demonstration and autonomous robot exploration. ::: ::: ::: ::: ::: Design/methodology/approach ::: ::: ::: ::: ::: To successfully plan the operations involved in assembly tasks, the planner needs to know the constraints of the desired task. In this paper, the authors propose a methodology for learning such constraints by demonstration and autonomous exploration. The learning of precedence constraints and object relative size and location constraints, which are needed to construct a planner for automated assembly, were investigated. In the developed system, the learning of symbolic constraints is integrated with low-level control algorithms, which is essential to enable active robot learning. ::: ::: ::: ::: ::: Findings ::: ::: ::: ::: ::: The authors demonstrated that the proposed reasoning algorithms can be used to learn previously unknown assembly constraints that are needed to implement a planner for automated assembly. Cranfield benchmark, which is a standardized benchmark for testing algorithms for robot assembly, was used to evaluate the proposed approaches. The authors evaluated the learning performance both in simulation and on a real robot. ::: ::: ::: ::: ::: Practical implications ::: ::: ::: ::: ::: The authors' approach reduces the amount of programming that is needed to set up new assembly cells and consequently the overall set up time when new products are introduced into the workcell. ::: ::: ::: ::: ::: Originality/value ::: ::: ::: ::: ::: In this paper, the authors propose a new approach for learning assembly constraints based on programming by demonstration and active robot exploration to reduce the computational complexity of the underlying search problems. The authors developed algorithms for success/failure detection of assembly operations based on the comparison of expected signals (forces and torques, positions and orientations of the assembly parts) with the actual signals sensed by a robot. In this manner, all precedence and object size and location constraints can be learned, thereby providing the necessary input for the optimal planning of the entire assembly process.
---
paper_title: Assembly system design and operations for product variety

paper_content:

Abstract Assembly is the capstone process for product realization where component parts and subassemblies are integrated together to form the final products. As product variety increases due to the shift from mass production to mass customization, assembly systems must be designed and operated to handle such high variety. In this paper we first review the state of the art research in the areas of assembly system design, planning and operations in the presence of product variety. Methods for assembly representation, sequence generation and assembly line balancing are reviewed and summarized. Operational complexity and the role of human operators in assembly systems are then discussed in the context of product variety. Challenges in disassembly and remanufacturing in the presence of high variety are presented. We then conjecture a future manufacturing paradigm of personalized products and production and discuss the assembly challenge for such a paradigm. Opportunities for assembly system research are summarized at the end of the paper.
---
paper_title: Fast programming of Peg-in-hole Actions by human demonstration

paper_content:

Peg-in-Hole actions play an important role in industrial assembly. In this paper, we present a system which performs such actions after learning by human demonstration. Strategies for getting robust human-demonstrated PiH trajectories are investigated, and a number of PiH experiments are conducted to verify the performance of the proposed approach. The results show that our approach leads to a high PiH success rate and allows the robot to fine-tune the PiH trajectories to achieve faster performance.
---
paper_title: Robot programming from demonstration, feedback and transfer

paper_content:

This paper presents a novel approach for robot instruction for assembly tasks. We consider that robot programming can be made more efficient, precise and intuitive if we leverage the advantages of complementary approaches such as learning from demonstration, learning from feedback and knowledge transfer. Starting from low-level demonstrations of assembly tasks, the system is able to extract a high-level relational plan of the task. A graphical user interface (GUI) allows then the user to iteratively correct the acquired knowledge by refining high-level plans, and low-level geometrical knowledge of the task. This combination leads to a faster programming phase, more precise than just demonstrations, and more intuitive than just through a GUI. A final process allows to reuse high-level task knowledge for similar tasks in a transfer learning fashion. Finally we present a user study illustrating the advantages of this approach.
---
paper_title: Generalization of orientation trajectories and force-torque profiles for robotic assembly

paper_content:

Abstract A typical robot assembly operation involves contacts with the parts of the product to be assembled and consequently requires the knowledge of not only position and orientation trajectories but also the accompanying force-torque profiles for successful performance. To learn the execution of assembly operations even when the geometry of the product varies across task executions, the robot needs to be able to adapt its motion based on a parametric description of the current task condition, which is usually provided by geometrical properties of the parts involved in the assembly. In our previous work we showed how positional control policies can be generalized to different task conditions. In this paper we propose a complete methodology to generalize also the orientational trajectories and the accompanying force-torque profiles to compute the necessary control policy for a given condition of the assembly task. Our method is based on statistical generalization of successfully recorded executions at different task conditions, which are acquired by kinesthetic guiding. The parameters that describe the varying task conditions define queries into the recorded training data. To improve the execution of the skill after generalization, we combine the proposed approach with an adaptation method, thus enabling the refinement of the generalized assembly operation.
---
paper_title: Hole detection algorithm for chamferless square peg-in-hole based on shape recognition using F/T sensor

paper_content:

Precision parts are difficult to assemble using only position control. To deal with this difficulty, force control approaches, which provide proper motion responses against contact forces, are investigated. Moreover, hole detection is a crucial step in eliminating the uncertainty in robotic assembly. Without a proper hole detection algorithm, assembly time increases with the position difference between the peg and the hole. In this study, we propose a shape recognition algorithm based on a 6-axis F/T sensor and a hole detection algorithm. The proposed hole detection algorithm can find the direction of a hole regardless of the size of the position error between the peg and the hole with some overlap of the hole and the peg. The same algorithm can be implemented not only for a circular peg, but also for a polygonal convex peg. A series of experimental results show that the proposed algorithms can estimate the shape and location of a peg reasonably well.
---
paper_title: Narrow passage sampling in the observation of robotic assembly tasks

paper_content:

The observation of robotic assembly tasks is required as feedback for decisions and adaption of the task execution on the current situation. A sequential Monte Carlo observation algorithm is proposed, which uses a fast and accurate collision detection algorithm as a reference model for the contacts between complex shaped parts. The main contribution of the paper is the extension of the classic random motion model in the propagation step with sampling methods known from the domain of probabilistic roadmap planning in order to increase the sample density in narrow passages of the configuration space. As a result, the observation performance can be improved and a risk of sample impoverishment reduced. Experimental validation is provided for a peg-in-hole task executed by a lightweight-robot arm equipped with joint torque sensors.
---
paper_title: Teach industrial robots peg-hole-insertion by human demonstration

paper_content:

Programming robotic assembly tasks usually requires delicate force tuning. In contrast, human may accomplish assembly tasks with much less time and fewer trials. It will be a great benefit if robots can learn the human inherent skill of force control and apply it autonomously. Recent works on Learning from Demonstration (LfD) have shown the possibility to teach robots by human demonstration. The basic idea is to collect the force and corrective velocity that human applies during assembly, and then use them to regress a proper gain for the robot admittance controller. However, many of the LfD methods are tested on collaborative robots with compliant joints and relatively large assembly clearance. For industrial robots, the non-backdrivable mechanism and strict tolerance requirement make the assembly tasks more challenging. This paper modifies the original LfD to be suitable for industrial robots. A new demonstration tool is designed to acquire the human demonstration data. The force control gains are learned by Gaussian Mixture Regression (GMR) and the closed-loop stability is analysed. A series of peg-hole-insertion experiments with H7h7 tolerance on a FANUC manipulator validate the performance of the proposed learning method.
---
paper_title: Transfer of assembly operations to new workpiece poses by adaptation to the desired force profile

paper_content:

In this paper we propose a new algorithm that can be used for adaptation of robot trajectories in automated assembly tasks. Initial trajectories and forces are obtained by demonstration and iteratively adapted to specific environment configurations. The algorithm adapts Cartesian space trajectories to match the forces recorded during the human demonstration. Experimentally we show the effectiveness of our approach on learning of Peg-in-Hole (PiH) task. We performed our experiments on two different robotic platforms with workpieces of different shapes.
---
paper_title: SCARA based peg-in-hole assembly using compliant IPMC micro gripper

paper_content:

Robotic assembly is difficult as there always exist position errors between two mating parts. Compliance is added in a selective compliant assembly robot arm (SCARA) in the form of a two ionic polymer metal composite (IPMC) fingers based micro gripper. This micro gripper is integrated at the end effector position of a SCARA robot. Peg-hole interaction is analytically modeled and based on it the force required to correct the lateral and angular errors by IPMC is calculated. A proportional-derivative (PD) controller is designed to actuate the IPMC to get the desired force for correcting the peg position before assembly. Simulations and experiments were carried out by developing an IPMC micro gripper and using it to analyze various cases of peg in hole assembly. The experimental results prove that adding compliance through IPMC helps in peg-in-hole assembly.
---
paper_title: Multi-robot grasp planning for sequential assembly operations

paper_content:

This paper addresses the problem of finding robot configurations to grasp assembly parts during a sequence of collaborative assembly operations. We formulate the search for such configurations as a constraint satisfaction problem (CSP). Collision constraints in an operation and transfer constraints between operations determine the sets of feasible robot configurations. We show that solving the connected constraint graph with off-the-shelf CSP algorithms can quickly become infeasible even for a few sequential assembly operations. We present an algorithm which, through the assumption of feasible regrasps, divides the CSP into independent smaller problems that can be solved exponentially faster. The algorithm then uses local search techniques to improve this solution by removing a gradually increasing number of regrasps from the plan. The algorithm enables the user to stop the planner anytime and use the current best plan if the cost of removing regrasps from the plan exceeds the cost of executing those regrasps. We present simulation experiments to compare our algorithm’s performance to a naive algorithm which directly solves the connected constraint graph. We also present a physical robot system which uses the output of our planner to grasp and bring parts together in assembly configurations.
---
paper_title: Learning and generalization of motor skills by learning from demonstration

paper_content:

We provide a general approach for learning robotic motor skills from human demonstration. To represent an observed movement, a non-linear differential equation is learned such that it reproduces this movement. Based on this representation, we build a library of movements by labeling each recorded movement according to task and context (e.g., grasping, placing, and releasing). Our differential equation is formulated such that generalization can be achieved simply by adapting a start and a goal parameter in the equation to the desired position values of a movement. For object manipulation, we present how our framework extends to the control of gripper orientation and finger position. The feasibility of our approach is demonstrated in simulation as well as on the Sarcos dextrous robot arm. The robot learned a pick-and-place operation and a water-serving task and could generalize these tasks to novel situations.
---
paper_title: Getting Humanoids to Move and Imitate

paper_content:

A behavior-based approach to structuring and controlling complex robotic systems is described that uses imitation for interaction and learning. The use of basis behaviors, or primitives, lets humanoid robots dance the Macarena, do aerobics and throw objects.
---
paper_title: Linguistic transfer of human assembly tasks to robots

paper_content:

We demonstrate the automatic transfer of an assembly task from human to robot. This work extends efforts showing the utility of linguistic models in verifiable robot control policies by now performing real visual analysis of human demonstrations to automatically extract a policy for the task. This method tokenizes each human demonstration into a sequence of object connection symbols, then transforms the set of sequences from all demonstrations into an automaton, which represents the task-language for assembling a desired object. Finally, we combine this assembly automaton with a kinematic model of a robot arm to reproduce the demonstrated task.
---
paper_title: Robot programming by demonstration

paper_content:

The presented paper shows a pilot development of a robot ‘task programming method’. In this method, the user programs the robot task by demonstrating it (Programming by Demonstration, PbD). PbD is applied on a robotic arm with two degrees-of-freedom for programming a constrained motion task.
---
paper_title: A geometric approach for learning compliant motions from demonstration

paper_content:

This paper proposes a method to learn from human demonstration compliant contact motions, which take advantage of interaction forces between workpieces to align them, even when contact force may occur from different directions on different instances of reproduction. To manage the uncertainty in unstructured conditions, the motions learned with our method can be reproduced with an impedance controller. Learning from Demonstration is used because the planning of compliant motions in 3-D is computationally intractable. The proposed method will learn an individual compliant motion, many of which can be combined to solve more complex tasks. The method is based on measuring simultaneously the direction of motion and the forces acting on the end-effector. From these measurements we construct a set of constraints for motion directions which, with correct compliance, result in the observed motion. Constraints from multiple demonstrations are projected into a 2-D angular coordinate system where their intersection is determined to find a set of feasible desired directions, of which a single motion direction is chosen. The work is based on the assumption that movement in directions other than the desired direction is caused by interaction forces. Using this assumption, we infer the number of compliant axes and, if required, their directions. Experiments with a KUKA LWR4+ show that our method can successfully reproduce motions which require taking advantage of the environment.
---
paper_title: Robot See, Robot Do : An Overview of Robot Imitation

paper_content:

A belt grinding machine which is equipped with an elastically flexible load beam extending over the entire operative length of an endless grinding belt for applying the necessary grinding pressure to the belt. The effective pressure-applying surface of said beam is suspended for elevational control from support elements distributed at intervals along the length of the beam and attached to an elevationally movable carrier preferably in the form of transverse beams mounted on push rods operable by a pressure medium. The push rods are associated with constant delay mechanism adjustable to the difference in time between the instant of response of a sensing element to the passage of the leading end of a workpiece and the instant the grinding action begins, on the one hand, and between the instant of response of a sensing element to the passage of the trailing end of the workpiece and the instant the grinding action terminates, on the other hand.
---
paper_title: From Demonstrations to Skills for High-level Programming of Industrial Robots

paper_content:

In this paper we describe our approach to robotic skill representation and a prototypical implementation of a programming-by-demonstration approach that allows users to generate skills and robot program primitives for later re- finement and re-use. We intend to evaluate the applicability of this approach to high-level programming in a user study, which we also explain.
---
paper_title: Recognizing Assembly Tasks Through Human Demonstration

paper_content:

As one of the methods for reducing the work of programming, the Learning-from-Observation (LFO) paradigm has been heavily promoted. This paradigm requires the programmer only to perform a task in front of a robot and does not require expertise. In this paper, the LFO paradigm is applied to assembly tasks by two rigid polyhedral objects. A method is proposed for recognizing these tasks as a sequence of movement primitives from noise-contaminated data obtained by a conventional 6 degree-of-freedom (DOF) object-tracking system. The system is implemented on a robot with a real-time stereo vision system and dual arms with dexterous hands, and its effectiveness is demonstrated.
---
paper_title: A robot learning from demonstration framework to perform force-based manipulation tasks

paper_content:

This paper proposes an end-to-end learning from demonstration framework for teaching force-based manipulation tasks to robots. The strengths of this work are manyfold. First, we deal with the problem of learning through force perceptions exclusively. Second, we propose to exploit haptic feedback both as a means for improving teacher demonstrations and as a human---robot interaction tool, establishing a bidirectional communication channel between the teacher and the robot, in contrast to the works using kinesthetic teaching. Third, we address the well-known what to imitate? problem from a different point of view, based on the mutual information between perceptions and actions. Lastly, the teacher's demonstrations are encoded using a Hidden Markov Model, and the robot execution phase is developed by implementing a modified version of Gaussian Mixture Regression that uses implicit temporal information from the probabilistic model, needed when tackling tasks with ambiguous perceptions. Experimental results show that the robot is able to learn and reproduce two different manipulation tasks, with a performance comparable to the teacher's one.
---
paper_title: Toward programming of assembly tasks by demonstration in virtual environments

paper_content:

Service robots require simple programming techniques allowing users with little or no technical expertise to integrate new tasks in a robotic platform. A promising solution for automatic acquisition of robot behaviours is the programming by demonstration (PbD) paradigm. Its aim is to let robot systems learn new behaviours from a human operator demonstration. This paper describes a PbD system able to deal with assembly operation in a 3D block world. The main objective of the research is to investigate the benefits of a virtual demonstration environment. Overcoming some difficulties of real world demonstrations, a virtual environment can improve the effectiveness of the instruction phase. Moreover, the user can also supervise and validate the learned task by means of a simulation module, thereby reducing errors in the generation process. Some experiments involving the whole set of system components demonstrate the viability and effectiveness of the approach.
---
paper_title: Learning Collaborative Impedance-based Robot Behaviors ∗

paper_content:

Research in learning from demonstration has focused on transferring movements from humans to robots. However, a need is arising for robots that do not just replicate the task on their own, but that also interact with humans in a safe and natural way to accomplish tasks cooperatively. Robots with variable impedance capabilities opens the door to new challenging applications, where the learning algorithms must be extended by encapsulating force and vision information. In this paper we propose a framework to transfer impedance-based behaviors to a torque-controlled robot by kinesthetic teaching. The proposed model encodes the examples as a task-parameterized statistical dynamical system, where the robot impedance is shaped by estimating virtual stiffness matrices from the set of demonstrations. A collaborative assembly task is used as testbed. The results show that the model can be used to modify the robot impedance along task execution to facilitate the collaboration, by triggering stiff and compliant behaviors in an on-line manner to adapt to the user's actions.
---
paper_title: Technologies for the Fast Set-Up of Automated Assembly Processes

paper_content:

In this article, we describe technologies facilitating the set-up of automated assembly solutions which have been developed in the context of the IntellAct project (2011–2014). Tedious procedures are currently still required to establish such robot solutions. This hinders especially the automation of so called few-of-a-kind production. Therefore, most production of this kind is done manually and thus often performed in low-wage countries. In the IntellAct project, we have developed a set of methods which facilitate the set-up of a complex automatic assembly process, and here we present our work on tele-operation, dexterous grasping, pose estimation and learning of control strategies. The prototype developed in IntellAct is at a TRL4 (corresponding to ‘demonstration in lab environment’).
---
paper_title: A survey of robot learning from demonstration

paper_content:

We present a comprehensive survey of robot Learning from Demonstration (LfD), a technique that develops policies from example state to action mappings. We introduce the LfD design choices in terms of demonstrator, problem space, policy derivation and performance, and contribute the foundations for a structure in which to categorize LfD research. Specifically, we analyze and categorize the multiple ways in which examples are gathered, ranging from teleoperation to imitation, as well as the various techniques for policy derivation, including matching functions, dynamics models and plans. To conclude we discuss LfD limitations and related promising areas for future research.
---
paper_title: Recognition of dynamic environments for robotic assembly on moving workpieces

paper_content:

Automated robotic assembly on a moving workpiece, referred to as assembly in motion, demands that the assembly robot is synchronised in all degrees of freedom to the moving workpiece, on which assembly parts are installed. Currently, this requirement cannot be met due to the lack of robust recognition of the 3D position and the trajectory of the moving workpiece. In this paper, an assembly robot-guided, monocular camera system approaching this problem of recognition of dynamic environments is introduced, which considers the motion trajectory of the workpiece as a linear combination of trajectory bases, such as Discrete Cosine Transform (DCT) bases. The experimental results show that the proposed method is able to reconstruct arbitrary trajectories of a predefined assembly point on the workpiece moving in 3D space. The limitation of the developed method of environment recognition for robotic assembly in motion is also analysed.
---
paper_title: Solving peg-in-hole tasks by human demonstration and exception strategies

paper_content:

Purpose – The purpose of this paper is to propose a new algorithm based on programming by demonstration and exception strategies to solve assembly tasks such as peg-in-hole. Design/methodology/approach – Data describing the demonstrated tasks are obtained by kinesthetic guiding. The demonstrated trajectories are transferred to new robot workspaces using three-dimensional (3D) vision. Noise introduced by vision when transferring the task to a new configuration could cause the execution to fail, but such problems are resolved through exception strategies. Findings – This paper demonstrated that the proposed approach combined with exception strategies outperforms traditional approaches for robot-based assembly. Experimental evaluation was carried out on Cranfield Benchmark, which constitutes a standardized assembly task in robotics. This paper also performed statistical evaluation based on experiments carried out on two different robotic platforms. Practical implications – The developed framework can have an...
---
paper_title: Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge

paper_content:

Robot warehouse automation has attracted significant interest in recent years, perhaps most visibly in the Amazon Picking Challenge (APC). A fully autonomous warehouse pick-and-place system requires robust vision that reliably recognizes and locates objects amid cluttered environments, self-occlusions, sensor noise, and a large variety of objects. In this paper we present an approach that leverages multi-view RGB-D data and self-supervised, data-driven learning to overcome those difficulties. The approach was part of the MIT-Princeton Team system that took 3rd- and 4th- place in the stowing and picking tasks, respectively at APC 2016. In the proposed approach, we segment and label multiple views of a scene with a fully convolutional neural network, and then fit pre-scanned 3D object models to the resulting segmentation to get the 6D object pose. Training a deep neural network for segmentation typically requires a large amount of training data. We propose a self-supervised method to generate a large labeled dataset without tedious manual segmentation. We demonstrate that our system can reliably estimate the 6D pose of objects under a variety of scenarios. All code, data, and benchmarks are available at this http URL
---
paper_title: Fast programming of Peg-in-hole Actions by human demonstration

paper_content:

Peg-in-Hole actions play an important role in industrial assembly. In this paper, we present a system which performs such actions after learning by human demonstration. Strategies for getting robust human-demonstrated PiH trajectories are investigated, and a number of PiH experiments are conducted to verify the performance of the proposed approach. The results show that our approach leads to a high PiH success rate and allows the robot to fine-tune the PiH trajectories to achieve faster performance.
---
paper_title: Extracting Semantic Information from Visual Data: A Survey

paper_content:

The traditional environment maps built by mobile robots include both metric ones and topological ones. These maps are navigation-oriented and not adequate for service robots to interact with or serve human users who normally rely on the conceptual knowledge or semantic contents of the environment. Therefore, the construction of semantic maps becomes necessary for building an effective human-robot interface for service robots. This paper reviews recent research and development in the field of visual-based semantic mapping. The main focus is placed on how to extract semantic information from visual data in terms of feature extraction, object/place recognition and semantic representation methods.
---
paper_title: Voting-based pose estimation for robotic assembly using a 3D sensor

paper_content:

We propose a voting-based pose estimation algorithm applicable to 3D sensors, which are fast replacing their 2D counterparts in many robotics, computer vision, and gaming applications. It was recently shown that a pair of oriented 3D points, which are points on the object surface with normals, in a voting framework enables fast and robust pose estimation. Although oriented surface points are discriminative for objects with sufficient curvature changes, they are not compact and discriminative enough for many industrial and real-world objects that are mostly planar. As edges play the key role in 2D registration, depth discontinuities are crucial in 3D. In this paper, we investigate and develop a family of pose estimation algorithms that better exploit this boundary information. In addition to oriented surface points, we use two other primitives: boundary points with directions and boundary line segments. Our experiments show that these carefully chosen primitives encode more information compactly and thereby provide higher accuracy for a wide class of industrial parts and enable faster computation. We demonstrate a practical robotic bin-picking system using the proposed algorithm and a 3D sensor.
---
paper_title: Reducing uncertainty in robotic surface assembly tasks based on contact information

paper_content:

In contrast to hard automation, flexible assembly using a general-purpose robot holds the promise of reducing costs and increasing productivity of small batch products as well as adaptivity to product changes. However, robotic assembly remains one of the most challenging problems in robotics research. In particular, it has to overcome uncertainty in the environment and the location of parts. This paper addresses how to estimate and correct part positioning uncertainties with respect to the assembly task by using the contact trajectory data collected during active part interaction. We first devise a nominal assembly motion sequence based on an approximate knowledge of the poses of the parts, which includes exploratory compliant moves for the purpose of parameter estimation. As the robot manipulator executes the nominal motion sequence, data collected during exploratory compliant moves are used to estimate part poses more accurately, and the improved information is used to update the subsequent assembly motion sequence to correct errors in the nominal plan. We implemented the strategy using a back-drivable Barrett WAM robotic arm without a force sensor and BarrettHand BH8-262. The empirical results have demonstrated the effectiveness of our approach.
---
paper_title: Teaching a Robot the Semantics of Assembly Tasks

paper_content:

We present a three-level cognitive system in a learning by demonstration context. The system allows for learning and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level representation based on so-called “semantic event chains.” We describe details of the representations and quantify the effect of the associated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the overall system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6.
---
paper_title: Learning compliant assembly motions from demonstration

paper_content:

Automating assembly processes outside controlled factory environments is still rare, mostly because of the inherent position uncertainties. The use of compliant motions allows robustness against the uncertainty, but automatic planning of compliant motion sequences is not computationally feasible. In this paper, we show how compliant assembly motions can be learned from human demonstrations. A human teacher will kinesthetically demonstrate compliant motions where the physical shapes of assembled parts guide the motion. From these demonstrations, the proposed method identifies desired direction of movement, the number of compliant axes and their directions. We use this information to construct an impedance controller which can reproduce the assembly motion despite uncertainty in the starting position. The method is studied with a KUKA LWR4+ arm in two test setups with different number of physically constrained degrees of freedom. The experimental study shows that the method is able to correctly identify the motion parameters and allows the robot to successfully perform the demonstrated assembly motion from various unseen starting positions.
---
paper_title: Robotic assembly solution by human-in-the-loop teaching method based on real-time stiffness modulation

paper_content:

We propose a novel human-in-the-loop approach for teaching robots how to solve assembly tasks in unpredictable and unstructured environments. In the proposed method the human sensorimotor system is integrated into the robot control loop though a teleoperation setup. The approach combines a 3-DoF end-effector force feedback with an interface for modulation of the robot end-effector stiffness. When operating in unpredictable and unstructured environments, modulation of limb impedance is essential in terms of successful task execution, stability and safety. We developed a novel hand-held stiffness control interface that is controlled by the motion of the human finger. A teaching approach was then used to achieve autonomous robot operation. In the experiments, we analysed and solved two part-assembly tasks: sliding a bolt fitting inside a groove and driving a self-tapping screw into a material of unknown properties. We experimentally compared the proposed method to complementary robot learning methods and analysed the potential benefits of direct stiffness modulation in the force-feedback teleoperation.
---
paper_title: Contact-state modeling of robotic assembly tasks using Gaussian mixture models

paper_content:

Abstract This article addresses the Contact-State (CS) modeling problem for the force-controlled robotic peg-in-hole assembly tasks. The wrench (Cartesian forces and torques) and pose (Cartesian position and orientation) signals, of the manipulated object, are captured for different phases of the robotic assembly task. Those signals are utilized in building a CS model for each phase. Gaussian Mixture Models (GMM) is employed in building the likelihood of each signal and Expectation Maximization (EM) is used in finding the GMM parameters. Experiments are performed on a KUKA Lightweight Robot (LWR) doing camshaft caps assembly of an automotive powertrain. Comparisons are also performed with the available assembly modeling schemes, and the superiority of the EM-GMM scheme is shown with a reduced computational time.
---
paper_title: On-line knowledge acquisition and enhancement in robotic assembly tasks

paper_content:

Industrial robots are reliable machines for manufacturing tasks such as welding, panting, assembly, palletizing or kitting operations. They are traditionally programmed by an operator using a teach pendant in a point-to-point scheme with limited sensing capabilities such as industrial vision systems and force/torque sensing. The use of these sensing capabilities is associated to the particular robot controller, operative systems and programming language. Today, robots can react to environment changes specific to their task domain but are still unable to learn skills to effectively use their current knowledge. The need for such a skill in unstructured environments where knowledge can be acquired and enhanced is desirable so that robots can effectively interact in multimodal real-world scenarios.In this article we present a multimodal assembly controller (MAC) approach to embed and effectively enhance knowledge into industrial robots working in multimodal manufacturing scenarios such as assembly during kitting operations with varying shapes and tolerances. During learning, the robot uses its vision and force capabilities resembling a human operator carrying out the same operation. The approach consists of using a MAC based on the Fuzzy ARTMAP artificial neural network in conjunction with a knowledge base. The robot starts the operation having limited initial knowledge about what task it has to accomplish. During the operation, the robot learns the skill for recognising assembly parts and how to assemble them. The skill acquisition is evaluated by counting the steps to complete the assembly, length of the followed assembly path and compliant behaviour. The performance improves with time so that the robot becomes an expert demonstrated by the assembly of a kit with different part geometries. The kit is unknown by the robot at the beginning of the operation; therefore, the kit type, location and orientation are unknown as well as the parts to be assembled since they are randomly fed by a conveyor belt. A novel multimodal assembly controller (MAC) was designed having minimal assembly information (-Z assembly direction).It was demonstrated that knowledge can be refined when other different matting pair are assembled without the need to acquire another primitive knowledge base (PKB).The robot learns a new assembly and improves its skills from experience observed by a reduced number of patterns, lower compliant forces and shorter assembly trajectories.The MAC demonstrated that can be used in non-structured environments for the kitting process with uncertainties (in position and geometry) for both, the kits and the pegs.
---
paper_title: A geometric approach for learning compliant motions from demonstration

paper_content:

This paper proposes a method to learn from human demonstration compliant contact motions, which take advantage of interaction forces between workpieces to align them, even when contact force may occur from different directions on different instances of reproduction. To manage the uncertainty in unstructured conditions, the motions learned with our method can be reproduced with an impedance controller. Learning from Demonstration is used because the planning of compliant motions in 3-D is computationally intractable. The proposed method will learn an individual compliant motion, many of which can be combined to solve more complex tasks. The method is based on measuring simultaneously the direction of motion and the forces acting on the end-effector. From these measurements we construct a set of constraints for motion directions which, with correct compliance, result in the observed motion. Constraints from multiple demonstrations are projected into a 2-D angular coordinate system where their intersection is determined to find a set of feasible desired directions, of which a single motion direction is chosen. The work is based on the assumption that movement in directions other than the desired direction is caused by interaction forces. Using this assumption, we infer the number of compliant axes and, if required, their directions. Experiments with a KUKA LWR4+ show that our method can successfully reproduce motions which require taking advantage of the environment.
---
paper_title: Learning of assembly constraints by demonstration and active exploration

paper_content:

Purpose ::: ::: ::: ::: ::: In this paper, the authors aim to propose a method for learning robotic assembly sequences, where precedence constraints and object relative size and location constraints can be learned by demonstration and autonomous robot exploration. ::: ::: ::: ::: ::: Design/methodology/approach ::: ::: ::: ::: ::: To successfully plan the operations involved in assembly tasks, the planner needs to know the constraints of the desired task. In this paper, the authors propose a methodology for learning such constraints by demonstration and autonomous exploration. The learning of precedence constraints and object relative size and location constraints, which are needed to construct a planner for automated assembly, were investigated. In the developed system, the learning of symbolic constraints is integrated with low-level control algorithms, which is essential to enable active robot learning. ::: ::: ::: ::: ::: Findings ::: ::: ::: ::: ::: The authors demonstrated that the proposed reasoning algorithms can be used to learn previously unknown assembly constraints that are needed to implement a planner for automated assembly. Cranfield benchmark, which is a standardized benchmark for testing algorithms for robot assembly, was used to evaluate the proposed approaches. The authors evaluated the learning performance both in simulation and on a real robot. ::: ::: ::: ::: ::: Practical implications ::: ::: ::: ::: ::: The authors' approach reduces the amount of programming that is needed to set up new assembly cells and consequently the overall set up time when new products are introduced into the workcell. ::: ::: ::: ::: ::: Originality/value ::: ::: ::: ::: ::: In this paper, the authors propose a new approach for learning assembly constraints based on programming by demonstration and active robot exploration to reduce the computational complexity of the underlying search problems. The authors developed algorithms for success/failure detection of assembly operations based on the comparison of expected signals (forces and torques, positions and orientations of the assembly parts) with the actual signals sensed by a robot. In this manner, all precedence and object size and location constraints can be learned, thereby providing the necessary input for the optimal planning of the entire assembly process.
---
paper_title: Adaptation of bimanual assembly tasks using iterative learning framework

paper_content:

The paper deals with the adaptation of bimanual assembly tasks. First, the desired policy is shown by human demonstration using kinesthetic guidance, where both trajectories and interaction forces are captured. Captured entities are portioned to absolute and relative coordinates. During the execution, small discrepancies in object geometry as well as the influence of an imperfect control can result in large contact forces. Force control can diminish the above mentioned problems only to some extent. Therefore, we propose a framework that iteratively modifies the original demonstrated trajectory in order to increase the performance of the typical assembly tasks. The approach is validated on bimanual peg in a hole task using two KUKA LWR robots.
---
paper_title: Fast programming of Peg-in-hole Actions by human demonstration

paper_content:

Peg-in-Hole actions play an important role in industrial assembly. In this paper, we present a system which performs such actions after learning by human demonstration. Strategies for getting robust human-demonstrated PiH trajectories are investigated, and a number of PiH experiments are conducted to verify the performance of the proposed approach. The results show that our approach leads to a high PiH success rate and allows the robot to fine-tune the PiH trajectories to achieve faster performance.
---
paper_title: Generalization of orientation trajectories and force-torque profiles for robotic assembly

paper_content:

Abstract A typical robot assembly operation involves contacts with the parts of the product to be assembled and consequently requires the knowledge of not only position and orientation trajectories but also the accompanying force-torque profiles for successful performance. To learn the execution of assembly operations even when the geometry of the product varies across task executions, the robot needs to be able to adapt its motion based on a parametric description of the current task condition, which is usually provided by geometrical properties of the parts involved in the assembly. In our previous work we showed how positional control policies can be generalized to different task conditions. In this paper we propose a complete methodology to generalize also the orientational trajectories and the accompanying force-torque profiles to compute the necessary control policy for a given condition of the assembly task. Our method is based on statistical generalization of successfully recorded executions at different task conditions, which are acquired by kinesthetic guiding. The parameters that describe the varying task conditions define queries into the recorded training data. To improve the execution of the skill after generalization, we combine the proposed approach with an adaptation method, thus enabling the refinement of the generalized assembly operation.
---
paper_title: Human-in-the-loop approach for teaching robot assembly tasks using impedance control interface

paper_content:

In this paper we propose a human-in-the-loop approach for teaching robots how to solve part assembly tasks. In the proposed setup the human tutor controls the robot through a haptic interface and a hand-held impedance control interface. The impedance control interface is based on a linear spring-return potentiometer that maps the button position to the robot arm stiffness. This setup allows the tutor to modulate the robot compliance based on the given task requirements. The demonstrated motion and stiffness trajectories are encoded using Dynamical Movement Primitives and learnt using Locally Weight Regression. To validate the proposed approach we performed experiments using Kuka Light Weight Robot and HapticMaster robot. The task of the experiment was to teach the robot how to perform an assembly task involving sliding a bolt fitting inside a groove in order to mount two parts together. Different stiffness was required in different stages of the task execution to accommodate the interaction of the robot with the environment and possible human-robot cooperation.
---
paper_title: Hole detection algorithm for chamferless square peg-in-hole based on shape recognition using F/T sensor

paper_content:

Precision parts are difficult to assemble using only position control. To deal with this difficulty, force control approaches, which provide proper motion responses against contact forces, are investigated. Moreover, hole detection is a crucial step in eliminating the uncertainty in robotic assembly. Without a proper hole detection algorithm, assembly time increases with the position difference between the peg and the hole. In this study, we propose a shape recognition algorithm based on a 6-axis F/T sensor and a hole detection algorithm. The proposed hole detection algorithm can find the direction of a hole regardless of the size of the position error between the peg and the hole with some overlap of the hole and the peg. The same algorithm can be implemented not only for a circular peg, but also for a polygonal convex peg. A series of experimental results show that the proposed algorithms can estimate the shape and location of a peg reasonably well.
---
paper_title: Narrow passage sampling in the observation of robotic assembly tasks

paper_content:

The observation of robotic assembly tasks is required as feedback for decisions and adaption of the task execution on the current situation. A sequential Monte Carlo observation algorithm is proposed, which uses a fast and accurate collision detection algorithm as a reference model for the contacts between complex shaped parts. The main contribution of the paper is the extension of the classic random motion model in the propagation step with sampling methods known from the domain of probabilistic roadmap planning in order to increase the sample density in narrow passages of the configuration space. As a result, the observation performance can be improved and a risk of sample impoverishment reduced. Experimental validation is provided for a peg-in-hole task executed by a lightweight-robot arm equipped with joint torque sensors.
---
paper_title: Teach industrial robots peg-hole-insertion by human demonstration

paper_content:

Programming robotic assembly tasks usually requires delicate force tuning. In contrast, human may accomplish assembly tasks with much less time and fewer trials. It will be a great benefit if robots can learn the human inherent skill of force control and apply it autonomously. Recent works on Learning from Demonstration (LfD) have shown the possibility to teach robots by human demonstration. The basic idea is to collect the force and corrective velocity that human applies during assembly, and then use them to regress a proper gain for the robot admittance controller. However, many of the LfD methods are tested on collaborative robots with compliant joints and relatively large assembly clearance. For industrial robots, the non-backdrivable mechanism and strict tolerance requirement make the assembly tasks more challenging. This paper modifies the original LfD to be suitable for industrial robots. A new demonstration tool is designed to acquire the human demonstration data. The force control gains are learned by Gaussian Mixture Regression (GMR) and the closed-loop stability is analysed. A series of peg-hole-insertion experiments with H7h7 tolerance on a FANUC manipulator validate the performance of the proposed learning method.
---
paper_title: Transfer of assembly operations to new workpiece poses by adaptation to the desired force profile

paper_content:

In this paper we propose a new algorithm that can be used for adaptation of robot trajectories in automated assembly tasks. Initial trajectories and forces are obtained by demonstration and iteratively adapted to specific environment configurations. The algorithm adapts Cartesian space trajectories to match the forces recorded during the human demonstration. Experimentally we show the effectiveness of our approach on learning of Peg-in-Hole (PiH) task. We performed our experiments on two different robotic platforms with workpieces of different shapes.
---
paper_title: Force controlled robotic assembly without a force sensor

paper_content:

The traditional way of controlling an industrial robot is to program it to follow desired trajectories. This approach is sufficient as long as the accuracy of the robot and the calibration of the workcell is good enough. In robotic assembly these conditions are usually not fulfilled, because of uncertainties, e.g., variability in involved parts and objects not gripped accurately. Using force control is one way to handle these difficulties. This paper presents a method of doing force control without a force sensor. The method is based on detuning of the low-level joint control loops, and the force is estimated from the control error. It is experimentally verified in a small part assembly task with a kinematically redundant robotic manipulator.
---
paper_title: Contact force estimation for robotic assembly using motor torques

paper_content:

For tasks in robotic assembly, a basic requirement is the capability of the robotic system to detect contacts with the environment. While this information can readily be obtained from dedicated force-torque sensors, it might either be technically difficult to mount such sensors or too expensive to do so. In this paper, an alternative approach is presented which reconstructs the external forces and torques based on signals that can be obtained without a force sensor, namely motor currents/torques and joint angles. After an automatic offline calibration of the algorithm, the method allows for a lean online implementation. Apart from the theoretical development of the method, its applicability to contact force estimation is demonstrated by measurement results obtained from an ABB Dual-Arm Concept Robot.
---
paper_title: Influence of assembly predicate consideration on optimal assembly sequence generation

paper_content:

Purpose – The purpose of this paper is to find out the significant influence of assembly predicate consideration on optimal assembly sequence generation (ASG) in terms of search space, computational time and possibility of resulting practically not feasible assembly sequences. An appropriate assembly sequence results in minimal lead time and low cost of assembly. ASG is a complex combinatorial optimisation problem which deals with several assembly predicates to result an optimal assembly sequence. The consideration of each assembly predicate highly influences the search space and thereby computational time to achieve valid assembly sequence. Often, the ignoring an assembly predicate leads to inappropriate assembly sequence, which may not be physically possible, sometimes predicate assumption drastic ally raises the search space with high computational time. Design/methodology/approach – The influence of assuming and considering different assembly predicates on optimal assembly sequence generation have bee...
---
paper_title: Integrated assembly and motion planning using regrasp graphs

paper_content:

This paper presents an integrated assembly and motion planning system to recursively find the assembly sequence and motions to assemble two objects with the help of a horizontal surface as the supporting fixture. The system is implemented in both assembly level and motion level. In the assembly level, the system checks all combinations of the assembly sequences and gets a set of candidates. Then, for each candidate assembly sequence, the system incrementally builds regrasp graphs and performs recursive search to find a pick-and-place motion in the motion level to manipulate the base object as well as to assemble the other object to the base. The system integrates the candidate assembly sequences computed in the assembly level incrementally and recursively with graph searching and motion planning in the motion level and plans the assembly sequences and motions integratedly for assembly tasks. Both simulation and real-world experiments are performed to demonstrate the efficacy of the integrated planning system.
---
paper_title: Evaluation of a robot programming framework for non-experts using symbolic planning representations

paper_content:

Cobots (collaborative robots) are revolutionising industries by allowing robots to work in close collaboration with humans. But many companies hesitate their adoption, due to the lack of programming experts. In this work, we evaluate a robot programming framework for non-expert users, that requires users to teach action models expressed in a symbolic planning language (PDDL). These action models would allow the robot to leverage modern automated planners to achieve any user-defined goal. We conducted qualitative user experiments with a Baxter robot to evaluate the non-expert user's understanding of the symbolic planning language and the usability of the framework. We showed that users with little to no programming experience can adopt the symbolic planning language, and use the framework.
---
paper_title: Spatial Programming for Industrial Robots through Task Demonstration

paper_content:

We present an intuitive system for the programming of industrial robots using markerless gesture recognition and mobile augmented reality in terms of programming by demonstration. The approach covers gesture-based task definition and adaption by human demonstration, as well as task evaluation through augmented reality. A 3D motion tracking system and a handheld device establish the basis for the presented spatial programming system. In this publication, we present a prototype toward the programming of an assembly sequence consisting of several pick-and-place tasks. A scene reconstruction provides pose estimation of known objects with the help of the 2D camera of the handheld. Therefore, the programmer is able to define the program through natural bare-hand manipulation of these objects with the help of direct visual feedback in the augmented reality application. The program can be adapted by gestures and transmitted subsequently to an arbitrary industrial robot controller using a unified interface. Finally, we discuss an application of the presented spatial programming approach toward robot-based welding tasks.
---
paper_title: A geometric approach for learning compliant motions from demonstration

paper_content:

This paper proposes a method to learn from human demonstration compliant contact motions, which take advantage of interaction forces between workpieces to align them, even when contact force may occur from different directions on different instances of reproduction. To manage the uncertainty in unstructured conditions, the motions learned with our method can be reproduced with an impedance controller. Learning from Demonstration is used because the planning of compliant motions in 3-D is computationally intractable. The proposed method will learn an individual compliant motion, many of which can be combined to solve more complex tasks. The method is based on measuring simultaneously the direction of motion and the forces acting on the end-effector. From these measurements we construct a set of constraints for motion directions which, with correct compliance, result in the observed motion. Constraints from multiple demonstrations are projected into a 2-D angular coordinate system where their intersection is determined to find a set of feasible desired directions, of which a single motion direction is chosen. The work is based on the assumption that movement in directions other than the desired direction is caused by interaction forces. Using this assumption, we infer the number of compliant axes and, if required, their directions. Experiments with a KUKA LWR4+ show that our method can successfully reproduce motions which require taking advantage of the environment.
---
paper_title: Learning of assembly constraints by demonstration and active exploration

paper_content:

Purpose ::: ::: ::: ::: ::: In this paper, the authors aim to propose a method for learning robotic assembly sequences, where precedence constraints and object relative size and location constraints can be learned by demonstration and autonomous robot exploration. ::: ::: ::: ::: ::: Design/methodology/approach ::: ::: ::: ::: ::: To successfully plan the operations involved in assembly tasks, the planner needs to know the constraints of the desired task. In this paper, the authors propose a methodology for learning such constraints by demonstration and autonomous exploration. The learning of precedence constraints and object relative size and location constraints, which are needed to construct a planner for automated assembly, were investigated. In the developed system, the learning of symbolic constraints is integrated with low-level control algorithms, which is essential to enable active robot learning. ::: ::: ::: ::: ::: Findings ::: ::: ::: ::: ::: The authors demonstrated that the proposed reasoning algorithms can be used to learn previously unknown assembly constraints that are needed to implement a planner for automated assembly. Cranfield benchmark, which is a standardized benchmark for testing algorithms for robot assembly, was used to evaluate the proposed approaches. The authors evaluated the learning performance both in simulation and on a real robot. ::: ::: ::: ::: ::: Practical implications ::: ::: ::: ::: ::: The authors' approach reduces the amount of programming that is needed to set up new assembly cells and consequently the overall set up time when new products are introduced into the workcell. ::: ::: ::: ::: ::: Originality/value ::: ::: ::: ::: ::: In this paper, the authors propose a new approach for learning assembly constraints based on programming by demonstration and active robot exploration to reduce the computational complexity of the underlying search problems. The authors developed algorithms for success/failure detection of assembly operations based on the comparison of expected signals (forces and torques, positions and orientations of the assembly parts) with the actual signals sensed by a robot. In this manner, all precedence and object size and location constraints can be learned, thereby providing the necessary input for the optimal planning of the entire assembly process.
---
paper_title: Fast programming of Peg-in-hole Actions by human demonstration

paper_content:

Peg-in-Hole actions play an important role in industrial assembly. In this paper, we present a system which performs such actions after learning by human demonstration. Strategies for getting robust human-demonstrated PiH trajectories are investigated, and a number of PiH experiments are conducted to verify the performance of the proposed approach. The results show that our approach leads to a high PiH success rate and allows the robot to fine-tune the PiH trajectories to achieve faster performance.
---
paper_title: From Demonstrations to Skills for High-level Programming of Industrial Robots

paper_content:

In this paper we describe our approach to robotic skill representation and a prototypical implementation of a programming-by-demonstration approach that allows users to generate skills and robot program primitives for later re- finement and re-use. We intend to evaluate the applicability of this approach to high-level programming in a user study, which we also explain.
---
paper_title: Robot programming from demonstration, feedback and transfer

paper_content:

This paper presents a novel approach for robot instruction for assembly tasks. We consider that robot programming can be made more efficient, precise and intuitive if we leverage the advantages of complementary approaches such as learning from demonstration, learning from feedback and knowledge transfer. Starting from low-level demonstrations of assembly tasks, the system is able to extract a high-level relational plan of the task. A graphical user interface (GUI) allows then the user to iteratively correct the acquired knowledge by refining high-level plans, and low-level geometrical knowledge of the task. This combination leads to a faster programming phase, more precise than just demonstrations, and more intuitive than just through a GUI. A final process allows to reuse high-level task knowledge for similar tasks in a transfer learning fashion. Finally we present a user study illustrating the advantages of this approach.
---
paper_title: Force controlled robotic assembly without a force sensor

paper_content:

The traditional way of controlling an industrial robot is to program it to follow desired trajectories. This approach is sufficient as long as the accuracy of the robot and the calibration of the workcell is good enough. In robotic assembly these conditions are usually not fulfilled, because of uncertainties, e.g., variability in involved parts and objects not gripped accurately. Using force control is one way to handle these difficulties. This paper presents a method of doing force control without a force sensor. The method is based on detuning of the low-level joint control loops, and the force is estimated from the control error. It is experimentally verified in a small part assembly task with a kinematically redundant robotic manipulator.
---
paper_title: Robotic assembly solution by human-in-the-loop teaching method based on real-time stiffness modulation

paper_content:

We propose a novel human-in-the-loop approach for teaching robots how to solve assembly tasks in unpredictable and unstructured environments. In the proposed method the human sensorimotor system is integrated into the robot control loop though a teleoperation setup. The approach combines a 3-DoF end-effector force feedback with an interface for modulation of the robot end-effector stiffness. When operating in unpredictable and unstructured environments, modulation of limb impedance is essential in terms of successful task execution, stability and safety. We developed a novel hand-held stiffness control interface that is controlled by the motion of the human finger. A teaching approach was then used to achieve autonomous robot operation. In the experiments, we analysed and solved two part-assembly tasks: sliding a bolt fitting inside a groove and driving a self-tapping screw into a material of unknown properties. We experimentally compared the proposed method to complementary robot learning methods and analysed the potential benefits of direct stiffness modulation in the force-feedback teleoperation.
---
paper_title: Automatic error recovery in robot assembly operations using reverse execution

paper_content:

Robotic assembly tasks are in general difficult to program and require a high degree of precision. As the complexity of the task increases it becomes increasingly unlikely that tasks can always be executed without errors. Preventing errors beyond a certain point is economically infeasible, in particular for small-batch productions. As an alternative, we propose a system for automatically handling certain classes of errors instead of preventing them. Specifically, we show that many operations can be automatically reversed. Errors can be handled through automatic reverse execution of the control program to a safe point, from which forward execution can be resumed. This paper describes the principles behind automatic reversal of robotic assembly operations, and experimentally demonstrates the use of a domain-specific language that supports automatic error handling through reverse execution. Our contribution represents the first experimental demonstration of reversible computing principles applied to industrial robotics.
---
paper_title: Full-Body Compliant Human–Humanoid Interaction: Balancing in the Presence of Unknown External Forces

paper_content:

This paper proposes an effective framework of human-humanoid robot physical interaction. Its key component is a new control technique for full-body balancing in the presence of external forces, which is presented and then validated empirically. We have adopted an integrated system approach to develop humanoid robots. Herein, we describe the importance of replicating human-like capabilities and responses during human-robot interaction in this context. Our balancing controller provides gravity compensation, making the robot passive and thereby facilitating safe physical interactions. The method operates by setting an appropriate ground reaction force and transforming these forces into full-body joint torques. It handles an arbitrary number of force interaction points on the robot. It does not require force measurement at interested contact points. It requires neither inverse kinematics nor inverse dynamics. It can adapt to uneven ground surfaces. It operates as a force control process, and can therefore, accommodate simultaneous control processes using force-, velocity-, or position-based control. Forces are distributed over supporting contact points in an optimal manner. Joint redundancy is resolved by damping injection in the context of passivity. We present various force interaction experiments using our full-sized bipedal humanoid platform, including compliant balance, even when affected by unknown external forces, which demonstrates the effectiveness of the method.
---
paper_title: An exoskeleton robot for human arm movement study

paper_content:

A new experimental platform permits us to study a novel variety of issues of human motor control, particularly full 3D movements involving the major seven degrees-of-freedom (DOF) of the human arm. We incorporate a seven DOF robot exoskeleton, and minimize weight and inertia through gravity, Coriolis, and inertia compensation, such that subjects' arm movements are largely unaffected by the manipulandum. Torque perturbations can be individually applied to any or all seven joints of the human arm, thus creating novel dynamic environments, or force fields, for subjects to respond and adapt to. Our first study investigates a joint space force field where the shoulder velocity drives a disturbing force in the elbow joint. Results demonstrate that subjects learn to compensate for the force field within about 100 trials, and, from the strong presence of aftereffects when removing the field in some randomized catch trials, that an inverse dynamics, or internal model, of the force field is formed by the nervous system. Interestingly, while after learning, hand trajectories return to baseline, joint space trajectories remained changed in response to the field, indicating that, besides learning a model of the force field, the nervous system also chose to exploit the null space to minimize the effects of the force field on the realization of the endpoint trajectory plan. We discuss applications of these results in the light of current theories of robotic control, including inverse kinematics and optimal control.
---
paper_title: Reinforcement learning of motor skills with policy gradients

paper_content:

Autonomous learning is one of the hallmarks of human and animal behavior, and understanding the principles of learning will be crucial in order to achieve true autonomy in advanced machines like humanoid robots. In this paper, we examine learning of complex motor skills with human-like limbs. While supervised learning can offer useful tools for bootstrapping behavior, e.g., by learning from demonstration, it is only reinforcement learning that offers a general approach to the final trial-and-error improvement that is needed by each individual acquiring a skill. Neither neurobiological nor machine learning studies have, so far, offered compelling results on how reinforcement learning can be scaled to the high-dimensional continuous state and action spaces of humans or humanoids. Here, we combine two recent research developments on learning motor control in order to achieve this scaling. First, we interpret the idea of modular motor control by means of motor primitives as a suitable way to generate parameterized control policies for reinforcement learning. Second, we combine motor primitives with the theory of stochastic policy gradient learning, which currently seems to be the only feasible framework for reinforcement learning for humanoids. We evaluate different policy gradient methods with a focus on their applicability to parameterized motor primitives. We compare these algorithms in the context of motor primitive learning, and show that our most modern algorithm, the Episodic Natural Actor-Critic outperforms previous algorithms by at least an order of magnitude. We demonstrate the efficiency of this reinforcement learning method in the application of learning to hit a baseball with an anthropomorphic robot arm.
---
paper_title: Dynamical System Modulation for Robot Learning via Kinesthetic Demonstrations

paper_content:

We present a system for robust robot skill acquisition from kinesthetic demonstrations. This system allows a robot to learn a simple goal-directed gesture and correctly reproduce it despite changes in the initial conditions and perturbations in the environment. It combines a dynamical system control approach with tools of statistical learning theory and provides a solution to the inverse kinematics problem when dealing with a redundant manipulator. The system is validated on two experiments involving a humanoid robot: putting an object into a box and reaching for and grasping an object.
---
paper_title: On learning the statistical representation of a task and generalizing it to various contexts

paper_content:

This paper presents an architecture for solving generically the problem of extracting the constraints of a given task in a programming by demonstration framework and the problem of generalizing the acquired knowledge to various contexts. We validate the architecture in a series of experiments, where a human demonstrator teaches a humanoid robot simple manipulatory tasks. First, the combined joint angles and hand path motions are projected into a generic latent space, composed of a mixture of Gaussians (GMM) spreading across the spatial dimensions of the motion. Second, the temporal variation of the latent representation of the motion is encoded in a hidden Markov model (HMM). This two-step probabilistic encoding provides a measure of the spatio-temporal correlations across the different modalities collected by the robot, which determines a metric of imitation performance. A generalization of the demonstrated trajectories is then performed using Gaussian mixture regression (GMR). Finally, to generalize skills across contexts, we compute formally the trajectory that optimizes the metric, given the new context and the robot's specific body constraints
---
paper_title: Teaching robots the use of human tools from demonstration with non-dexterous end-effectors

paper_content:

Commercial, affordable and general-purpose robots like the PR-2, Baxter and UBR-1 robots can take over a wide range of tasks or assist human workers in a mixed human-robot environment. However, end-effectors on these robots are usually restricted to low-cost, non-dexterous grippers which constrains the application scenarios. We aim at increasing their range by teaching such robots the use of human tools by demonstration. We present a novel and compact model for the use of human tools and propose a dual-gripper strategy in replacement of the much less widely deployed dexterous hand for tool manipulation. Especially, we propose a hierarchical architecture to embed tool use in a learning from demonstration framework, learning temporal order for dual-arm coordination at higher level and Dynamic Movement Primitives at lower level for a multi-step execution. The approach is demonstrated and evaluated on a Baxter research robot for three human tools.
---
paper_title: On Learning, Representing, and Generalizing a Task in a Humanoid Robot

paper_content:

We present a programming-by-demonstration framework for generically extracting the relevant features of a given task and for addressing the problem of generalizing the acquired knowledge to different contexts. We validate the architecture through a series of experiments, in which a human demonstrator teaches a humanoid robot simple manipulatory tasks. A probability-based estimation of the relevance is suggested by first projecting the motion data onto a generic latent space using principal component analysis. The resulting signals are encoded using a mixture of Gaussian/Bernoulli distributions (Gaussian mixture model/Bernoulli mixture model). This provides a measure of the spatio-temporal correlations across the different modalities collected from the robot, which can be used to determine a metric of the imitation performance. The trajectories are then generalized using Gaussian mixture regression. Finally, we analytically compute the trajectory which optimizes the imitation metric and use this to generalize the skill to different contexts
---
paper_title: Demonstration-Guided Motion Planning

paper_content:

We present demonstration-guided motion planning (DGMP), a new frame-work for planning motions for personal robots to perform household tasks. DGMP combines the strengths of sampling-based motion planning and robot learning from demonstrations to generate plans that (1) avoid novel obstacles in cluttered environments, and (2) learn and maintain critical aspects of the motion required to successfully accomplish a task. Sampling-based motion planning methods are highly effective at computing paths from start to goal configurations that avoid obstacles, but task constraints (e.g. a glass of water must be held upright to avoid a spill) must be explicitly enumerated and programmed. Instead, we use a set of expert demonstrations and automatically extract time-dependent task constraints by learning low variance aspects of the demonstrations, which are correlated with the task constraints. We then introduce multi-component rapidly-exploring roadmaps (MC-RRM), a sampling-based method that incrementally computes a motion plan that avoids obstacles and optimizes a learned cost metric. We demonstrate the effectiveness of DGMP using the Aldebaran Nao robot performing household tasks in a cluttered environment, including moving a spoon full of sugar from a bowl to a cup and cleaning the surface of a table.
---
paper_title: Humanoid robot motion generation with sequential physical constraints

paper_content:

This paper presents a method to optimize and filter trajectories generated from recorded human motion for a humanoid robot with physical limits. The objective function is responsible for mimicking human trainers, enhancing the possibility for fast convergence, while constraints are used to transform motion within the limit of the capabilities of the humanoid robot. Those constraints for angle, velocity, and dynamic force are represented as B-spline coefficients. An iterative soft-constraint paradigm is proposed to enhance the quality of velocity and force constraints. Collision avoidance is also considered as a constraint. For precision refinement, in regions of high-frequency motion not adequately modeled by an initial splining, B-spline is extensible into a hierarchy so that optimization that meets global criteria can be performed locally. Furthermore, all of the constraints can be used solely as filtering. To use these filters, an effective method to directly decompose a trajectory to a B-spline is also presented
---
paper_title: Scaffolding on-line segmentation of full body human motion patterns

paper_content:

This paper develops an approach for on-line segmentation of whole body human motion patterns during human motion observation and learning. A Hidden Markov Model is used to represent the incoming data sequence, where each model state represents the probability density estimate over a window of the data. Based on the assumption that data belonging to the same motion primitive will have the same underlying distribution, the segmentation is implemented by finding the optimum state sequence over the developed model. The basic algorithm is modified to add the capability for modifying the model based on known motion primitives. The inclusion of such scaffolding motion primitives can improve the performance of the basic segmentation algorithm. The modified algorithm is tested on a corpus of continuous human motion data to show the efficacy of the proposed approach.
---
paper_title: Programming full-body movements for humanoid robots by observation

paper_content:

Abstract The formulation and optimization of joint trajectories for humanoid robots is quite different from this same task for standard robots because of the complexity of humanoid robots’ kinematics and dynamics. In this paper we exploit the similarity between human motion and humanoid robot motion to generate joint trajectories for humanoids. In particular, we show how to transform human motion information captured by an optical tracking device into a high dimensional trajectory for a humanoid robot. We propose an automatic approach to relate humanoid robot kinematic parameters to the kinematic parameters of a human performer. Based on this relationship we infer the desired trajectories in robot joint space. B-spline wavelets are utilized to efficiently represent the trajectories. The density of the basis functions on the time axis is selected automatically. Large-scale optimization techniques are employed to solve the underlying computational problems efficiently. We applied our method to the task of teaching a humanoid robot how to make various naturally looking movements.
---
paper_title: Programming-by-Demonstration of reaching motions—A next-state-planner approach

paper_content:

This paper presents a novel approach to skill acquisition from human demonstration. A robot manipulator with a morphology which is very different from the human arm simply cannot copy a human motion, but has to execute its own version of the skill. When a skill once has been acquired the robot must also be able to generalize to other similar skills, without a new learning process. By using a motion planner that operates in an object-related world frame called hand-state, we show that this representation simplifies skill reconstruction and preserves the essential parts of the skill.
---
paper_title: Methods for Motion Generation and Interaction with a Humanoid Robot: Case Studies of Dancing and Catching

paper_content:

We focus on creating realistic, adaptable movement for humanoid robots and virtual characters. Here we present motion synthesis of dance movements for a humanoid robot, and interactive behavior for catching. Our approach to motion generation includes collection of example human movements, handling of marker occlusion, extraction of motion parameters, and trajectory generation, all of which must be handled in such a way as to be faithful to the style of the original movements. In our interactive behavior, we generate ball-glove impact predictions and intercept motion trajectories for a real time catching task. In this paper we present our results and discuss ideas for future improvements.
---
paper_title: Incremental learning of full body motion primitives and their sequencing through human motion observation

paper_content:

In this paper we describe an approach for on-line, incremental learning of full body motion primitives from observation of human motion. The continuous observation sequence is first partitioned into motion segments, using stochastic segmentation. Next, motion segments are incrementally clustered and organized into a hierarchical tree structure representing the known motion primitives. Motion primitives are encoded using hidden Markov models, so that the same model can be used for both motion recognition and motion generation. At the same time, the temporal relationship between motion primitives is learned via the construction of a motion primitive graph. The motion primitive graph can then be used to construct motions, consisting of sequences of motion primitives. The approach is implemented and tested during on-line observation and on the IRT humanoid robot.
---
paper_title: Adapting human motion for the control of a humanoid robot

paper_content:

Using the pre-recorded human motion and trajectory tracking, we can control the motion of a humanoid robot for free-space, upper body gestures. However, the number of degrees of freedom, range of joint motion, and achievable joint velocities of today's humanoid robots are far more limited than those of the average human subject. In this paper, we explore a set of techniques for limiting human motion of upper body gestures to that achievable by a Sarcos humanoid robot located at ATR. We assess the quality of the results by comparing the motion of the human actor to that of the robot, both visually and quantitatively.
---
paper_title: Feeling the force: Integrating force and pose for fluent discovery through imitation learning to open medicine bottles

paper_content:

Learning complex robot manipulation policies for real-world objects is challenging, often requiring significant tuning within controlled environments. In this paper, we learn a manipulation model to execute tasks with multiple stages and variable structure, which typically are not suitable for most robot manipulation approaches. The model is learned from human demonstration using a tactile glove that measures both hand pose and contact forces. The tactile glove enables observation of visually latent changes in the scene, specifically the forces imposed to unlock the child-safety mechanisms of medicine bottles. From these observations, we learn an action planner through both a top-down stochastic grammar model (And-Or graph) to represent the compositional nature of the task sequence and a bottom-up discriminative model from the observed poses and forces. These two terms are combined during planning to select the next optimal action. We present a method for transferring this human-specific knowledge onto a robot platform and demonstrate that the robot can perform successful manipulations of unseen objects with similar task structure.
---
paper_title: Recognition of affective communicative intent in robot-directed speech

paper_content:

Human speech provides a natural and intuitive interface for both communicating with humanoid robots as well as for teaching them. In general, the acoustic pattern of speech contains three kinds of information: who the speaker is, what the speaker said, and how the speaker said it. This paper focuses on the question of recognizing affective communicative intent in robot-directed speech without looking into the linguistic content. We present an approach for recognizing four distinct prosodic patterns that communicate praise, prohibition, attention, and comfort to preverbal infants. These communicative intents are well matched to teaching a robot since praise, prohibition, and directing the robot's attention to relevant aspects of a task, could be used by a human instructor to intuitively facilitate the robot's learning process. We integrate this perceptual ability into our robot's “emotion” system, thereby allowing a human to directly manipulate the robot's affective state. This has a powerful organizing influence on the robot's behavior, and will ultimately be used to socially communicate affective reinforcement. Communicative efficacy has been tested with people very familiar with the robot as well as with naïve subjects.
---
paper_title: Teaching a Robot the Semantics of Assembly Tasks

paper_content:

We present a three-level cognitive system in a learning by demonstration context. The system allows for learning and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level representation based on so-called “semantic event chains.” We describe details of the representations and quantify the effect of the associated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the overall system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6.
---
paper_title: Learning and generalization of motor skills by learning from demonstration

paper_content:

We provide a general approach for learning robotic motor skills from human demonstration. To represent an observed movement, a non-linear differential equation is learned such that it reproduces this movement. Based on this representation, we build a library of movements by labeling each recorded movement according to task and context (e.g., grasping, placing, and releasing). Our differential equation is formulated such that generalization can be achieved simply by adapting a start and a goal parameter in the equation to the desired position values of a movement. For object manipulation, we present how our framework extends to the control of gripper orientation and finger position. The feasibility of our approach is demonstrated in simulation as well as on the Sarcos dextrous robot arm. The robot learned a pick-and-place operation and a water-serving task and could generalize these tasks to novel situations.
---
paper_title: A computational model of intention reading in imitation

paper_content:

Abstract Imitation in artificial systems involves a number of important aspects, such as extracting the relevant features of the demonstrated behaviour, inverse mapping observations, and executing motor commands. In this article we focus on how an artificial system can infer what the demonstrator intended to do. The model that we propose draws inspiration from developmental psychology and contains three crucial features. One is that the imitating agent needs repeated trials, thus stepping away from the one-shot learning by demonstration paradigm. The second is that the imitating agent needs a learning method in which it keeps track of intentions to reach goals. The third feature is that the model does not require an external measure of equivalence; instead, the demonstrator decides whether an attempted imitation was equivalent to the demonstration or not. We present a computational model and its simulation results, which underpin our theory of goal-directed imitation on an artificial system.
---
paper_title: Exploring interactive simulator in collaborative multi-site teleoperation

paper_content:

We have developed collision-free coordination technologies for multiple robots in a common environment remotely controlled from multiple operators over the network with large physical separation. Sensor guided approaches would make remote robot systems rather complicated and hard to control accordingly. The paper reviews human supervisory control approaches based on a delay-tolerant predictive graphics simulator, where different coordination strategies are incorporated to cope with the operator's delayed visual perception over the network. We have built an experimental test bed, where operators control their master to get two slave robots to cooperate in a common task. They detect and/or feel a priori the possibility of collision through the local graphics simulator that runs in near real-time. We have demonstrated various tasks by two slave robots and two operators over a LAN subject to the simulated delays and evaluated the simulator guided approach in multi-site tele-collaboration.
---
paper_title: An Embodied Cognition Approach to Mindreading Skills for Socially Intelligent Robots

paper_content:

Future applications for personal robots motivate research into developing robots that are intelligent in their interactions with people. Toward this goal, in this paper we present an integrated socio-cognitive architecture to endow an anthropomorphic robot with the ability to infer mental states such as beliefs, intents, and desires from the observable behavior of its human partner. The design of our architecture is informed by recent findings from neuroscience and embodies cognition that reveals how living systems leverage their physical and cognitive embodiment through simulation-theoretic mechanisms to infer the mental states of others. We assess the robot's mindreading skills on a suite of benchmark tasks where the robot interacts with a human partner in a cooperative scenario and a learning scenario. In addition, we have conducted human subjects experiments using the same task scenarios to assess human performance on these tasks and to compare the robot's performance with that of people. In the process, our human subject studies also reveal some interesting insights into human behavior.
---
paper_title: Programing by Demonstration: Coping with Suboptimal Teaching Actions

paper_content:

The difficulty associated with programing existing robots is one of the main impediments to them finding application in domestic environments such as the home. A promising method for simplifying robot programing is Programing by Demonstration (PbD). Here, an end user can provide a demonstration of the task to be programed, with a PbD “interface” interpreting the demonstration in order to determine low-level control details for the robot. A key aspect of the interpretation process is to make it robust to the noise typically included in a demonstration by the human. In this paper we present a method to help identify and eliminate any noise present in the demonstration. Our method involves two steps. The first step uses the demonstration to build up a partial knowledge of the geometry present in the task. Statistical regression analysis is used on demonstrated trajectories to determine equations describing curved surfaces in configuration space. The second step in our method uses the geometric information ob...
---
paper_title: Incremental Learning of Tasks From User Demonstrations, Past Experiences, and Vocal Comments

paper_content:

Since many years the robotics community is envisioning robot assistants sharing the same environment with humans. It became obvious that they have to interact with humans and should adapt to individual user needs. Especially the high variety of tasks robot assistants will be facing requires a highly adaptive and user-friendly programming interface. One possible solution to this programming problem is the learning-by-demonstration paradigm, where the robot is supposed to observe the execution of a task, acquire task knowledge, and reproduce it. In this paper, a system to record, interpret, and reason over demonstrations of household tasks is presented. The focus is on the model-based representation of manipulation tasks, which serves as a basis for incremental reasoning over the acquired task knowledge. The aim of the reasoning is to condense and interconnect the data, resulting in more general task knowledge. A measure for the assessment of information content of task features is introduced. This measure for the relevance of certain features relies both on general background knowledge as well as task-specific knowledge gathered from the user demonstrations. Beside the autonomous information estimation of features, speech comments during the execution, pointing out the relevance of features are considered as well. The results of the incremental growth of the task knowledge when more task demonstrations become available and their fusion with relevance information gained from speech comments is demonstrated within the task of laying a table
---
paper_title: Sensor-based path-planning of multiple mobile robots to overcome large transmission delays in teleoperation

paper_content:

We describe the robustness of our sensor-based path-planning algorithm of multiple mobile robots in teleoperation with large transmission delays between operation and remote sites. First of all, we propose a teleoperation system with shared autonomy. All the mobile robots at a remote site have a common set of sensor-based traffic rules, and therefore they avoid each other and arrive at their destinations autonomously. Then, a human at an operation site directs a robot. In an extended version of our sensor-based path-planning algorithm, if an operated robot comes close to some of the other robots, whose distance equals to or is smaller than a threshold D, the operated robot automatically stops and some robots autonomously avoid the operated robot. This is a kind of fail-safe function at a remote site. After the interference, the other robots asymptotically arrive at their destinations by the distributed sensor-based path-planning algorithm, and also the operated robot is directly conducted to the destination by a human being. This is a kind of autonomy at a remote site. Finally, the usefulness of fail-safe operation and autonomy is ascertained in a few simulation results.
---
paper_title: Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives

paper_content:

Acquisition of new sensorimotor knowledge by imitation is a promising paradigm for robot learning. To be effective, action learning should not be limited to direct replication of movements obtained during training but must also enable the generation of actions in situations a robot has never encountered before. This paper describes a methodology that enables the generalization of the available sensorimotor knowledge. New actions are synthesized by the application of statistical methods, where the goal and other characteristics of an action are utilized as queries to create a suitable control policy, taking into account the current state of the world. Nonlinear dynamic systems are employed as a motor representation. The proposed approach enables the generation of a wide range of policies without requiring an expert to modify the underlying representations to account for different task-specific features and perceptual feedback. The paper also demonstrates that the proposed methodology can be integrated with an active vision system of a humanoid robot. 3-D vision data are used to provide query points for statistical generalization. While 3-D vision on humanoid robots with complex oculomotor systems is often difficult due to the modeling uncertainties, we show that these uncertainties can be accounted for by the proposed approach.
---
paper_title: A pilot study in vision-based augmented telemanipulation for remote assembly over high-latency networks

paper_content:

In this paper we present an approach to extending the capabilities of telemanipulation systems by intelligently augmenting a human operator's motion commands based on quantitative three-dimensional scene perception at the remote telemanipulation site. This framework is the first prototype of the Augmented Shared-Control for Efficient, Natural Telemanipulation (ASCENT) System. ASCENT aims to enable new robotic applications in environments where task complexity precludes autonomous execution or where low-bandwidth and/or high-latency communication channels exist between the nearest human operator and the application site. These constraints can constrain the domain of telemanipulation to simple or static environments, reduce the effectiveness of telemanipulation, and even preclude remote intervention entirely. ASCENT is a semi-autonomous framework that increases the speed and accuracy of a human operator's actions via seamless transitions between one-to-one teleoperation and autonomous interventions. We report the promising results of a pilot study validating ASCENT in a transatlantic telemanipulation experiment between The Johns Hopkins University in Baltimore, MD, USA and the German Aerospace Center (DLR) in Oberpfaffenhofen, Germany. In these experiments, we observed average telemetry delays of 200ms, and average video delays of 2s with peaks of up to 6s for all data. We also observed 75% frame loss for video streams due to bandwidth limits, giving 4fps video.
---
paper_title: Movement imitation with nonlinear dynamical systems in humanoid robots

paper_content:

Presents an approach to movement planning, on-line trajectory modification, and imitation learning by representing movement plans based on a set of nonlinear differential equations with well-defined attractor dynamics. The resultant movement plan remains an autonomous set of nonlinear differential equations that forms a control policy (CP) which is robust to strong external perturbations and that can be modified on-line by additional perceptual variables. We evaluate the system with a humanoid robot simulation and an actual humanoid robot. Experiments are presented for the imitation of three types of movements: reaching movements with one arm, drawing movements of 2-D patterns, and tennis swings. Our results demonstrate (a) that multi-joint human movements can be encoded successfully by the CPs, (b) that a learned movement policy can readily be reused to produce robust trajectories towards different targets, (c) that a policy fitted for one particular target provides a good predictor of human reaching movements towards neighboring targets, and (d) that the parameter space which encodes a policy is suitable for measuring to which extent two trajectories are qualitatively similar.
---
paper_title: Goal-Directed Imitation in a Humanoid Robot

paper_content:

Our work aims at developing a robust discriminant controller for robot programming by demonstration. It addresses two core issues of imitation learning, namely “what to imitate” and “how to imitate”. This paper presents a method by which a robot extracts the goals of a demonstrated task and determines the imitation strategy that satisfies best these goals. The method is validated in a humanoid platform, taking inspiration of an influential experiment from developmental psychology.
---
paper_title: Learning human arm movements by imitation: : Evaluation of a biologically inspired connectionist architecture

paper_content:

Abstract This paper evaluates a model of human imitation of abstract, two-arm movements. The model consists of a hierarchy of artificial neural networks, which are abstractions of brain regions involved in visuo-motor control. The model is validated in a biomechanical simulation of a 37 degree of freedom (DOF) humanoid. Input to the model are data from human arm movements recorded using video and marker-based tracking systems. Results show a high qualitative and quantitative agreement with human data. The model’s reproduction is better or comparable to that of human subjects imitating the same movements.
---
paper_title: Enabling effective human-robot interaction using perspective-taking in robots

paper_content:

We propose that an important aspect of human-robot interaction is perspective-taking. We show how perspective-taking occurs in a naturalistic environment (astronauts working on a collaborative project) and present a cognitive architecture for performing perspective-taking called Polyscheme. Finally, we show a fully integrated system that instantiates our theoretical framework within a working robot system. Our system successfully solves a series of perspective-taking problems and uses the same frames of references that astronauts do to facilitate collaborative problem solving with a person.
---
paper_title: A model of shared grasp affordances from demonstration

paper_content:

This paper presents a hierarchical, statistical topic model for representing the grasp preshapes of a set of objects. Observations provided by teleoperation are clustered into latent affordances shared among all objects. Each affordance defines a joint distribution over position and orientation of the hand relative to the object and conditioned on visual appearance. The parameters of the model are learned using a Gibbs sampling method. After training, the model can be used to compute grasp preshapes for a novel object based on its visual appearance. The model is evaluated experimentally on a set of objects for its ability to generate grasp preshapes that lead to successful grasps, and compared to a baseline approach.
---
paper_title: On Learning, Representing, and Generalizing a Task in a Humanoid Robot

paper_content:

We present a programming-by-demonstration framework for generically extracting the relevant features of a given task and for addressing the problem of generalizing the acquired knowledge to different contexts. We validate the architecture through a series of experiments, in which a human demonstrator teaches a humanoid robot simple manipulatory tasks. A probability-based estimation of the relevance is suggested by first projecting the motion data onto a generic latent space using principal component analysis. The resulting signals are encoded using a mixture of Gaussian/Bernoulli distributions (Gaussian mixture model/Bernoulli mixture model). This provides a measure of the spatio-temporal correlations across the different modalities collected from the robot, which can be used to determine a metric of the imitation performance. The trajectories are then generalized using Gaussian mixture regression. Finally, we analytically compute the trajectory which optimizes the imitation metric and use this to generalize the skill to different contexts
---
paper_title: Computational Approaches to Motor Learning by Imitation

paper_content:

Movement imitation requires a complex set of mechanisms that map an observed movement of a teacher onto one's own movement apparatus. Relevant problems include movement recognition, pose estimation, pose tracking, body correspondence, coordinate transformation from external to egocentric space, matching of observed against previously learned movement, resolution of redundant degrees-of-freedom that are unconstrained by the observation, suitable movement representations for imitation, modularization of motor control, etc. All of these topics by themselves are active research problems in computational and neurobiological sciences, such that their combination into a complete imitation system remains a daunting undertaking-indeed, one could argue that we need to understand the complete perception-action loop. As a strategy to untangle the complexity of imitation, this paper will examine imitation purely from a computational point of view, i.e. we will review statistical and mathematical approaches that have been suggested for tackling parts of the imitation problem, and discuss their merits, disadvantages and underlying principles. Given the focus on action recognition of other contributions in this special issue, this paper will primarily emphasize the motor side of imitation, assuming that a perceptual system has already identified important features of a demonstrated movement and created their corresponding spatial information. Based on the formalization of motor control in terms of control policies and their associated performance criteria, useful taxonomies of imitation learning can be generated that clarify different approaches and future research directions.
---
paper_title: Motion learning in variable environments using probabilistic flow tubes

paper_content:

Commanding an autonomous system through complex motions at a low level can be tedious or impractical for systems with many degrees of freedom. Allowing an operator to demonstrate the desired motions directly can often enable more intuitive and efficient interaction. Two challenges in the field of learning from demonstration include (1) how to best represent learned motions to accurately reflect a human's intentions, and (2) how to enable learned motions to be easily applicable in new situations. This paper introduces a novel representation of continuous actions called probabilistic flow tubes that can provide flexibility during execution while robustly encoding a human's intended motions. Our approach also automatically determines certain qualitative characteristics of a motion so that these characteristics can be preserved when autonomously executing the motion in a new situation. We demonstrate the effectiveness of our motion learning approach both in a simulated two-dimensional environment and on the All-Terrain Hex-Limbed Extra-Terrestrial Explorer (ATHLETE) robot performing object manipulation tasks.
---
paper_title: Imitation Learning of Dual-Arm Manipulation Tasks in Humanoid Robots

paper_content:

In this paper, we deal with imitation learning of arm movements in humanoid robots. Hidden Markov Models (HMM) are used to generalize movements demonstrated to a robot multiple times. They are trained with the characteristic features (key points) of each demonstration. Using the same HMM, key points that are common to all demonstrations are identified; only those are considered when reproducing a movement. We also show how HMM can be used to detect temporal dependencies between both arms in dual-arm tasks. We created a model of the human upper body to simulate the reproduction of dual-arm movements and generate natural-looking joint configurations from tracked hand paths. Results are presented and discussed.
---
paper_title: Learning robot motion control with demonstration and advice-operators

paper_content:

As robots become more commonplace within society, the need for tools to enable non-robotics-experts to develop control algorithms, or policies, will increase. Learning from demonstration (LfD) offers one promising approach, where the robot learns a policy from teacher task executions. Our interests lie with robot motion control policies which map world observations to continuous low-level actions. In this work, we introduce advice-operator policy improvement (A-OPI) as a novel approach for improving policies within LfD. Two distinguishing characteristics of the A-OPI algorithm are data source and continuous state-action space. Within LfD, more example data can improve a policy. In A-OPI, new data is synthesized from a student execution and teacher advice. By contrast, typical demonstration approaches provide the learner with exclusively teacher executions. A-OPI is effective within continuous state-action spaces because high level human advice is translated into continuous-valued corrections on the student execution. This work presents a first implementation of the A-OPI algorithm, validated on a Segway RMP robot performing a spatial positioning task. A-OPI is found to improve task performance, both in success and accuracy. Furthermore, performance is shown to be similar or superior to the typical exclusively teacher demonstrations approach.
---
paper_title: Prediction of intent in robotics and multi-agent systems

paper_content:

Moving beyond the stimulus contained in observable agent behaviour, i.e. understanding the underlying intent of the observed agent is of immense interest in a variety of domains that involve collaborative and competitive scenarios, for example assistive robotics, computer games, robot–human interaction, decision support and intelligent tutoring. This review paper examines approaches for performing action recognition and prediction of intent from a multi-disciplinary perspective, in both single robot and multi-agent scenarios, and analyses the underlying challenges, focusing mainly on generative approaches.
---
paper_title: Teleoperation for learning by demonstration: Data glove versus object manipulation for intuitive robot control

paper_content:

Learning by demonstration is a useful technique to augment a robot's behavioral inventory, and teleoperation allows lay users to demonstrate novel behaviors intuitively to the robot. In this paper, we compare two modes of teleoperation of an industrial robot, the demonstration by means of a data glove and by means of a control object (peg). Experiments with 16 lay users, performing assembly task on the Cranfield benchmark objects, show that the control peg leads to more success, more efficient demonstration and fewer errors.
---
paper_title: Task-level imitation learning using variance-based movement optimization

paper_content:

Recent advances in the field of humanoid robotics increase the complexity of the tasks that such robots can perform. This makes it increasingly difficult and inconvenient to program these tasks manually. Furthermore, humanoid robots, in contrast to industrial robots, should in the distant future behave within a social environment. Therefore, it must be possible to extend the robot's abilities in an easy and natural way. To address these requirements, this work investigates the topic of imitation learning of motor skills. The focus lies on providing a humanoid robot with the ability to learn new bi-manual tasks through the observation of object trajectories. For this, an imitation learning framework is presented, which allows the robot to learn the important elements of an observed movement task by application of probabilistic encoding with Gaussian Mixture Models. The learned information is used to initialize an attractor-based movement generation algorithm that optimizes the reproduced movement towards the fulfillment of additional criteria, such as collision avoidance. Experiments performed with the humanoid robot ASIMO show that the proposed system is suitable for transferring information from a human demonstrator to the robot. These results provide a good starting point for more complex and interactive learning tasks.
---
paper_title: Teleplanning by human demonstration for VR-based teleoperation of a mobile robotic assistant

paper_content:

Focuses on the integration of local path planning techniques in a multimodal teleoperation interface, for the efficient remote control of a mobile robotic assistant. The main principle underlying this scheme is related to finding new ways to establish an efficient human-robot cooperation framework, where humans and robots take charge of the parts of the tasks that they can perform more efficiently. For the teleoperation of a mobile robotic platform, a simple application of this general principle could be to commit the human operator in performing the necessary global planning operations, which are more demanding in terms of complex reasoning and required "intelligence", while other more local tasks such as collision avoidance and trajectory optimization are dedicated to the telerobotic system. We propose an implementation of this principle within a mobile robot teleoperation interface integrating virtual reality techniques and Web standards. The paper describes the multimodal interface and the design principles followed, as well as the integration of a local path planning method. This scheme, called "computer-assisted teleplanning by human demonstration", aims at providing active assistance to the human operator, enabling him to indicate in a natural way the desired global motion plan, for a more efficient teleoperation of a mobile robotic assistant.
---
paper_title: Learning Robot Manipulation Tasks With Task-Parameterized Semitied Hidden Semi-Markov Model

paper_content:

In this paper, we investigate the semi-tied Gaussian mixture models for robust learning and adaptation of robot manipulation tasks. We make use of the spatial and temporal correlation in the data by tying the covariance matrices of the mixture model with common synergistic directions/basis vectors, instead of estimating full covariance matrices for each cluster in the mixture. This allows the reuse of the discovered synergies in different parts of the task having similar coordination patterns. We extend the approach to task-parameterized and hidden semi-Markov models for autonomous adaptation to changing environmental situations. The planned movement sequence from the model is smoothly followed with a finite horizon linear quadratic tracking controller. Experiments to encode whole body motion data in simulation, followed by valve opening and pick-and-place via obstacle avoidance tasks with the Baxter robot, show improvement over standard Gaussian mixture models with much less parameters and better generalization ability.
---
paper_title: Robonaut task learning through teleoperation

paper_content:

This paper addresses the problem of automatic skill acquisition by a robot. It reports that six trials of a reach-grasp-release-retract skill are sufficient for learning a canonical description of the task under the following circumstances: The robot is Robonaut, NASA's space-capable, dexterous humanoid. Robonaut was teleoperated by a person using full immersion Virtual Reality technology that transforms the operator's arm and hand motions into those of the robot. The operator's sole source of real-time feedback was visual. During the six trials all of the Robot's sensory inputs and motor control parameters were recorded as time-series. Later the time-series from each trial was partitioned into the same number of episodes as a function of changes in the motor parameter sequence. The episodes were time normalized and averaged across trials The resultant motor parameter sequence and sensor signals were used to control the robot without the teleoperator. The robot was able to perform the task autonomously with robot starting positions and object locations both similar to, and different from the original trials.
---
paper_title: Mimesis Scheme using a Monocular Vision System on a Humanoid Robot

paper_content:

Optical motion capturing systems are widely used to acquire human beings' motion patterns in humanoid imitation learning research. However, optical motion capturing systems have a restricted movable area. This paper proposes the HMM based mimesis scheme using a monocular camera mounted on a humanoid. This scheme releases the restriction of movable area and enables imitation in daily life environments. Also, natural human-robot-interaction is expected during imitation. From two-dimensional image sequences of the demonstrator's motion, the demonstrator's pose and motion is estimated and recognized through the mimesis model and the humanoid generates its joint motor commands for imitation in 3D space. The feasibility of the proposed scheme is demonstrated by simulation.
---
paper_title: Recognizing teleoperated manipulations

paper_content:

The many degrees-of-freedom and distributed sensing capability of dextrous robot hands permit the use of control programs that rely on qualitative changes in sensor feedback. One way of designing such a control program is to have the robot learn the qualitative control characteristics from examples. These examples may be provided via teleoperation. Results are presented for recognizing and segmenting manipulation primitives from a teleoperated task by analysis of features in sensor feedback. k-nearest quantized pattern vectors determine potential classifications. A hidden Markov model provides task context for the final segmentation. The illustrative task is picking up a plastic egg with a spatula. >
---
paper_title: Confidence-based policy learning from demonstration using Gaussian mixture models

paper_content:

We contribute an approach for interactive policy learning through expert demonstration that allows an agent to actively request and effectively represent demonstration examples. In order to address the inherent uncertainty of human demonstration, we represent the policy as a set of Gaussian mixture models (GMMs), where each model, with multiple Gaussian components, corresponds to a single action. Incrementally received demonstration examples are used as training data for the GMM set. We then introduce our confident execution approach, which focuses learning on relevant parts of the domain by enabling the agent to identify the need for and request demonstrations for specific parts of the state space. The agent selects between demonstration and autonomous execution based on statistical analysis of the uncertainty of the learned Gaussian mixture set. As it achieves proficiency at its task and gains confidence in its actions, the agent operates with increasing autonomy, eliminating the need for unnecessary demonstrations of already acquired behavior, and reducing both the training time and the demonstration workload of the expert. We validate our approach with experiments in simulated and real robot domains.
---
paper_title: Online, interactive learning of gestures for human/robot interfaces

paper_content:

We have developed a gesture recognition system, based on hidden Markov models, which can interactively recognize gestures and perform online learning of new gestures. In addition, it is able to update its model of a gesture iteratively with each example it recognizes. This system has demonstrated reliable recognition of 14 different gestures after only one or two examples of each. The system is currently interfaced to a Cyberglove for use in recognition of gestures from the sign language alphabet. The system is being implemented as part of an interactive interface for robot teleoperation and programming by example.
---
paper_title: Incremental Learning, Clustering and Hierarchy Formation of Whole Body Motion Patterns using Adaptive Hidden Markov Chains

paper_content:

This paper describes a novel approach for autonomous and incremental learning of motion pattern primitives by observation of human motion. Human motion patterns are abstracted into a dynamic stochastic model, which can be used for both subsequent motion recognition and generation, analogous to the mirror neuron hypothesis in primates. The model size is adaptable based on the discrimination requirements in the associated region of the current knowledge base. A new algorithm for sequentially training the Markov chains is developed, to reduce the computation cost during model adaptation. As new motion patterns are observed, they are incrementally grouped together using hierarchical agglomerative clustering based on their relative distance in the model space. The clustering algorithm forms a tree structure, with specialized motions at the tree leaves, and generalized motions closer to the root. The generated tree structure will depend on the type of training data provided, so that the most specialized motions will be those for which the most training has been received. Tests with motion capture data for a variety of motion primitives demonstrate the efficacy of the algorithm.
---
paper_title: Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives

paper_content:

Acquisition of new sensorimotor knowledge by imitation is a promising paradigm for robot learning. To be effective, action learning should not be limited to direct replication of movements obtained during training but must also enable the generation of actions in situations a robot has never encountered before. This paper describes a methodology that enables the generalization of the available sensorimotor knowledge. New actions are synthesized by the application of statistical methods, where the goal and other characteristics of an action are utilized as queries to create a suitable control policy, taking into account the current state of the world. Nonlinear dynamic systems are employed as a motor representation. The proposed approach enables the generation of a wide range of policies without requiring an expert to modify the underlying representations to account for different task-specific features and perceptual feedback. The paper also demonstrates that the proposed methodology can be integrated with an active vision system of a humanoid robot. 3-D vision data are used to provide query points for statistical generalization. While 3-D vision on humanoid robots with complex oculomotor systems is often difficult due to the modeling uncertainties, we show that these uncertainties can be accounted for by the proposed approach.
---
paper_title: Human action learning via hidden Markov model

paper_content:

To successfully interact with and learn from humans in cooperative modes, robots need a mechanism for recognizing, characterizing, and emulating human skills. In particular, it is our interest to develop the mechanism for recognizing and emulating simple human actions, i.e., a simple activity in a manual operation where no sensory feedback is available. To this end, we have developed a method to model such actions using a hidden Markov model (HMM) representation. We proposed an approach to address two critical problems in action modeling: classifying human action-intent, and learning human skill, for which we elaborated on the method, procedure, and implementation issues in this paper. This work provides a framework for modeling and learning human actions from observations. The approach can be applied to intelligent recognition of manual actions and high-level programming of control input within a supervisory control paradigm, as well as automatic transfer of human skills to robotic systems.
---
paper_title: Stochastic Model of Imitating a New Observed Motion Based on the Acquired Motion Primitives

paper_content:

Generally, imitation of a motion means generation of a close motion to the observation. Moreover, it means that conversion into its own motion, which is adoptable to its body structure, by integrating with its prior knowledge. From this perspective, a new imitation scheme is proposed. The scheme is based on Hidden Markov Models by employing Viterbi algorithm. The proposed scheme enables to imitate a new observed motion without learning the motion by applying its prior knowledge. Online motion primitive acquisition method is considered. Evaluation factors, such as inheritance coordinate and matching error, are introduced to evaluate imitation perfor- mance. The feasibility of the proposed scheme is demonstrated by simulation on a 20 degrees of freedom humanoid robot configuration with the evaluation factors.
---
paper_title: Teaching robots the use of human tools from demonstration with non-dexterous end-effectors

paper_content:

Commercial, affordable and general-purpose robots like the PR-2, Baxter and UBR-1 robots can take over a wide range of tasks or assist human workers in a mixed human-robot environment. However, end-effectors on these robots are usually restricted to low-cost, non-dexterous grippers which constrains the application scenarios. We aim at increasing their range by teaching such robots the use of human tools by demonstration. We present a novel and compact model for the use of human tools and propose a dual-gripper strategy in replacement of the much less widely deployed dexterous hand for tool manipulation. Especially, we propose a hierarchical architecture to embed tool use in a learning from demonstration framework, learning temporal order for dual-arm coordination at higher level and Dynamic Movement Primitives at lower level for a multi-step execution. The approach is demonstrated and evaluated on a Baxter research robot for three human tools.
---
paper_title: Hidden Markov Model Approach to Skill Learning and Its Application to Telerobotics

paper_content:

In this paper, we discuss the problem of how human skill can be represented as a parametric model using a hidden Markov model (HMM), and how an HMM-based skill model can be used to learn human skill. HMM is feasible to characterize a doubly stochastic process--measurable action and immeasurable mental states--that is involved in the skill learning. We formulated the learning problem as a multidimensional HMM and developed a testbed for a variety of skill learning applications. Based on "the most likely performance" criterion, the best action sequence can be selected from all previously measured action data by modeling the skill as an HMM. The proposed method has been implemented in the teleoperation control of space station robot system, and some important implementation issues have been discussed. The method allows a robot to learn human skill in certain tasks and to improve motion performance.
---
paper_title: Skill acquisition from human demonstration using a hidden Markov model

paper_content:

A new approach to skill acquisition in assembly is proposed. An assembly skill is represented by a hybrid dynamic system where a discrete event controller models the skill at the task level. The output of the discrete event controller provides the reference commands for the underlying robot controller. This structure is naturally encoded by a hidden Markov model (HMM). The HMM parameters are obtained by training on sensory data from human demonstrations of the skill. Currently, assembly tasks have to be performed by human operators or by robots using expensive fixtures. Our approach transfers the assembly skill from an expert human operator to the robot, thus making it possible for a robot to perform assembly tasks without the use of expensive fixtures.
---
paper_title: Interactive task training of a mobile robot through human gesture recognition

paper_content:

This paper describes a demonstration-based programming system in which a mobile robot observes the actions of a human performing a multi-step task. From these observations, the robot determines which of its pre-learned capabilities are required to replicate the task and in what sequence they must be ordered. The focus of this paper is on the hidden Markov model method used to learn and classify the actions as "gestures". A preliminary system demonstration is also described in which the robot observes the human performing a block distribution task. During the demonstration, the robot actively follows the demonstrator to maintain its vantage point and to infer spatial relationships.
---
paper_title: On Learning, Representing, and Generalizing a Task in a Humanoid Robot

paper_content:

We present a programming-by-demonstration framework for generically extracting the relevant features of a given task and for addressing the problem of generalizing the acquired knowledge to different contexts. We validate the architecture through a series of experiments, in which a human demonstrator teaches a humanoid robot simple manipulatory tasks. A probability-based estimation of the relevance is suggested by first projecting the motion data onto a generic latent space using principal component analysis. The resulting signals are encoded using a mixture of Gaussian/Bernoulli distributions (Gaussian mixture model/Bernoulli mixture model). This provides a measure of the spatio-temporal correlations across the different modalities collected from the robot, which can be used to determine a metric of the imitation performance. The trajectories are then generalized using Gaussian mixture regression. Finally, we analytically compute the trajectory which optimizes the imitation metric and use this to generalize the skill to different contexts
---
paper_title: Intent imitation using wearable motion capturing system with on-line teaching of task attention

paper_content:

In order for humanoids to imitate humans behavior, it is important to extract a needful parameter for target of imitation. Especially in daily-life environment, only simple joint angles are insufficiency because position and posture of hands and remarkable point of target object are needed for intent imitation. In this paper, we describe a development methods of motion capturing system with interactive teaching of task attention, and show its feasibility in daily-life environments
---
paper_title: A syntactic approach to robot imitation learning using probabilistic activity grammars

paper_content:

This paper describes a syntactic approach to imitation learning that captures important task structures in the form of probabilistic activity grammars from a reasonably small number of samples under noisy conditions. We show that these learned grammars can be recursively applied to help recognize unforeseen, more complicated tasks that share underlying structures. The grammars enforce an observation to be consistent with the previously observed behaviors which can correct unexpected, out-of-context actions due to errors of the observer and/or demonstrator. To achieve this goal, our method (1) actively searches for frequently occurring action symbols that are subsets of input samples to uncover the hierarchical structure of the demonstration, and (2) considers the uncertainties of input symbols due to imperfect low-level detectors. We evaluate the proposed method using both synthetic data and two sets of real-world humanoid robot experiments. In our Towers of Hanoi experiment, the robot learns the important constraints of the puzzle after observing demonstrators solving it. In our Dance Imitation experiment, the robot learns 3 types of dances from human demonstrations. The results suggest that under reasonable amount of noise, our method is capable of capturing the reusable task structures and generalizing them to cope with recursions.
---
paper_title: Incremental Semantically Grounded Learning from Demonstration

paper_content:

Much recent work in robot learning from demonstration has focused on automatically segmenting continuous task demonstrations into simpler, reusable primitives. However, strong assumptions are often made about how these primitives can be sequenced, limiting the potential for data reuse. We introduce a novel method for discovering semantically grounded primitives and incrementally building and improving a finite-state representation of a task in which various contingencies can arise. Specifically, a Beta Process Autoregressive Hidden Markov Model is used to automatically segment demonstrations into motion categories, which are then further subdivided into semantically grounded states in a finite-state automaton. During replay of the task, a data-driven approach is used to collect additional data where they are most needed through interactive corrections, which are then used to improve the finite-state automaton. Together, this allows for intelligent sequencing of primitives to create novel, adaptive behavior that can be incrementally improved as needed. We demonstrate the utility of this technique on a furniture assembly task using the PR2 mobile manipulator.
---
paper_title: Hidden Markov model for intelligent extraction of robot trajectory command from demonstrated trajectories

paper_content:

This paper proposes a scheme for selecting the best robot trajectory from a number of demonstrated trajectories. The selection scheme is based on the hidden Markov model (HMM) technique and is divided into four stages. The first stage is the representation of human demonstration by a HMM. The second stage is the preprocessing of input trajectories, which includes transformation of the position trajectory to its frequency spectrum function by the short-time Fourier transformation and mapping of the frequency spectrum function to discretized codes by vector quantization. The third stage relates to the training of the HMM. Having a number of repeated demonstrations, we get multiple observation sequences to tune the HMM parameters so that the trained model is the best one to represent the demonstrations. The last stage is the measurement of the quality of each trajectory. With each trajectory sent through the trained HMM model, a generated likelihood index is obtained which reflects the consistency of the trajectory with the HMM. The trajectory with the maximum likelihood index is considered to be the best for the robot to follow.
---
paper_title: A multi-agent system for programming robots by human demonstration

paper_content:

This paper explores Gesture-Based Programming as a paradigm for programming robotic agents. Gesture-Based Programming is a form of programming by human demonstration that focuses the development of robotic systems on {\it task experts} rather than {\it programming experts}. The technique relies on the existence of previously acquired robotic skills (called ``sensorimotor primitives'') which are intended to be the robotic equivalent of that which humans acquire through everyday physical interactions. The interpretation of the human's demonstration and subsequent matching to robotic primitives is a qualitative problem that we approach with a community of skilled agents. A simple manipulative task is programmed to demonstrate the system.
---
paper_title: Learning Robot Manipulation Tasks With Task-Parameterized Semitied Hidden Semi-Markov Model

paper_content:

In this paper, we investigate the semi-tied Gaussian mixture models for robust learning and adaptation of robot manipulation tasks. We make use of the spatial and temporal correlation in the data by tying the covariance matrices of the mixture model with common synergistic directions/basis vectors, instead of estimating full covariance matrices for each cluster in the mixture. This allows the reuse of the discovered synergies in different parts of the task having similar coordination patterns. We extend the approach to task-parameterized and hidden semi-Markov models for autonomous adaptation to changing environmental situations. The planned movement sequence from the model is smoothly followed with a finite horizon linear quadratic tracking controller. Experiments to encode whole body motion data in simulation, followed by valve opening and pick-and-place via obstacle avoidance tasks with the Baxter robot, show improvement over standard Gaussian mixture models with much less parameters and better generalization ability.
---
paper_title: Learning by observation with mobile robots: a computational approach

paper_content:

We present a computational approach to learning by observation (LBO) that allows users to program mobile robots by demonstrating a task. Unlike previous approaches, our system incorporates statistical-learning techniques and concepts from control theory to reduce the amount of domain knowledge needed to infer the intent of the user. To improve the generalization ability of the system, the user can demonstrate the task multiple times. We extract task subgoals from these demonstrations and automatically associate them with objects in the environment. As these objects move, the subgoals are updated accordingly. This gives our system the ability to learn from demonstrations performed in different environments. In this paper, we present the concepts used in our LBO system as well as experimental laboratory results in learning motor-skill tasks.
---
paper_title: Acquiring Motion Elements for Bidirectional Computation of Motion Recognition and Generation

paper_content:

Mimesis theory is one of the primitive skill of imitative learning, which is regarded as an origin of human intelligence because imitation is fundamental function for communication and symbol manipulation. When the mimesis is adopted as learning method for humanoids, convenience for designing full body behavior would decrease because bottom-up learning approaches from robot side and top-down teaching approaches from user side involved each other. Therefore, we propose a behavior acquisition and understanding system for humanoids based on the mimesis theory. This system is able to abstract observed others’ behaviors into symbols, to recognize others’ behavior using the symbols, and to generate motion patterns using the symbols. In this paper, we mention the integration of mimesis loop, and confirmation of the feasibility on virtual humanoids.
---
paper_title: On learning the statistical representation of a task and generalizing it to various contexts

paper_content:

This paper presents an architecture for solving generically the problem of extracting the constraints of a given task in a programming by demonstration framework and the problem of generalizing the acquired knowledge to various contexts. We validate the architecture in a series of experiments, where a human demonstrator teaches a humanoid robot simple manipulatory tasks. First, the combined joint angles and hand path motions are projected into a generic latent space, composed of a mixture of Gaussians (GMM) spreading across the spatial dimensions of the motion. Second, the temporal variation of the latent representation of the motion is encoded in a hidden Markov model (HMM). This two-step probabilistic encoding provides a measure of the spatio-temporal correlations across the different modalities collected by the robot, which determines a metric of imitation performance. A generalization of the demonstrated trajectories is then performed using Gaussian mixture regression (GMR). Finally, to generalize skills across contexts, we compute formally the trajectory that optimizes the metric, given the new context and the robot's specific body constraints
---
paper_title: Incremental learning of full body motion primitives and their sequencing through human motion observation

paper_content:

In this paper we describe an approach for on-line, incremental learning of full body motion primitives from observation of human motion. The continuous observation sequence is first partitioned into motion segments, using stochastic segmentation. Next, motion segments are incrementally clustered and organized into a hierarchical tree structure representing the known motion primitives. Motion primitives are encoded using hidden Markov models, so that the same model can be used for both motion recognition and motion generation. At the same time, the temporal relationship between motion primitives is learned via the construction of a motion primitive graph. The motion primitive graph can then be used to construct motions, consisting of sequences of motion primitives. The approach is implemented and tested during on-line observation and on the IRT humanoid robot.
---
paper_title: Learning and generalization of motor skills by learning from demonstration

paper_content:

We provide a general approach for learning robotic motor skills from human demonstration. To represent an observed movement, a non-linear differential equation is learned such that it reproduces this movement. Based on this representation, we build a library of movements by labeling each recorded movement according to task and context (e.g., grasping, placing, and releasing). Our differential equation is formulated such that generalization can be achieved simply by adapting a start and a goal parameter in the equation to the desired position values of a movement. For object manipulation, we present how our framework extends to the control of gripper orientation and finger position. The feasibility of our approach is demonstrated in simulation as well as on the Sarcos dextrous robot arm. The robot learned a pick-and-place operation and a water-serving task and could generalize these tasks to novel situations.
---
paper_title: Robotic assembly solution by human-in-the-loop teaching method based on real-time stiffness modulation

paper_content:

We propose a novel human-in-the-loop approach for teaching robots how to solve assembly tasks in unpredictable and unstructured environments. In the proposed method the human sensorimotor system is integrated into the robot control loop though a teleoperation setup. The approach combines a 3-DoF end-effector force feedback with an interface for modulation of the robot end-effector stiffness. When operating in unpredictable and unstructured environments, modulation of limb impedance is essential in terms of successful task execution, stability and safety. We developed a novel hand-held stiffness control interface that is controlled by the motion of the human finger. A teaching approach was then used to achieve autonomous robot operation. In the experiments, we analysed and solved two part-assembly tasks: sliding a bolt fitting inside a groove and driving a self-tapping screw into a material of unknown properties. We experimentally compared the proposed method to complementary robot learning methods and analysed the potential benefits of direct stiffness modulation in the force-feedback teleoperation.
---
paper_title: Learning of assembly constraints by demonstration and active exploration

paper_content:

Purpose ::: ::: ::: ::: ::: In this paper, the authors aim to propose a method for learning robotic assembly sequences, where precedence constraints and object relative size and location constraints can be learned by demonstration and autonomous robot exploration. ::: ::: ::: ::: ::: Design/methodology/approach ::: ::: ::: ::: ::: To successfully plan the operations involved in assembly tasks, the planner needs to know the constraints of the desired task. In this paper, the authors propose a methodology for learning such constraints by demonstration and autonomous exploration. The learning of precedence constraints and object relative size and location constraints, which are needed to construct a planner for automated assembly, were investigated. In the developed system, the learning of symbolic constraints is integrated with low-level control algorithms, which is essential to enable active robot learning. ::: ::: ::: ::: ::: Findings ::: ::: ::: ::: ::: The authors demonstrated that the proposed reasoning algorithms can be used to learn previously unknown assembly constraints that are needed to implement a planner for automated assembly. Cranfield benchmark, which is a standardized benchmark for testing algorithms for robot assembly, was used to evaluate the proposed approaches. The authors evaluated the learning performance both in simulation and on a real robot. ::: ::: ::: ::: ::: Practical implications ::: ::: ::: ::: ::: The authors' approach reduces the amount of programming that is needed to set up new assembly cells and consequently the overall set up time when new products are introduced into the workcell. ::: ::: ::: ::: ::: Originality/value ::: ::: ::: ::: ::: In this paper, the authors propose a new approach for learning assembly constraints based on programming by demonstration and active robot exploration to reduce the computational complexity of the underlying search problems. The authors developed algorithms for success/failure detection of assembly operations based on the comparison of expected signals (forces and torques, positions and orientations of the assembly parts) with the actual signals sensed by a robot. In this manner, all precedence and object size and location constraints can be learned, thereby providing the necessary input for the optimal planning of the entire assembly process.
---
paper_title: Teaching robots the use of human tools from demonstration with non-dexterous end-effectors

paper_content:

Commercial, affordable and general-purpose robots like the PR-2, Baxter and UBR-1 robots can take over a wide range of tasks or assist human workers in a mixed human-robot environment. However, end-effectors on these robots are usually restricted to low-cost, non-dexterous grippers which constrains the application scenarios. We aim at increasing their range by teaching such robots the use of human tools by demonstration. We present a novel and compact model for the use of human tools and propose a dual-gripper strategy in replacement of the much less widely deployed dexterous hand for tool manipulation. Especially, we propose a hierarchical architecture to embed tool use in a learning from demonstration framework, learning temporal order for dual-arm coordination at higher level and Dynamic Movement Primitives at lower level for a multi-step execution. The approach is demonstrated and evaluated on a Baxter research robot for three human tools.
---
paper_title: From Demonstrations to Skills for High-level Programming of Industrial Robots

paper_content:

In this paper we describe our approach to robotic skill representation and a prototypical implementation of a programming-by-demonstration approach that allows users to generate skills and robot program primitives for later re- finement and re-use. We intend to evaluate the applicability of this approach to high-level programming in a user study, which we also explain.
---
paper_title: Generalization of orientation trajectories and force-torque profiles for robotic assembly

paper_content:

Abstract A typical robot assembly operation involves contacts with the parts of the product to be assembled and consequently requires the knowledge of not only position and orientation trajectories but also the accompanying force-torque profiles for successful performance. To learn the execution of assembly operations even when the geometry of the product varies across task executions, the robot needs to be able to adapt its motion based on a parametric description of the current task condition, which is usually provided by geometrical properties of the parts involved in the assembly. In our previous work we showed how positional control policies can be generalized to different task conditions. In this paper we propose a complete methodology to generalize also the orientational trajectories and the accompanying force-torque profiles to compute the necessary control policy for a given condition of the assembly task. Our method is based on statistical generalization of successfully recorded executions at different task conditions, which are acquired by kinesthetic guiding. The parameters that describe the varying task conditions define queries into the recorded training data. To improve the execution of the skill after generalization, we combine the proposed approach with an adaptation method, thus enabling the refinement of the generalized assembly operation.
---
paper_title: Transfer of assembly operations to new workpiece poses by adaptation to the desired force profile

paper_content:

In this paper we propose a new algorithm that can be used for adaptation of robot trajectories in automated assembly tasks. Initial trajectories and forces are obtained by demonstration and iteratively adapted to specific environment configurations. The algorithm adapts Cartesian space trajectories to match the forces recorded during the human demonstration. Experimentally we show the effectiveness of our approach on learning of Peg-in-Hole (PiH) task. We performed our experiments on two different robotic platforms with workpieces of different shapes.
---
paper_title: Active learning with statistical models

paper_content:

For many types of machine learning algorithms, one can compute the statistically "optimal" way to select training data. In this paper, we review how optimal data selection techniques have been used with feedforward neural networks. We then show how the same principles may be used to select data for two alternative, statistically-based learning architectures: mixtures of Gaussians and locally weighted regression. While the techniques for neural networks are computationally expensive and approximate, the techniques for mixtures of Gaussians and locally weighted regression are both efficient and accurate. Empirically, we observe that the optimality criterion sharply decreases the number of training examples the learner needs in order to achieve good performance.
---
paper_title: Contact-state modeling of robotic assembly tasks using Gaussian mixture models

paper_content:

Abstract This article addresses the Contact-State (CS) modeling problem for the force-controlled robotic peg-in-hole assembly tasks. The wrench (Cartesian forces and torques) and pose (Cartesian position and orientation) signals, of the manipulated object, are captured for different phases of the robotic assembly task. Those signals are utilized in building a CS model for each phase. Gaussian Mixture Models (GMM) is employed in building the likelihood of each signal and Expectation Maximization (EM) is used in finding the GMM parameters. Experiments are performed on a KUKA Lightweight Robot (LWR) doing camshaft caps assembly of an automotive powertrain. Comparisons are also performed with the available assembly modeling schemes, and the superiority of the EM-GMM scheme is shown with a reduced computational time.
---
paper_title: Learning Robot Manipulation Tasks With Task-Parameterized Semitied Hidden Semi-Markov Model

paper_content:

In this paper, we investigate the semi-tied Gaussian mixture models for robust learning and adaptation of robot manipulation tasks. We make use of the spatial and temporal correlation in the data by tying the covariance matrices of the mixture model with common synergistic directions/basis vectors, instead of estimating full covariance matrices for each cluster in the mixture. This allows the reuse of the discovered synergies in different parts of the task having similar coordination patterns. We extend the approach to task-parameterized and hidden semi-Markov models for autonomous adaptation to changing environmental situations. The planned movement sequence from the model is smoothly followed with a finite horizon linear quadratic tracking controller. Experiments to encode whole body motion data in simulation, followed by valve opening and pick-and-place via obstacle avoidance tasks with the Baxter robot, show improvement over standard Gaussian mixture models with much less parameters and better generalization ability.
---
paper_title: Full-Body Postural Control of a Humanoid Robot with Both Imitation Learning and Skill Innovation

paper_content:

In this paper, we present a novel methodology to obtain imitative and innovative postural movements in a humanoid based on human demonstrations in a different kinematic scale. We collected motion data from a group of human participants standing up from a chair. Modeling the human as an actuated 3-link kinematic chain, and by defining a multi-objective reward function of zero moment point and joint torques to represent the stability and effort, we computed reward profiles for each demonstration. Since individual reward profiles show variability across demonstrating trials, the underlying state transition probabilities were modeled using a Markov chain. Based on the argument that the reward profiles of the robot should show the same temporal structure of those of the human, we used differential evolution to compute a trajectory that fits all humanoid constraints and minimizes the difference between the robot reward profile and the predicted profile if the robot imitates the human. Therefore, robotic imitation involves developing a policy that results in a temporal reward structure, matching that of a group of human demonstrators across an array of demonstrations. Skill innovation was achieved by optimizing a signed reward error after imitation was achieved. Experimental results using the humanoid HOAP-3 are shown.
---
paper_title: Discovering optimal imitation strategies

paper_content:

This paper develops a general policy for learning the relevant features of an imitation task. We restrict our study to imitation of manipulative tasks or gestures. The imitation process is modeled as a hierarchical optimization system, which minimizes the discrepancy between two multi-dimensional datasets. To classify across manipulation strategies, we apply a probabilistic analysis to data in Cartesian and joint spaces. We determine a general metric that optimizes the policy of task reproduction, following strategy determination. The model successfully discovers strategies in six different imitative tasks and controls task reproduction by a full body humanoid robot. © 2004 Elsevier B.V. All rights reserved.
---
paper_title: Evaluation of robot imitation attempts: comparison of the system's and the human's perspectives

paper_content:

Imitation is a powerful learning tool when humans and robots interact in a social context. A series of experimental runs and a small pilot user study were conducted to evaluate the performance of a system designed for robot imitation. Performance assessments of similarity of imitative behaviours were carried out by machines and by humans: the system was evaluated quantitatively (from a machine-centric perspective) and qualitatively (from a human perspective) in order to study the reconciliation of these views. The experimental results presented here illustrate how the number of exceptions can be used as a performance measure by a robotic or software imitator of an object manipulation behaviour. (In this context, exceptions are events when the optimal displacement and/or rotation that minimize the dissimilarity metrics used to generate a corresponding imitative behaviour cannot be directly achieved in the particular context.) Results of the user study giving similarity judgments on imitative behaviours were used to examine how the quantitative measure of the number of exceptions (from a robot's perspective) corresponds to the qualitative evaluation of similarity (from a human's perspective) for the imitative behaviours generated by the jabberwocky system. Results suggest that there is a good alignment between this quantitive system centered assessment and the more qualitative human-centered assessment of imitative performance.
---
paper_title: A Developmental Roadmap for Learning by Imitation in Robots

paper_content:

In this paper, we present a strategy whereby a robot acquires the capability to learn by imitation following a developmental pathway consisting on three levels: 1) sensory-motor coordination; 2) world interaction; and 3) imitation. With these stages, the system is able to learn tasks by imitating human demonstrators. We describe results of the different developmental stages, involving perceptual and motor skills, implemented in our humanoid robot, Baltazar. At each stage, the system's attention is drawn toward different entities: its own body and, later on, objects and people. Our main contributions are the general architecture and the implementation of all the necessary modules until imitation capabilities are eventually acquired by the robot. Also, several other contributions are made at each level: learning of sensory-motor maps for redundant robots, a novel method for learning how to grasp objects, and a framework for learning task description from observation for program-level imitation. Finally, vision is used extensively as the sole sensing modality (sometimes in a simplified setting) avoiding the need for special data-acquisition hardware
---
paper_title: From visuo-motor self learning to early imitation-a neural architecture for humanoid learning

paper_content:

Behavior imitation ability will be a key technology for future human friendly robots. In order to understand the principles and mechanisms of imitation, we take a synthetic cognitive developmental approach, starting with minimum components and create a system that can learn to imitate others. We developed a visuo-motor neural learning system which consists of orientation selective visual movement representation, distributed arm movement representation, and a high-dimensional temporal sequence learning mechanism. The vision and the movement representations model the findings in primate brain, i.e. macaque area MT(or human area V5) and the primary motor area. The learning mechanism is inspired by the finding that there are excessive connections in neonate brain. As our robot explores the visuo-motor self movement patterns, it learns coherent patterns as high-dimensional trajectory attractors. After the learning, a human comes in front of the robot showing arm movements which are similar to the ones in self learning. Although the robot has never seen or programmed to interpret human arm movement, and the detail of visual stimuli are very different, the robot identifies some of the patterns as similar to those in self learning, and responded by generating the previously learned arm movement. In other words, the robot exhibits early imitation ability based on self exploratory learning.
---
paper_title: Discriminative and Adaptive Imitation in Uni-Manual and Bi-Manual Tasks

paper_content:

This paper addresses the problems of what to imitate and how to imitate in simple uni- and bi-manual manipulatory tasks. To solve the what to imitate issue, we use a probabilistic method, based on Hidden Markov Models, for extracting the relative importance of reproducing either the gesture or the specific hand path in a given task. This allows us to determine a metric of imitation performance. To solve the how to imitate issue, we compute the trajectory that optimizes the metric, given a set of robot's body constraints. We validate the methods in a series of experiments, where a human demonstrator teaches through kinesthetic a humanoid robot how to manipulate simple objects.
---
paper_title: Learning and generalization of motor skills by learning from demonstration

paper_content:

We provide a general approach for learning robotic motor skills from human demonstration. To represent an observed movement, a non-linear differential equation is learned such that it reproduces this movement. Based on this representation, we build a library of movements by labeling each recorded movement according to task and context (e.g., grasping, placing, and releasing). Our differential equation is formulated such that generalization can be achieved simply by adapting a start and a goal parameter in the equation to the desired position values of a movement. For object manipulation, we present how our framework extends to the control of gripper orientation and finger position. The feasibility of our approach is demonstrated in simulation as well as on the Sarcos dextrous robot arm. The robot learned a pick-and-place operation and a water-serving task and could generalize these tasks to novel situations.
---
paper_title: Incremental Learning, Clustering and Hierarchy Formation of Whole Body Motion Patterns using Adaptive Hidden Markov Chains

paper_content:

This paper describes a novel approach for autonomous and incremental learning of motion pattern primitives by observation of human motion. Human motion patterns are abstracted into a dynamic stochastic model, which can be used for both subsequent motion recognition and generation, analogous to the mirror neuron hypothesis in primates. The model size is adaptable based on the discrimination requirements in the associated region of the current knowledge base. A new algorithm for sequentially training the Markov chains is developed, to reduce the computation cost during model adaptation. As new motion patterns are observed, they are incrementally grouped together using hierarchical agglomerative clustering based on their relative distance in the model space. The clustering algorithm forms a tree structure, with specialized motions at the tree leaves, and generalized motions closer to the root. The generated tree structure will depend on the type of training data provided, so that the most specialized motions will be those for which the most training has been received. Tests with motion capture data for a variety of motion primitives demonstrate the efficacy of the algorithm.
---
paper_title: Robot programming from demonstration, feedback and transfer

paper_content:

This paper presents a novel approach for robot instruction for assembly tasks. We consider that robot programming can be made more efficient, precise and intuitive if we leverage the advantages of complementary approaches such as learning from demonstration, learning from feedback and knowledge transfer. Starting from low-level demonstrations of assembly tasks, the system is able to extract a high-level relational plan of the task. A graphical user interface (GUI) allows then the user to iteratively correct the acquired knowledge by refining high-level plans, and low-level geometrical knowledge of the task. This combination leads to a faster programming phase, more precise than just demonstrations, and more intuitive than just through a GUI. A final process allows to reuse high-level task knowledge for similar tasks in a transfer learning fashion. Finally we present a user study illustrating the advantages of this approach.
---
paper_title: Maximum likelihood from incomplete data via the EM algorithm

paper_content:

Vibratory power unit for vibrating conveyers and screens comprising an asynchronous polyphase motor, at least one pair of associated unbalanced masses disposed on the shaft of said motor, with the first mass of a pair of said unbalanced masses being rigidly fastened to said shaft and with said second mass of said pair being movably arranged relative to said first mass, means for controlling and regulating the conveying rate during conveyer operation by varying the rotational speed of said motor between predetermined minimum and maximum values, said second mass being movably outwardly by centrifugal force against the pressure of spring means, said spring means being prestressed in such a manner that said second mass is, at rotational motor speeds lower than said minimum speed, held in its initial position, and at motor speeds between said lower and upper values in positions which are radially offset with respect to the axis of said motor to an extent depending on the value of said rotational motor speed.
---

Title: Robot Learning from Demonstration in Robotic Assembly: A Survey

Section 1: Robotic Assembly
Description 1: Provide an overview of the current state and challenges in robotic assembly.

Section 2: Learning from Demonstration
Description 2: Discuss traditional robots vs. new-generation robots capable of learning from human demonstrations, and outline key challenges.

Section 3: Research Problems in Robotic Assembly
Description 3: Identify the major research problems in robotic assembly, particularly those related to LfD.

Section 4: Pose Estimation
Description 4: Explain the importance of pose estimation in assembly tasks and various methods to achieve accurate pose determination.

Section 5: Force Estimation
Description 5: Discuss the role of force estimation in robotic assembly and the techniques for acquiring and using force/torque data.

Section 6: Assembly Sequence
Description 6: Describe the significance of optimizing assembly sequences and methods for learning sequences from demonstrations.

Section 7: Assembly with Screwing
Description 7: Focus on the specific challenges and methodologies for robotic screwing tasks in assembly operations.

Section 8: Demonstration Approach
Description 8: Outline various methods of acquiring demonstration trajectories, such as kinesthetic, motion-sensor, and teleoperated demonstrations.

Section 9: Kinesthetic Demonstration
Description 9: Detail the process and benefits of kinesthetic guiding for teaching robots through physical interaction.

Section 10: Motion-Sensor Demonstration
Description 10: Describe the use of optical or magnetic marker-based tracking systems for capturing human-demonstrator motions.

Section 11: Teleoperated Demonstration
Description 11: Explain teleoperation as a method for demonstrating tasks and its application in various robotic tasks.

Section 12: Feature Extraction
Description 12: Describe the process of extracting and generalizing key features from demonstration trajectories into mathematical models.

Section 13: Hidden Markov Models
Description 13: Introduce Hidden Markov Models and their application in encoding and generalizing demonstration examples.

Section 14: Dynamic Movement Primitives
Description 14: Discuss Dynamic Movement Primitives as a method for representing dynamic performances in robot joint motors.

Section 15: Gaussian Mixture Models
Description 15: Explain Gaussian Mixture Models for clustering and density estimation in robotic motion modeling.

Section 16: Metric of Imitation Performance
Description 16: Present various metrics for evaluating the performance of robots learning from demonstration, including weighted and generic similarity measures.

Section 17: Conclusions and Discussion
Description 17: Summarize the findings of the survey, discuss the current state of LfD in robotic assembly, and identify areas for future research.