3D Human Motion Editing and Synthesis: A Survey
---
paper_title: The EMOTE model for effort and shape

paper_content:

Human movements include limb gestures and postural attitude. Although many computer animation researchers have studied these classes of movements, procedurally generated movements still lack naturalness. We argue that looking only at the psychological notion of gesture is insufficient to capture movement qualities needed by animated charactes. We advocate that the domain of movement observation science, specifically Laban Movement Analysis (LMA) and its Effort and Shape components, provides us with valuable parameters for the form and execution of qualitative aspects of movements. Inspired by some tenets shared among LMA proponents, we also point out that Effort and Shape phrasing across movements and the engagement of the whole body are essential aspects to be considered in the search for naturalness in procedurally generated gestures. Finally, we present EMOTE (Expressive MOTion Engine), a 3D character animation system that applies Effort and Shape qualities to independently defined underlying movements and thereby generates more natural synthetic gestures.
---
paper_title: Efficient synthesis of physically valid human motion

paper_content:

Optimization is a promising way to generate new animations from a minimal amount of input data. Physically based optimization techniques, however, are difficult to scale to complex animated characters, in part because evaluating and differentiating physical quantities becomes prohibitively slow. Traditional approaches often require optimizing or constraining parameters involving joint torques; obtaining first derivatives for these parameters is generally an O(D2) process, where D is the number of degrees of freedom of the character. In this paper, we describe a set of objective functions and constraints that lead to linear time analytical first derivatives. The surprising finding is that this set includes constraints on physical validity, such as ground contact constraints. Considering only constraints and objective functions that lead to linear time first derivatives results in fast per-iteration computation times and an optimization problem that appears to scale well to more complex characters. We show that qualities such as squash-and-stretch that are expected from physically based optimization result from our approach. Our animation system is particularly useful for synthesizing highly dynamic motions, and we show examples of swinging and leaping motions for characters having from 7 to 22 degrees of freedom.
---
paper_title: Learning silhouette features for control of human motion

paper_content:

We present a vision-based performance interface for controlling animated human characters. The system interactively combines information about the user's motion contained in silhouettes from three viewpoints with domain knowledge contained in a motion capture database to produce an animation of high quality. Such an interactive system might be useful for authoring, for teleconferencing, or as a control interface for a character in a game. In our implementation, the user performs in front of three video cameras; the resulting silhouettes are used to estimate his orientation and body configuration based on a set of discriminative local features. Those features are selected by a machine-learning algorithm during a preprocessing step. Sequences of motions that approximate the user's actions are extracted from the motion database and scaled in time to match the speed of the user's motion. We use swing dancing, a complex human motion, to demonstrate the effectiveness of our approach. We compare our results to those obtained with a set of global features, Hu moments, and ground truth measurements from a motion capture system.
---
paper_title: Improv: a system for scripting interactive actors in virtual worlds

paper_content:

Improv is a system for the creation of real−time behavior−based animated actors. There have been several recent efforts to build network distributed autonomous agents. But in general these efforts do not focus on the author’s view. To create rich interactive worlds inhabited by believable animated actors, authors need the proper tools. Improv provides tools to create actors that respond to users and to each other in real−time, with personalities and moods consistent with the author’s goals and intentions. Improv consists of two subsystems. The first subsystem is an Animation Engine that uses procedural techniques to enable authors to create layered, continuous, non−repetitive motions and smooth transitions between them. The second subsystem is a Behavior Engine that enables authors to create sophisticated rules governing how actors communicate, change, and make decisions. The combined system provides an integrated set of tools for authoring the "minds" and "bodies" of interactive actors. The system uses an english−style scripting language so that creative experts who are not primarily programmers can create powerful interactive applications.
---
paper_title: Learning physics-based motion style with nonlinear inverse optimization

paper_content:

This paper presents a novel physics-based representation of realistic character motion. The dynamical model incorporates several factors of locomotion derived from the biomechanical literature, including relative preferences for using some muscles more than others. elastic mechanisms at joints due to the mechanical properties of tendons, ligaments, and muscles, and variable stiffness at joints depending on the task. When used in a spacetime optimization framework, the parameters of this model define a wide range of styles of natural human movement.Due to the complexity of biological motion, these style parameters are too difficult to design by hand. To address this, we introduce Nonlinear Inverse Optimization, a novel algorithm for estimating optimization parameters from motion capture data. Our method can extract the physical parameters from a single short motion sequence. Once captured, this representation of style is extremely flexible: motions can be generated in the same style but performing different tasks, and styles may be edited to change the physical properties of the body.
---
paper_title: Incomplete motion feature tracking algorithm in video sequences

paper_content:

To effectively track incomplete motion features, a novel feature tracking algorithm for motion capture is presented. According to feature attributes and relationship among features, extracted features are classified as four types of features. Then different strategies are applied to track different kinds of features. To verify the tracks, cross correlation test and predicted 3D model based test are used to test and remove outliers. Experimental results demonstrate the effectiveness of our algorithm.
---
paper_title: The EMOTE model for effort and shape

paper_content:

Human movements include limb gestures and postural attitude. Although many computer animation researchers have studied these classes of movements, procedurally generated movements still lack naturalness. We argue that looking only at the psychological notion of gesture is insufficient to capture movement qualities needed by animated charactes. We advocate that the domain of movement observation science, specifically Laban Movement Analysis (LMA) and its Effort and Shape components, provides us with valuable parameters for the form and execution of qualitative aspects of movements. Inspired by some tenets shared among LMA proponents, we also point out that Effort and Shape phrasing across movements and the engagement of the whole body are essential aspects to be considered in the search for naturalness in procedurally generated gestures. Finally, we present EMOTE (Expressive MOTion Engine), a 3D character animation system that applies Effort and Shape qualities to independently defined underlying movements and thereby generates more natural synthetic gestures.
---
paper_title: Composable controllers for physics-based character animation

paper_content:

An ambitious goal in the area of physics-based computer animation is the creation of virtual actors that autonomously synthesize realistic human motions and possess a broad repertoire of lifelike motor skills. To this end, the control of dynamic, anthropomorphic figures subject to gravity and contact forces remains a difficult open problem. We propose a framework for composing controllers in order to enhance the motor abilities of such figures. A key contribution of our composition framework is an explicit model of the “pre-conditions” under which motor controllers are expected to function properly. We demonstrate controller composition with pre-conditions determined not only manually, but also automatically based on Support Vector Machine (SVM) learning theory. We evaluate our composition framework using a family of controllers capable of synthesizing basic actions such as balance, protective stepping when balance is disturbed, protective arm reactions when falling, and multiple ways of standing up after a fall. We furthermore demonstrate these basic controllers working in conjunction with more dynamic motor skills within a prototype virtual stunt-person. Our composition framework promises to enable the community of physics-based animation practitioners to easily exchange motor controllers and integrate them into dynamic characters.
---
paper_title: Video-based character animation

paper_content:

In this paper we introduce a video-based representation for free viewpoint visualization and motion control of 3D character models created from multiple view video sequences of real people. Previous approaches to video-based rendering provide no control of scene dynamics to manipulate, retarget, and create new 3D content from captured scenes. Here we contribute a new approach, combining image based reconstruction and video-based animation to allow controlled animation of people from captured multiple view video sequences. We represent a character as a motion graph of free viewpoint video motions for animation control. We introduce the use of geometry videos to represent reconstructed scenes of people for free viewpoint video rendering. We describe a novel spherical matching algorithm to derive global surface to surface correspondence in spherical geometry images for motion blending and the construction of seamless transitions between motion sequences. Finally, we demonstrate interactive video-based character animation with real-time rendering and free viewpoint visualization. This approach synthesizes highly realistic character animations with dynamic surface shape and appearance captured from multiple view video of people.
---
paper_title: Free-viewpoint video of human actors

paper_content:

In free-viewpoint video, the viewer can interactively choose his viewpoint in 3-D space to observe the action of a dynamic real-world scene from arbitrary perspectives. The human body and its motion plays a central role in most visual media and its structure can be exploited for robust motion estimation and efficient visualization. This paper describes a system that uses multi-view synchronized video footage of an actor's performance to estimate motion parameters and to interactively re-render the actor's appearance from any viewpoint.The actor's silhouettes are extracted from synchronized video frames via background segmentation and then used to determine a sequence of poses for a 3D human body model. By employing multi-view texturing during rendering, time-dependent changes in the body surface are reproduced in high detail. The motion capture subsystem runs offline, is non-intrusive, yields robust motion parameter estimates, and can cope with a broad range of motion. The rendering subsystem runs at real-time frame rates using ubiquous graphics hardware, yielding a highly naturalistic impression of the actor. The actor can be placed in virtual environments to create composite dynamic scenes. Free-viewpoint video allows the creation of camera fly-throughs or viewing the action interactively from arbitrary perspectives.
---
paper_title: Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces

paper_content:

Optimization is an appealing way to compute the motion of an animated character because it allows the user to specify the desired motion in a sparse, intuitive way. The difficulty of solving this problem for complex characters such as humans is due in part to the high dimensionality of the search space. The dimensionality is an artifact of the problem representation because most dynamic human behaviors are intrinsically low dimensional with, for example, legs and arms operating in a coordinated way. We describe a method that exploits this observation to create an optimization problem that is easier to solve. Our method utilizes an existing motion capture database to find a low-dimensional space that captures the properties of the desired behavior. We show that when the optimization problem is solved within this low-dimensional subspace, a sparse sketch can be used as an initial guess and full physics constraints can be enabled. We demonstrate the power of our approach with examples of forward, vertical, and turning jumps; with running and walking; and with several acrobatic flips.
---
paper_title: Automatic Joint Parameter Estimation from Magnetic Motion Capture Data

paper_content:

This paper describes a technique for using magnetic motion capture data to determine the joint parameters of an articulated hierarchy. This technique makes it possible to determine limb lengths, joint locations, and sensor placement for a human subject without external measurements. Instead, the joint parameters are inferred with high accuracy from the motion data acquired during the capture session. The parameters are computed by performing a linear least squares fit of a rotary joint model to the input data. A hierarchical structure for the articulated model can also be determined in situations where the topology of the model is not known. Once the system topology and joint parameters have been recovered, the resulting model can be used to perform forward and inverse kinematic procedures. We present the results of using the algorithm on human motion capture data, as well as validation results obtained with data from a simulation and a wooden linkage of known dimensions.
---
paper_title: The Process of Motion Capture : Dealing with the Data

paper_content:

This paper presents a detailed description of the process of motion capture, whereby sensor information from a performer is transformed into an articulated, hierarchical rigid-body object. We describe the gathering of the data, the real-time construction of a virtual skeleton which a director can use for immediate feedback, and the offline processing which produces the articulated object. This offline process involves a robust statistical estimation of the size of the skeleton and an inverse kinematic optimization to produce the desired joint angle trajectories. Additionally, we discuss a variation on the inverse kinematic optimization which can be used when the standard approach does not yield satisfactory results for the special cases when joint angle consistency is desired between a group of motions. These procedures work well and have been used to produce motions for a number of commercial games.
---
paper_title: Mapping optical motion capture data to skeletal motion using a physical model

paper_content:

Motion capture has become a premiere technique for animation of humanlike characters. To facilitate its use, researchers have focused on the manipulation of data for retargeting, editing, combining, and reusing motion capture libraries. In many of these efforts joint angle plus root trajectories are used as input, although this format requires an inherent mapping from the raw data recorded by many popular motion capture set-ups. In this paper, we propose a novel solution to this mapping problem from 3D marker position data recorded by optical motion capture systems to joint trajectories for a fixed limb-length skeleton using a forward dynamic model. To accomplish the mapping, we attach virtual springs to marker positions located on the appropriate landmarks of a physical simulation and apply resistive torques to the skeleton's joints using a simple controller. For the motion capture samples, joint-angle postures are resolved from the simulation's equilibrium state, based on the internal torques and external forces. Additional constraints, such as foot plants and hand holds, may also be treated as addition forces applied to the system and are a trivial and natural extension to the proposed technique. We present results for our approach as applied to several motion-captured behaviors.
---
paper_title: Motion graphs

paper_content:

In this paper we present a novel method for creating realistic, controllable motion. Given a corpus of motion capture data, we automatically construct a directed graph called a motion graph that encapsulates connections among the database. The motion graph consists both of pieces of original motion and automatically generated transitions. Motion can be generated simply by building walks on the graph. We present a general framework for extracting particular graph walks that meet a user's specifications. We then show how this framework can be applied to the specific problem of generating different styles of locomotion along arbitrary paths.
---
paper_title: Motion Synthesis By Example

paper_content:

A technique is proposed for creating new animations from a set of representative example motions stored in a motion database. Animations are created by cutting-and-pasting together the example motion segments as required. Motion segments are selected based upon how well they fit into a desired motion and are then automatically tailored for a precise fit. Various fundamental problems associated with the use of motion databases are outlined. A prototype implementation is used to validate the proposed concepts and to explore possible solutions to the aforementioned problems.
---
paper_title: Human Motion Synthesis with Optimization-based Graphs

paper_content:

Continuous constrained optimization is a powerful tool for synthesizing novel human motion segments that are short. Graph-based motion synthesis methods such as motion graphs and move trees are popular ways to synthesize long motions by playing back a sequence of existing motion segments. However, motion graphs only support transitions between similar frames, and move trees only support transitions between the end of one motion segment and the start of another. In this paper, we introduce an optimization-based graph that combines continuous constrained optimization with graph-based motion synthesis. The constrained optimization is used to create a vast number of complex realistic-looking transitions in the graph. The graph can then be used to synthesize long motions with non-trivial transitions that for example allow the character to switch its behavior abruptly while retaining motion naturalness. We also propose to build this graph semi-autonomously by requiring a user to classify generated transitions as acceptable or not and explicitly minimizing the amount of required classifications. This process guarantees the quality consistency of the optimization-based graph at the cost of limited user involvement.
---
paper_title: Motion texture: a two-level statistical model for character motion synthesis

paper_content:

In this paper, we describe a novel technique, called motion texture, for synthesizing complex human-figure motion (e.g., dancing) that is statistically similar to the original motion captured data. We define motion texture as a set of motion textons and their distribution, which characterize the stochastic and dynamic nature of the captured motion. Specifically, a motion texton is modeled by a linear dynamic system (LDS) while the texton distribution is represented by a transition matrix indicating how likely each texton is switched to another. We have designed a maximum likelihood algorithm to learn the motion textons and their relationship from the captured dance motion. The learnt motion texture can then be used to generate new animations automatically and/or edit animation sequences interactively. Most interestingly, motion texture can be manipulated at different levels, either by changing the fine details of a specific motion at the texton level or by designing a new choreography at the distribution level. Our approach is demonstrated by many synthesized sequences of visually compelling dance motion.
---
paper_title: Evaluating motion graphs for character animation

paper_content:

Realistic and directable humanlike characters are an ongoing goal in animation. Motion graph data structures hold much promise for achieving this goal; however, the quality of the results obtainable from a motion graph may not be easy to predict from its input motion clips. This article describes a method for using task-based metrics to evaluate the capability of a motion graph to create the set of animations required by a particular application. We examine this capability for typical motion graphs across a range of tasks and environments. We find that motion graph capability degrades rapidly with increases in the complexity of the target environment or required tasks, and that addressing deficiencies in a brute-force manner tends to lead to large, unwieldy motion graphs. The results of this method can be used to evaluate the extent to which a motion graph will fulfill the requirements of a particular application, lessening the risk of the data structure performing poorly at an inopportune moment. The method can also be used to characterize the deficiencies of motion graphs whose performance will not be sufficient, and to evaluate the relative effectiveness of different options for improving those motion graphs.
---
paper_title: Motion retargeting and evaluation for VR-based training of free motions

paper_content:

Virtual reality (VR) has emerged as one of the important and effective tools for education and training. Most VR-based training systems are situation based, where the trainees are trained for discrete decision making in special situations presented by the VR environments. In contrast, this paper discusses the application of VR to a different class of training, for learning free motion, often required in sports and the arts. We propose a VR-based motion-training framework that contains an intuitive motion-guiding interface, posture-oriented motion retargeting, and an evaluation and advice scheme for corrective feedback. Applications of the proposed framework to simple fencing training and a dance imitation game are demonstrated .
---
paper_title: Physically based motion transformation

paper_content:

We introduce a novel algorithm for transforming character animation sequences that preserves essential physical properties of the motion. By using the spacetime constraints dynamics formulation our algorithm maintains realism of the original motion sequence without sacrificing full user control of the editing process. In contrast to most physically based animation techniques that synthesize motion from scratch, we take the approach of motion transformationas the underlying paradigm for generating computer animations. In doing so, we combine the expressive richness of an input animation sequence with the controllability of spacetime optimization to create a wide range of realistic character animations. The spacetime dynamics formulation also allows editing of intuitive, high-level motion concepts such as the time and placement of footprints, length and mass of various extremities, number of body joints and gravity. Our algorithm is well suited for the reuse of highly-detailed captured motion animations. In addition, we describe a new methodology for mapping a motion between characters with drastically different numbers of degrees of freedom. We use this method to reduce the complexity of the spacetime optimization problems. Furthermore, our approach provides a paradigm for controlling complex dynamic and kinematic systems with simpler ones.
---
paper_title: Achieving good connectivity in motion graphs

paper_content:

Motion graphs have been widely successful in the synthesis of human motions. However, the quality of the generated motions depends heavily on the connectivity of the graphs and the quality of transitions in them. Achieving both of these criteria simultaneously though is difficult. Good connectivity requires transitions between less similar poses, while good motion quality requires transitions only between very similar poses. This paper introduces a new method for building motion graphs. The method first builds a set of interpolated motion clips, which contains many more similar poses than the original data set. The method then constructs a well-connected motion graph (wcMG), by using as little of the interpolated motion clip frames as necessary to provide good connectivity and only smooth transitions. Based on experiments, wcMGs outperform standard motion graphs across different measures, generate good quality motions, allow for high responsiveness in interactive control applications, and do not even require post-processing of the synthesized motions.
---
paper_title: A physically-based motion retargeting filter

paper_content:

This article presents a novel constraint-based motion editing technique. On the basis of animator-specified kinematic and dynamic constraints, the method converts a given captured or animated motion to a physically plausible motion. In contrast to previous methods using spacetime optimization, we cast the motion editing problem as a constrained state estimation problem, based on the per-frame Kalman filter framework. The method works as a filter that sequentially scans the input motion to produce a stream of output motion frames at a stable interactive rate. Animators can tune several filter parameters to adjust to different motions, turn the constraints on or off based on their contributions to the final result, or provide a rough sketch (kinematic hint) as an effective way of producing the desired motion. Experiments on various systems show that the technique processes the motions of a human with 54 degrees of freedom, at about 150 fps when only kinematic constraints are applied, and at about 10 fps when both kinematic and dynamic constraints are applied. Experiments on various types of motion show that the proposed method produces remarkably realistic animations.
---
paper_title: Motion retargeting in the presence of topological variations

paper_content:

Research on motion retargeting and synthesis for character animation has been mostly focused on character scale variations. In our recent work we have addressed the motion retargeting problem for characters with slightly different topologies. In this paper we present a new method for retargeting captured motion data to an enhanced character skeleton having a topology that is different from that of the original captured motion. The new topology could include altered hierarchical structures and scaled segments. In order to solve this problem, we propose a framework based on the concept of a motion control net (MCN). This is an external structure analogous to the convex hull of a set of control points defining a parametric curve or a surface patch. The MCN encapsulates the motion characteristics of the character. Retargeting is achieved as a generalized inverse kinematics problem using an external MCN. The retargeting solution requires the dynamic modification of the MCN structure. This also allows US to interactively edit the MCN and modify the conditions for the motion analysis. The new method can automatically synthesize new segment information and, by combining the segment motion into the MCN domain with a suitable displacement of control points embedded in the original motion capture sensor data, it can also generate realistic new motions that resemble the motion patterns in the original data. Copyright © 2006 John Wiley & Sons, Ltd.
---
paper_title: Fourier principles for emotion-based human figure animation

paper_content:

This paper describes the method for modeling human figure locomotions with emotions. Fourier expansions of experimental data of actual human behaviors serve as a basis from which the method can interpolate or extrapolate the human locomotions. This means, for instance, that transition from a walk to a run is smoothly and realistically performed by the method. Moreover an individual's character or mood, appearing during the human behaviors, is also extracted by the method. For example, the method gets "briskness" from the experimental data for a "normal" walk and a "brisk" walk. Then the "brisk" run is generated by the method, using another Fourier expansion of the measured data of running. The superposition of these human behaviors is shown as an efficient technique for generating rich variations of human locomotions. In addition, step-length, speed, and hip position during the locomotions are also modeled, and then interactively controlled to get a desired animation. Abstract
---
paper_title: Motion signal processing

paper_content:

Techniques from the image and signal processing domain can be successfully applied to designing, modifying, and adapting animated motion. For this purpose, we introduce multiresolution motion filtering, multitarget motion interpolation with dynamic timewarping, waveshaping and motion displacement mapping. The techniques are well-suited for reuse and adaptation of existing motion data such as joint angles, joint coordinates or higher level motion parameters of articulated figures with many degrees of freedom. Existing motions can be modified and combined interactively and at a higher level of abstraction than conventional systems support. This general approach is thus complementary to keyframing, motion capture, and procedural animation.
---
paper_title: Path editing technique based on motion graphs

paper_content:

This paper improved the algorithm of generating transitions and searching for path,and proposed a path editing method based on motion graphs.With regard to generating transitions,this paper detected the motion clips which can be used to blend automatically by minimizing the average frame distance between blending frames,and proposed Enhanced Dynamic Time Wrapping(EDTW) algorithm to solve this optimization problem.Concerning path search in the motion graph,this paper used the area between two curves as the target function and improved the strategy of incremental search and the strategy of branch and bound.The result shows that the proposed algorithm can edit and generate the character motions that well match the paths specified by users.
---
paper_title: Motion editing with spacetime constraints

paper_content:

In this paper, we present a method for editing a pre-existing motion such that it meets new needs yet preserves as much of the original quality as possible. Our approach enables the user to interactively position characters using direct manipulation. A spacetime constraints solver finds these positions while considering the entire motion. This paper discusses the three central challenges of creating such an approach: defining a constraint formulation that is rich enough to be effective, yet simple enough to afford fast solution; providing a solver that is fast enough to solve the constraint problems at interactive rates; and creating an interface that allows users to specify and visualize changes to entire motions. We present examples with a prototype system that permits interactive motion editing for articulated 3D characters on personal computers. I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism – Animation; I.3.6 [Computer Graphics]: Methodology and Techniques Interaction Techniques; G.1.6 [Numerical Analysis]: Optimization. Spacetime Constraints, Motion Displacement Mapping.
---
paper_title: Snap-together motion: assembling run-time animations

paper_content:

Many virtual environments and games must be populated with synthetic characters to create the desired experience. These characters must move with sufficient realism, so as not to destroy the visual quality of the experience, yet be responsive, controllable, and efficient to simulate. In this paper we present an approach to character motion called Snap-Together Motion that addresses the unique demands of virtual environments. Snap-Together Motion (STM) preprocesses a corpus of motion capture examples into a set of short clips that can be concatenated to make continuous streams of motion. The result process is a simple graph structure that facilitates efficient planning of character motions. A user-guided process selects "common" character poses and the system automatically synthesizes multi-way transitions that connect through these poses. In this manner well-connected graphs can be constructed to suit a particular application, allowing for practical interactive control without the effort of manually specifying all transitions.
---
paper_title: A hierarchical approach to interactive motion editing for human-like figures

paper_content:

This paper presents a technique for adapting existing motion of a human-like character to have the desired features that are specified by a set of constraints. This problem can be typically formulated as a spacetime constraint problem. Our approach combines a hierarchical curve fitting technique with a new inverse kinematics solver. Using the kinematics solver, we can adjust the configuration of an articulated figure to meet the constraints in each frame. Through the fitting technique, the motion displacement of every joint at each constrained frame is interpolated and thus smoothly propagated to frames. We are able to adaptively add motion details to satisfy the constraints within a specified tolerance by adopting a multilevel Bspline representation which also provides a speedup for the interpolation. The performance of our system is further enhanced by the new inverse kinematics solver. We present a closed-form solution to compute the joint angles of a limb linkage. This analytical method greatly reduces the burden of a numerical optimization to find the solutions for full degrees of freedom of a human-like articulated figure. We demonstrate that the technique can be used for retargetting a motion to compensate for geometric variations caused by both characters and environments. Furthermore, we can also use this technique for directly manipulating a motion clip through a graphical interface. CR Categories: I.3.7 [Computer Graphics]: Threedimensional Graphics—Animation; G.1.2 [Numerical Analysis]: Approximation—Spline and piecewise polynomial approximation
---
paper_title: Continuous motion graph for crowd simulation

paper_content:

This paper presents a new constrained motion synthesis algorithm for crowd. The algorithm is based on a novel continuous motion graph where infinite number of motion clips can be created by using path editing technique. Because we can create motions with arbitrary trajectories, we can speed up the motion synthesizing time significantly as well as satisfying constraints exactly. The resulting artifact of the motion such as foot skating is solved by the foot skating solver. The algorithm is tested on several different scenarios.
---
paper_title: Verbs and adverbs: multidimensional motion interpolation

paper_content:

The article describes a system for real-time interpolated animation that addresses some of these problems. Through creating parameterized motions-which the authors call verbs parameterized by adverbs-a single authored verb produces a continuous range of subtle variations of a given motion at real-time rates. As a result, simulated figures alter their actions based on their momentary mood or in response to changes in their goals or environmental stimuli. For example, they demonstrate a walk verb that can show emotions such as happiness and sadness, and demonstrate subtle variations due to walking up or down hill while turning to the left and right. They also describe verb graphs, which act as the glue to assemble verbs and their adverbs into a runtime data structure. Verb graphs provide the means for seamless transition from verb to verb for the simulated figures within an interactive runtime system. Finally they briefly discuss the discrete event simulator that handles the runtime main loop.
---
paper_title: Interactive motion generation from examples

paper_content:

There are many applications that demand large quantities of natural looking motion. It is difficult to synthesize motion that looks natural, particularly when it is people who must move. In this paper, we present a framework that generates human motions by cutting and pasting motion capture data. Selecting a collection of clips that yields an acceptable motion is a combinatorial problem that we manage as a randomized search of a hierarchy of graphs. This approach can generate motion sequences that satisfy a variety of constraints automatically. The motions are smooth and human-looking. They are generated in real time so that we can author complex motions interactively. The algorithm generates multiple motions that satisfy a given set of constraints, allowing a variety of choices for the animator. It can easily synthesize multiple motions that interact with each other using constraints. This framework allows the extensive re-use of motion capture data for new purposes.
---
paper_title: Motion path editing

paper_content:

In this paper we provide methods that allow for path-based editing of existing motion data. We begin by exploring the concept of path as an abstraction of motion, and show how many of the common motion editing tools fail to provide proper control over this useful property. We provide a simple extension to displacement mapping methods that provide better control over the path in a manner that is easy to implement in current systems. We then extend this simple method to avoid violation of geometric constraints such as foot- skate. demonstrate how path transformations work with constraint-based approaches to provide an interactive method for altering the path of a motion. This leads to several useful applications. A path is an abstraction of the positional movement of a character. The path encodes the direction of motion, which is different from, but related to, the orientation of the character as it moves along the path. This abstraction leads to the idea of representing a motion relative to the path, allowing the path to be altered and the motion to be adjusted accordingly. The methods we present maintain the relationship between the motion and the path. This paper is organized as follows. We begin by describing an ex- ample of what our techniques are capable of and useful for (Section 2). This discussion both motivates our methods as well as discusses their relationship to existing techniques. We then introduce the ab- straction of a path for a motion (Section 3) including methods for automatically creating it. The most basic form of path transformation, presented in Section 4, can create a new motion that follows an arbitrary path and ori- ents the character appropriately. However, this transformation may damage some of the fine details in the motion such as the crispness of footplants. Better results can be obtained by using constraint processing to explicitly maintain details, as described in Section 5. The motion is continuously updated as the user drags portions of the path. Even the most sophisticated of the methods presented works interactively in all of our examples. We conclude by discussing experiments with our prototype implementation.
---
paper_title: Motion graphs

paper_content:

In this paper we present a novel method for creating realistic, controllable motion. Given a corpus of motion capture data, we automatically construct a directed graph called a motion graph that encapsulates connections among the database. The motion graph consists both of pieces of original motion and automatically generated transitions. Motion can be generated simply by building walks on the graph. We present a general framework for extracting particular graph walks that meet a user's specifications. We then show how this framework can be applied to the specific problem of generating different styles of locomotion along arbitrary paths.
---
paper_title: Using an Intermediate Skeleton and Inverse Kinematics for Motion Retargeting

paper_content:

In this paper, we present a new method for solving the Motion Retargeting Problem, by using an intermediate skeleton. This allows us to convert movements between hierarchically and geometrically different characters. An Inverse Kinematics engine is then used to enforce Cartesian constraints while staying as close as possible to the captured motion.
---
paper_title: Improv: a system for scripting interactive actors in virtual worlds

paper_content:

Improv is a system for the creation of real−time behavior−based animated actors. There have been several recent efforts to build network distributed autonomous agents. But in general these efforts do not focus on the author’s view. To create rich interactive worlds inhabited by believable animated actors, authors need the proper tools. Improv provides tools to create actors that respond to users and to each other in real−time, with personalities and moods consistent with the author’s goals and intentions. Improv consists of two subsystems. The first subsystem is an Animation Engine that uses procedural techniques to enable authors to create layered, continuous, non−repetitive motions and smooth transitions between them. The second subsystem is a Behavior Engine that enables authors to create sophisticated rules governing how actors communicate, change, and make decisions. The combined system provides an integrated set of tools for authoring the "minds" and "bodies" of interactive actors. The system uses an english−style scripting language so that creative experts who are not primarily programmers can create powerful interactive applications.
---
paper_title: Style translation for human motion

paper_content:

Style translation is the process of transforming an input motion into a new style while preserving its original content. This problem is motivated by the needs of interactive applications, which require rapid processing of captured performances. Our solution learns to translate by analyzing differences between performances of the same content in input and output styles. It relies on a novel correspondence algorithm to align motions, and a linear time-invariant model to represent stylistic differences. Once the model is estimated with system identification, our system is capable of translating streaming input with simple linear operations at each frame.
---
paper_title: Automated derivation of behavior vocabularies for autonomous humanoid motion

paper_content:

In this paper we address the problem of automatically deriving vocabularies of motion modules from human motion data, taking advantage of the underlying spatio-temporal structure in motion. We approach this problem with a data-driven methodology for modularizing a motion stream (or time-series of human motion) into a vocabulary of parameterized primitive motion modules and a set of meta-level behaviors characterizing extended combinations of the primitives. Central to this methodology is the discovery of spatio-temporal structure in a motion stream. We estimate this structure by extending an existing nonlinear dimension reduction technique, Isomap, to handle motion data with spatial and temporal dependencies. The motion vocabularies derived by our methodology provide a substrate of autonomous behavior and can be used in a variety of applications. We demonstrate the utility of derived vocabularies for the application of synthesizing new humanoid motion that is structurally similar to the original demonstrated motion.
---
paper_title: Animating by multi-level sampling

paper_content:

We describe a method for synthesizing joint angle and translation data based on the information in motion capture data. The synthetic data is realistic not only in that it resembles the original training data, but in that it has random variations that are statistically similar to what one would find in repeated measurements of the motion. To achieve this result, the training data is broken into frequency bands using a wavelet decomposition, and the information in these bands is used to create the synthetic data one frequency band at a time. The method takes into account the fact that there are correlations among numerous features of the data. For example, a point characterized by a particular time and frequency band will depend upon points close to it in time in other frequency bands. Such correlations are modeled with a kernel-based representation of the joint probability distributions of the features. The data is synthesized by sampling from these densities and improving the results using a new iterative maximization technique. We have applied this technique to the synthesis of joint angle and translation data of a wallaby hopping on a treadmill. The synthetic data was used to animate characters that have limbs proportional to the wallaby.
---
paper_title: Learning Statistical Models of Human Motion

paper_content:

Non-linear statistical models of deformation provide methods to learn a priori shape and deformation for an object or class of objects by example. This paper extends these models of deformation to that of motion by augmenting the discrete representation of piecewise nonlinear principle component analysis of shape with a markov chain which represents the temporal dynamics of the model. In this manner, mean trajectories can be learnt and reproduced for either the simulation of movement or for object tracking. This paper demonstrates the use of these techniques in learning human motion from capture data.
---
paper_title: Verbs and adverbs: multidimensional motion interpolation

paper_content:

The article describes a system for real-time interpolated animation that addresses some of these problems. Through creating parameterized motions-which the authors call verbs parameterized by adverbs-a single authored verb produces a continuous range of subtle variations of a given motion at real-time rates. As a result, simulated figures alter their actions based on their momentary mood or in response to changes in their goals or environmental stimuli. For example, they demonstrate a walk verb that can show emotions such as happiness and sadness, and demonstrate subtle variations due to walking up or down hill while turning to the left and right. They also describe verb graphs, which act as the glue to assemble verbs and their adverbs into a runtime data structure. Verb graphs provide the means for seamless transition from verb to verb for the simulated figures within an interactive runtime system. Finally they briefly discuss the discrete event simulator that handles the runtime main loop.
---
paper_title: Physically valid statistical models for human motion generation

paper_content:

This article shows how statistical motion priors can be combined seamlessly with physical constraints for human motion modeling and generation. The key idea of the approach is to learn a nonlinear probabilistic force field function from prerecorded motion data with Gaussian processes and combine it with physical constraints in a probabilistic framework. In addition, we show how to effectively utilize the new model to generate a wide range of natural-looking motions that achieve the goals specified by users. Unlike previous statistical motion models, our model can generate physically realistic animations that react to external forces or changes in physical quantities of human bodies and interaction environments. We have evaluated the performance of our system by comparing against ground-truth motion data and alternative methods.
---
paper_title: Gaussian Process Dynamical Models for Human Motion

paper_content:

We introduce Gaussian process dynamical models (GPDMs) for nonlinear time series analysis, with applications to learning models of human pose and motion from high-dimensional motion capture data. A GPDM is a latent variable model. It comprises a low-dimensional latent space with associated dynamics, as well as a map from the latent space to an observation space. We marginalize out the model parameters in closed form by using Gaussian process priors for both the dynamical and the observation mappings. This results in a nonparametric model for dynamical systems that accounts for uncertainty in the model. We demonstrate the approach and compare four learning algorithms on human motion capture data, in which each pose is 50-dimensional. Despite the use of small data sets, the GPDM learns an effective representation of the nonlinear dynamics in these spaces.
---
paper_title: Optimized keyframe extraction for 3D character animations

paper_content:

In this paper, we propose a new method to automatically extract keyframes from animation sequences. Our method can be applied equally to both skeletal and mesh animations. It uses animation saliency computed on the original data to help select the group of keyframes that can reconstruct the input animation with less perception error. For computational efficiency, we perform nonlinear dimension reduction using locally linear embedding and then carry out the optimal search in much lower-dimensional space. With this approach, reconstruction of the animation from the extracted keyframes shows much better results as compared with earlier approaches. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Optimization-based key frame extraction for motion capture animation

paper_content:

In this paper, we present a new solution for extracting key frames from motion capture data using an optimization algorithm to obtain compact and sparse key frame data that can represent the original dense human body motion capture animation. The use of the genetic algorithm helps determine the optimal solution with global exploration capability while the use of a probabilistic simplex method helps expedite the speed of convergence. By finding the chromosome that maximizes the fitness function, the algorithm provides the optimal number of key frames as well as the low reconstruction error with an ordinary interpolation technique. The reconstruction error is computed between the original motion and the reconstruction one by the weighted differences of joint positions and velocities. The resulting set of key frames is obtained by iterative application of the algorithm with initial populations generated randomly and intelligently. We also present experiments which demonstrate that the method can effectively extract key frames with a high compression ratio and reconstruct all other non key frames with high quality.
---
paper_title: Motion capture assisted animation: texturing and synthesis

paper_content:

We discuss a method for creating animations that allows the animator to sketch an animation by setting a small number of keyframes on a fraction of the possible degrees of freedom. Motion capture data is then used to enhance the animation. Detail is added to degrees of freedom that were keyframed, a process we call texturing. Degrees of freedom that were not keyframed are synthesized. The method takes advantage of the fact that joint motions of an articulated figure are often correlated, so that given an incomplete data set, the missing degrees of freedom can be predicted from those that are present.
---
paper_title: 3D Human Motion Synthesis based on Nonlinear Manifold Learning

paper_content:

Due to the popularity of optical motion capture system,more realistic human motion data can be acquired easily and widely used in various applications such as video games,animation films,sports simulation and virtual reality. This paper proposes a framework and algorithm for 3D human motion synthesis based on nonlinear manifold learning. In this framework,high-dimensional motion samples are mapped into low-dimensional manifold,with nonlinear dimensionality reduction method,to the intrinsic representation of motion semantic features. Furthermore,the sample which is generated by user interactions in low-dimensional manifold can be reconstructed to obtain a 3D motion sequence which owns a new motion semantic feature by reverse mapping. The experimental results show that the method proposed in this paper can not only precisely control the physical features of motions(such as the location of a specific joint),but also can be used to synthesize new motion data which owns Abstract motion semantic,such as motion styles.
---
paper_title: The human motion database: A cognitive and parametric sampling of human motion

paper_content:

Motion databases have a strong potential to guide progress in the field of machine recognition and motion-based animation. Existing databases either have a very loose structure that does not sample the domain according to any controlled methodology or too few action samples which limits their potential to quantitatively evaluate the performance of motion-based techniques. The controlled sampling of the motor domain in the database may lead investigators to identify the fundamental difficulties of motion cognition problems and allow the addressing of these issues in a more objective way. In this paper, we describe the construction of our Human Motion Database using controlled sampling methods (parametric and cognitive sampling) to obtain the structure necessary for the quantitative evaluation of several motion-based research problems. The Human Motion Database is organized into several components: the praxicon dataset, the cross-validation dataset, the generalization dataset, the compositionality dataset, and the interaction dataset. The main contributions of this paper include (1) a survey of human motion databases describing data sources related to motion synthesis and analysis problems, (2) a sampling methodology that takes advantage of a systematic controlled capture, denoted as cognitive sampling and parametric sampling, and (3) a novel structured motion database organized into several datasets addressing a number of aspects in the motion domain.
---
paper_title: Perception of Human Motion With Different Geometric Models

paper_content:

Human figures have been animated using a variety of geometric models, including stick figures, polygonal models and NURBS-based models with muscles, flexible skin or clothing. This paper reports on experimental results indicating that a viewer's perception of motion characteristics is affected by the geometric model used for rendering. Subjects were shown a series of paired motion sequences and asked if the two motions in each pair were the same or different. The motion sequences in each pair were rendered using the same geometric model. For the three types of motion variation tested, sensitivity scores indicate that subjects were better able to observe changes with the polygonal model than they were with the stick-figure model.
---
paper_title: Hybrid control for interactive character animation

paper_content:

We implement a framework for animating interactive characters by combining kinematic animation with physical simulation. The combination of animation techniques allows the characters to exploit the advantages of each technique. For example, characters can perform natural-looking kinematic gaits and react dynamically to unexpected situations. Kinematic techniques such as those based on motion capture data can create very natural-looking animation. However, motion capture based techniques are not suitable for modeling the complex interactions between dynamically interacting characters. Physical simulation, on the other hand, is well suited for such tasks. Our work develops kinematic and dynamic controllers and transition methods between the two control methods for interactive character animation. In addition, we utilize the motion graph technique to develop complex kinematic animation from shorter motion clips as a method of kinematic control.
---
paper_title: Dynamic response for motion capture animation

paper_content:

Human motion capture embeds rich detail and style which is difficult to generate with competing animation synthesis technologies. However, such recorded data requires principled means for creating responses in unpredicted situations, for example reactions immediately following impact. This paper introduces a novel technique for incorporating unexpected impacts into a motion capture-driven animation system through the combination of a physical simulation which responds to contact forces and a specialized search routine which determines the best plausible re-entry into motion library playback following the impact. Using an actuated dynamic model, our system generates a physics-based response while connecting motion capture segments. Our method allows characters to respond to unexpected changes in the environment based on the specific dynamic effects of a given contact while also taking advantage of the realistic movement made available through motion capture. We show the results of our system under various conditions and with varying responses using martial arts motion capture as a testbed.
---
paper_title: An Efficient Keyframe Extraction from Motion Capture Data

paper_content:

This paper proposes a keyframe extraction method based on a novel layered curve simplification algorithm for motion capture data. Bone angles are employed as motion features and keyframe candidates can be selected based on them. After that, the layered curve simplification algorithm will be used to refine those candidates and the keyframe collection can be gained. To meet different requirements for compression and level of detail of motion abstraction, adaptive extraction parameters are also applied. The experiments demonstrate that our method can not only compress and summarize the motion capture data efficiently, but also keep the consistency of keyframe collection between similar human motion sequences, which is of great benefit to further motion data retrieval or editing.
---
paper_title: A Hierarchical Model Incorporating Segmented Regions and Pixel Descriptors for Video Background Subtraction

paper_content:

Background subtraction is important for detecting moving objects in videos. Currently, there are many approaches to performing background subtraction. However, they usually neglect the fact that the background images consist of different objects whose conditions may change frequently. In this paper, a novel hierarchical background model is proposed based on segmented background images. It first segments the background images into several regions by the mean-shift algorithm. Then, a hierarchical model, which consists of the region models and pixel models, is created. The region model is a kind of approximate Gaussian mixture model extracted from the histogram of a specific region. The pixel model is based on the cooccurrence of image variations described by histograms of oriented gradients of pixels in each region. Benefiting from the background segmentation, the region models and pixel models corresponding to different regions can be set to different parameters. The pixel descriptors are calculated only from neighboring pixels belonging to the same object. The experimental results are carried out with a video database to demonstrate the effectiveness, which is applied to both static and dynamic scenes by comparing it with some well-known background subtraction methods.
---
paper_title: Incomplete motion feature tracking algorithm in video sequences

paper_content:

To effectively track incomplete motion features, a novel feature tracking algorithm for motion capture is presented. According to feature attributes and relationship among features, extracted features are classified as four types of features. Then different strategies are applied to track different kinds of features. To verify the tracks, cross correlation test and predicted 3D model based test are used to test and remove outliers. Experimental results demonstrate the effectiveness of our algorithm.
---
paper_title: Group Animation Based on Multiple Autonomous Agents

paper_content:

Group animation is a continuous challenge in computer animation. In this paper, a group animation framework based on multiple autonomous agents. Each animated character in a group is modeled as an autonomous agent, which can perceive information from the virtual environment, generate appropriate intentions, manage rational behaviors, and create motions to realize the intention. Contrary to traditional intelligent animated character techniques, motions for the animated character are generated using a motion capture data, instead of modeling the complex motion generation mechanism. First a fundamental motion library is built using motion capture system, then appropriate fundamental motions are selected from the fundamental motion library, and desired motions are synthesized from these fundamental ones. The animated characters manage their movement autonomically in the virtual environment. To produce the required group animation, animators only need to specify some simple information and record the useful group motion. The experiment shows the effectiveness of this framework.
---
paper_title: Interactive generation of falling motions

paper_content:

Interactive generation of falling motions for virtual character with realistic responses to unexpected push, hit or collision with the environment is interesting work to many applications, such as computer games, film production, and virtual training environments. In this paper, we propose a new method to simulate protective behaviors in response to the ways a human may fall to the ground as well as incorporate the reactive motions into motion capture animation. It is based on simulated trajectory prediction and biomechanics inspired adjustment. According to the external perturbations, our system predicts a motion trajectory and uses it to select a desired transition-to sequence. At the same time, physically generated falling motions will fill in the gap between the two-motion capture sequences before and after the transition. Utilizing a parallel simulation, our method is able to predict a character's motion trajectory real-time under dynamics, which ensures that the character moves towards the target sequence and makes the character's behavior more life-like. Our controller is designed to generate physically plausible motion following an upcoming motion with adjustment from biomechanics rules, which is key to avoid an unconscious look for a character during the transition. Based on a relatively small motion database, our system is effective in generating various interactive falling behaviors. Copyright © 2006 John Wiley & Sons, Ltd.
---
paper_title: Style synthesis and editing of motion data in non-linear subspace

paper_content:

A new framework for automatic synthesis and editing of human motion style based on 3D human motion data and isometric feature mapping(ISOMAP) dimension reduction was proposed.In this framework,the generalized ISOMAP was extended to process out-of-samples data by building an optimal mapping function from the input high dimensional space to the embedding low dimensional nonlinear stylized space.The decomposable generative model was used to learn separate style parameters and content parameters of human motions.New motions with new style could be edited and reconstructed by adjusting and mapping these parameters from subspace to original motion data space.The experimental results show that the proposed method can generate new complex motion styles automatically and accurately in virtual reality scene.
---
paper_title: Real-time density-based crowd simulation

paper_content:

Virtual characters in games and simulations often need to plan visually convincing paths through a crowded environment. This paper describes how crowd density information can be used to guide a large number of characters through a crowded environment. Crowd density information helps characters avoid congested routes that could lead to traffic jams. It also encourages characters to use a wide variety of routes to reach their destination. Our technique measures the desirability of a route by combining distance information with crowd density information. We start by building a navigation mesh for the walkable regions in a polygonal two-dimensional (2-D) or multilayered three-dimensional (3-D) environment. The skeleton of this navigation mesh is the medial axis. Each walkable region in the navigation mesh maintains an up-to-date density value. This density value is equal to the area occupied by all the characters inside a given region divided by the total area of this region. These density values are mapped onto the medial axis to form a weighted graph. An A* search on this graph yields a backbone path for each character, and forces are used to guide the characters through the weighted environment. The characters periodically replan their routes as the density values are updated. Our experiments show that we can compute congestion-avoiding paths for tens of thousands of characters in real-time. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Obscuring length changes during animated motion

paper_content:

In this paper we examine to what extent the lengths of the links in an animated articulated-figure can be changed without the viewer being aware of the change. This is investigated in terms of a framework that emphasizes the role of attention in visual perception. We conducted a set of five experiments to establish bounds for the sensitivity to changes in length as a function of several parameters and the amount of attention available. We found that while length changes of 3% can be perceived when the relevant links are given full attention, changes of over 20% can go unnoticed when attention is not focused in this way. These results provide general guidelines for algorithms that produce or process character motion data and also bring to light some of the potential gains that stand to be achieved with attention-based algorithms.
---
paper_title: Controllable real-time locomotion using mobility maps

paper_content:

Graph-based approaches for sequencing motion capture data have produced some of the most realistic and controllable character motion to date. Most previous graph-based approaches have employed a run-time global search to find paths through the motion graph that meet user-defined constraints such as a desired locomotion path. Such searches do not scale well to large numbers of characters. In this paper, we describe a locomotion approach that benefits from the realism of graph-based approaches while maintaining basic user control and scaling well to large numbers of characters. Our approach is based on precomputing multiple least cost sequences from every state in a state-action graph. We store these precomputed sequences in a data structure called a mobility map and perform a local search of this map at run-time to generate motion sequences in real time that achieve user constraints in a natural manner. We demonstrate the quality of the motion through various example locomotion tasks including target tracking and collision avoidance. We demonstrate scalability by animating crowds of up to 150 rendered articulated walking characters at real-time rates.
---
paper_title: An efficient search algorithm for motion data using weighted PCA

paper_content:

Good motion data is costly to create. Such an expense often makes the reuse of motion data through transformation and retargetting a more attractive option than creating new motion from scratch. Reuse requires the ability to search automatically and efficiently a growing corpus of motion data, which remains a difficult open problem. We present a method for quickly searching long, unsegmented motion clips for subregions that most closely match a short query clip. Our search algorithm is based on a weighted PCA-based pose representation that allows for flexible and efficient pose-to-pose distance calculations. We present our pose representation and the details of the search algorithm. We evaluate the performance of a prototype search application using both synthetic and captured motion data. Using these results, we propose ways to improve the application's performance. The results inform a discussion of the algorithm's good scalability characteristics.
---
paper_title: Compression of motion capture databases

paper_content:

We present a lossy compression algorithm for large databases of motion capture data. We approximate short clips of motion using Bezier curves and clustered principal component analysis. This approximation has a smoothing effect on the motion. Contacts with the environment (such as foot strikes) have important detail that needs to be maintained. We compress these environmental contacts using a separate, JPEG like compression algorithm and ensure these contacts are maintained during decompression.Our method can compress 6 hours 34 minutes of human motion capture from 1080 MB data into 35.5 MB with little visible degradation. Compression and decompression is fast: our research implementation can decompress at about 1.2 milliseconds/frame, 7 times faster than real-time (for 120 frames per second animation). Our method also yields smaller compressed representation for the same error or produces smaller error for the same compressed size.
---
paper_title: Human Motion Database with a Binary Tree and Node Transition Graphs

paper_content:

Database of human motion has been widely used for recognizing human motion and synthesizing humanoid motions. In this paper, we propose a data structure for storing and extracting human motion data and demonstrate that the database can be applied to the recognition and motion synthesis problems in robotics. We develop an efficient method for building a human motion database from a collection of continuous, multi-dimensional motion clips. The database consists of a binary tree representing the hierarchical clustering of the states observed in the motion clips, as well as node transition graphs representing the possible transitions among the nodes in the binary tree. Using databases constructed from real human motion data, we demonstrate that the proposed data structure can be used for human motion recognition, state estimation and prediction, and robot motion planning.
---
paper_title: Efficient content-based retrieval of motion capture data

paper_content:

The reuse of human motion capture data to create new, realistic motions by applying morphing and blending techniques has become an important issue in computer animation. This requires the identification and extraction of logically related motions scattered within some data set. Such content-based retrieval of motion capture data, which is the topic of this paper, constitutes a difficult and time-consuming problem due to significant spatio-temporal variations between logically related motions. In our approach, we introduce various kinds of qualitative features describing geometric relations between specified body points of a pose and show how these features induce a time segmentation of motion capture data streams. By incorporating spatio-temporal invariance into the geometric features and adaptive segments, we are able to adopt efficient indexing methods allowing for flexible and efficient content-based retrieval and browsing in huge motion capture databases. Furthermore, we obtain an efficient preprocessing method substantially accelerating the cost-intensive classical dynamic time warping techniques for the time alignment of logically similar motion data streams. We present experimental results on a test data set of more than one million frames, corresponding to 180 minutes of motion. The linearity of our indexing algorithms guarantees the scalability of our results to much larger data sets.
---
paper_title: Human Motion Retrieval from Hand-Drawn Sketch

paper_content:

The rapid growth of motion capture data increases the importance of motion retrieval. The majority of the existing motion retrieval approaches are based on a labor-intensive step in which the user browses and selects a desired query motion clip from the large motion clip database. In this work, a novel sketching interface for defining the query is presented. This simple approach allows users to define the required motion by sketching several motion strokes over a drawn character, which requires less effort and extends the users' expressiveness. To support the real-time interface, a specialized encoding of the motions and the hand-drawn query is required. Here, we introduce a novel hierarchical encoding scheme based on a set of orthonormal spherical harmonic (SH) basis functions, which provides a compact representation, and avoids the CPU/processing intensive stage of temporal alignment used by previous solutions. Experimental results show that the proposed approach can well retrieve the motions, and is capable of retrieve logically and numerically similar motions, which is superior to previous approaches. The user study shows that the proposed system can be a useful tool to input motion query if the users are familiar with it. Finally, an application of generating a 3D animation from a hand-drawn comics strip is demonstrated.
---
paper_title: Perceptual metrics for character animation: sensitivity to errors in ballistic motion

paper_content:

Motion capture data and techniques for blending, editing, and sequencing that data can produce rich, realistic character animation; however, the output of these motion processing techniques sometimes appears unnatural. For example, the motion may violate physical laws or reflect unreasonable forces from the character or the environment. While problems such as these can be fixed, doing so is not yet feasible in real time environments. We are interested in developing ways to estimate perceived error in animated human motion so that the output quality of motion processing techniques can be better controlled to meet user goals.This paper presents results of a study of user sensitivity to errors in animated human motion. Errors were systematically added to human jumping motion, and the ability of subjects to detect these errors was measured. We found that users were able to detect motion with errors, and noted some interesting trends: errors in horizontal velocity were easier to detect than errors in vertical velocity, and added accelerations were easier to detect than added decelerations. On the basis of our results, we propose a perceptually based metric for measuring errors in ballistic human motion.
---
paper_title: Automated extraction and parameterization of motions in large data sets

paper_content:

Large motion data sets often contain many variants of the same kind of motion, but without appropriate tools it is difficult to fully exploit this fact. This paper provides automated methods for identifying logically similar motions in a data set and using them to build a continuous and intuitively parameterized space of motions. To find logically similar motions that are numerically dissimilar, our search method employs a novel distance metric to find "close" motions and then uses them as intermediaries to find more distant motions. Search queries are answered at interactive speeds through a precomputation that compactly represents all possibly similar motion segments. Once a set of related motions has been extracted, we automatically register them and apply blending techniques to create a continuous space of motions. Given a function that defines relevant motion parameters, we present a method for extracting motions from this space that accurately possess new parameters requested by the user. Our algorithm extends previous work by explicitly constraining blend weights to reasonable values and having a run-time cost that is nearly independent of the number of example motions. We present experimental results on a test data set of 37,000 frames, or about ten minutes of motion sampled at 60 Hz.
---
paper_title: A puppet interface for retrieval of motion capture data

paper_content:

Intuitive and efficient retrieval of motion capture data is essential for effective use of motion capture databases. In this paper, we describe a system that allows the user to retrieve a particular sequence by performing an approximation of the motion with an instrumented puppet. This interface is intuitive because both adults and children have experience playacting with puppets and toys to express particular behaviors or to tell stories with style and emotion. The puppet has 17 degrees of freedom and can therefore represent a variety of motions. We develop a novel similarity metric between puppet and human motion by computing the reconstruction errors of the puppet motion in the latent space of the human motion and those of the human motion in the latent space of the puppet motion. This metric works even for relatively large databases. We conducted a user study of the system and subjects could find the desired motion with reasonable accuracy from a database consisting of everyday, exercise, and acrobatic behaviors.
---
paper_title: 3D motion retrieval with motion index tree

paper_content:

With the development of Motion capture techniques, more and more 3D motion libraries become available. In this paper, we present a novel content-based 3D motion retrieval algorithm. We partition the motion library and construct a motion index tree based on a hierarchical motion description. The motion index tree serves as a classifier to determine the sub-library that contains the promising similar motions to the query sample. The Nearest Neighbor rule-based dynamic clustering algorithm is adopted to partition the library and construct the motion index tree. The similarity between the sample and the motion in the sub-library is calculated through elastic match. To improve the efficiency of the similarity calculation, an adaptive clustering-based key-frame extraction algorithm is adopted. The experiment demonstrates the effectiveness of this algorithm.
---
paper_title: A data-driven approach to quantifying natural human motion

paper_content:

In this paper, we investigate whether it is possible to develop a measure that quantifies the naturalness of human motion (as defined by a large database). Such a measure might prove useful in verifying that a motion editing operation had not destroyed the naturalness of a motion capture clip or that a synthetic motion transition was within the space of those seen in natural human motion. We explore the performance of mixture of Gaussians (MoG), hidden Markov models (HMM), and switching linear dynamic systems (SLDS) on this problem. We use each of these statistical models alone and as part of an ensemble of smaller statistical models. We also implement a Naive Bayes (NB) model for a baseline comparison. We test these techniques on motion capture data held out from a database, keyframed motions, edited motions, motions with noise added, and synthetic motion transitions. We present the results as receiver operating characteristic (ROC) curves and compare the results to the judgments made by subjects in a user study.
---
paper_title: A system for analyzing and indexing human-motion databases

paper_content:

We demonstrate a data-driven approach for representing, compressing, and indexing human-motion databases. Our modeling approach is based on piecewise-linear components that are determined via a divisive clustering method. Selection of the appropriate linear model is determined automatically via a classifier using a subspace of the most significant, or principle features (markers). We show that, after offline training, our model can accurately estimate and classify human motions. We can also construct indexing structures for motion sequences according to their transition trajectories through these linear components. Our method not only provides indices for whole and/or partial motion sequences, but also serves as a compressed representation for the entire motion database. Our method also tends to be immune to temporal variations, and thus avoids the expense of time-warping.
---
paper_title: Group motion graphs

paper_content:

We introduce Group Motion Graphs, a data-driven animation technique for groups of discrete agents, such as flocks, herds, or small crowds. Group Motion Graphs are conceptually similar to motion graphs constructed from motion-capture data, but have some important differences: we assume simulated motion; transition nodes are found by clustering group configurations from the input simulations: and clips to join transitions are explicitly constructed via constrained simulation. Graphs built this way offer known bounds on the trajectories that they generate, making it easier to search for particular output motions. The resulting animations show realistic motion at significantly reduced computational cost compared to simulation, and improved control.
---
paper_title: Capturing and animating skin deformation in human motion

paper_content:

During dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture system and approximately 350 markers. Although the number of markers is significantly larger than that used in conventional motion capture, it is only a sparse representation of the true shape of the body. We supplement this sparse sample with a detailed, actor-specific surface model. The motion of the skin can then be computed by segmenting the markers into the motion of a set of rigid parts and a residual deformation (approximated first as a quadratic transformation and then with radial basis functions). We demonstrate the power of this approach by capturing flexing muscles, high frequency motions, and abrupt decelerations on several actors. We compare these results both to conventional motion capture and skinning and to synchronized video of the actors.
---
paper_title: Discover Novel Visual Categories From Dynamic Hierarchies Using Multimodal Attributes

paper_content:

Learning novel visual categories from observations and experiences in unexplored environment is a vitally important cognitive ability for human beings. A dynamic category hierarchy that is an inherent structure in a human mind is a key component for this ability. This paper develops a framework to build dynamic category hierarchy based on object attributes and a topic model. Since humans trend to utilize multimodal information to learn novel categories, we also develop an algorithm to learn multimodal object attributes from multimodal data. The new multimodal attributes can describe objects efficiently and can generalize from learned categories to novel ones. By comparison with a state-of-the-art unimodal attribute, the multimodal attributes can achieve 4%-19% improvements on average. We also develop a constrained topic model, which can accurately construct category hierarchies for large-scale categories. Based on them, the novel framework can effectively detect novel categories and relate them with known categories for further category learning. Extensive experiments are conducted using a public multimodal dataset, i.e., color and point cloud data, to evaluate the multimodal attributes and the dynamic category hierarchy. The experimental results show the effectiveness of multimodal attributes to describe objects and the satisfactory performance of the dynamic category hierarchy to discover novel categories. By comparison with state-of-the-art methods, the dynamic category hierarchy achieves 7% improvements.
---
paper_title: A robust method for analyzing the physical correctness of motion capture data

paper_content:

The physical correctness of motion capture data is important for human motion analysis and athlete training. However, until now there is little work that wholly explores this problem of analyzing the physical correctness of motion capture data. In this paper, we carefully discuss this problem and solve two major issues in it. Firstly, a new form of Newton-Euler equations encoded by quaternions and Euler angles which are very fit for analyzing the motion capture data are proposed. Secondly, a robust optimization method is proposed to correct the motion capture data to satisfy the physical constraints. We demonstrate the advantage of our method with several experiments.
---

Title: 3D Human Motion Editing and Synthesis: A Survey

Section 1: Introduction
Description 1: Discusses the challenges and difficulties in achieving realistic 3D human motion data and introduces the four categories of motion synthesis methods, with a focus on motion capture data-driven methods.

Section 2: Classification of 3D Human Motion Synthesis
Description 2: Elaborates on the four main categories of motion synthesis methods: manual methods, physics-based methods, video-based methods, and motion capture data-driven methods, with detailed analysis of each.

Section 3: Motion Capture Data Representation
Description 3: Describes the various formats and hierarchical structures used to store and represent motion capture data, as well as the processing stages required to obtain structured information.

Section 4: Motion Capture Data-Driven Methods
Description 4: Details the motion capture data-driven methods, including the techniques for motion editing and synthesis to generate realistic motion, and discusses their advantages and limitations.

Section 5: Motion Editing Methods 
Description 5: Discusses the different motion editing methods that focus on modifying attributes of motion data to meet specific animation requirements while maintaining other attributes unchanged.

Section 6: Motion Synthesis Methods
Description 6: Explains the concept of motion synthesis, focusing on techniques to synthesize continuous, long-time motions that conform to constraints and user requirements.

Section 7: Motion Synthesis Methods Based on Motion Graph
Description 7: Describes motion graph construction, detection of similarity, generation of transitions, graph construction methods, and goal-achieved graph search, including several notable implementations.

Section 8: The Statistical Motion Synthesis
Description 8: Discusses statistical motion synthesis techniques, covering various statistical models and their application to human motion synthesis with a focus on different approaches and their effectiveness.

Section 9: Other Motion Synthesis Methods Based on Motion Capture Data
Description 9: Explores additional motion synthesis methods that leverage motion capture data, aside from motion graphs and statistical models, for synthesizing detailed and constraint-satisfied motion.

Section 10: Discussion
Description 10: Provides an overview of the developments in 3D human motion synthesis technology, identifies ongoing problems and new research directions, and discusses solutions for database organization, compression, search, evaluation, group motion synthesis, and reactive human motion synthesis.