Techniques of Facial Synthesis: A Comprehensive Literature Review
---
paper_title: Automatic construction of 3D human face models based on 2D images

paper_content:

In order to achieve low bit-rate video coding, model-based coding systems have attracted great interests in visual telecommunications, e.g., videophone and teleconferencing where the human faces are the major part in the scenes. The main idea of this approach is to construct a 3D model for human face. Only the moving parts on the face are analyzed and the motion parameters are transmitted, finally the original facial expressions could be synthesized by deforming the face model using the facial motion parameters. We propose an approach to the automatic construction of 3D human face models using a generic face model and several 2D face images. A template matching based algorithm is developed to automatically extract all necessary facial features from the front and side profile face images. Then the generic face model is fitted to these feature points by geometric transforms. Finally, texture mapping is performed to achieve realistic results.
---
paper_title: Face Synthesis (FASY) System for Generation of a Face Image from Human Description

paper_content:

* AICTE Emeritus Fellow Abstract-This paper aims at generating a new face based on the human like description using a new concept. The FASY (FAce SYnthesis) System is a Face Database Retrieval and new Face generation System that is under development. One of its main features is the generation of the requested face when it is not found in the existing database, which allows a continuous growing of the database also. Index Terms- Adjusting Intensities, Assembling the Components, Face Generation, Databases I. INTRODUCTION Face is most important visual identity of a person and while meeting an unknown person, it is the face that attracts our attention most. We often describe a person in terms of the characteristic features of important face components like eyes, eyebrows, nose, lips together with the overall shape of the face. The criminal Investigation agencies employ artists to sketch human faces from the description given by a witness about criminal's appearance which is then searched in the face database of the known criminals for recognition. Since early 1990's, Face Recognition Technology (FRT) becomes an active research area and there are a lot of works on facial recognition and facial feature extraction (1)(2)(3)(4)(5)(6). In 1994, Jian Kang Wu and Arcot Desai Narasimhalu (7) worked on "Identifying Faces Using Multiple Retrievals" where they retrieved the existing faces based on some properties of the different face components. But if the desired face is not in the database, then it is necessary to construct a new face from the user's description and this work is a new one which is one of the objective of the present work. This paper is a part of a main research effort, whose aim is the construction of the FASY-System based on human like description of each components of the face. The FASY System has the following features: (i) face queries using human-like description of the face, (ii) searching the required face in the database and (iii) generation of the requested face when they are not in the database. This paper is focused in this last feature. All the face components are stored in the database and are retrieved according to the user given queries and combined together to form the whole face which is then shown to the user. The Face Generation system works in 3 steps: (i) Searching phase to search the database for each single components of the face according to the user's query. (ii) Assembling phase to place all the components at proper places to generate the full face (iii) Tuning phase to adjust the intensities to cover up the stitching lines between different pair of face components so that the generated face looks natural. A variety of face databases have already been described in literature and here we have worked with the databases found from AT & T Laboratories Cambridge (8). The specifications for the database described in this paper are as follows: • In all blank facial images, the ears should be visible. • Sufficient variety of facial components and their properties should be stored in the database. • The database should contain subject of different ethnicities, sample of both men and women, and sample from different age groups. • Variation due to head tilt, shift, rotation and scaling should be minimized as much as possible.
---
paper_title: Facial analysis and synthesis using image-based models

paper_content:

In this paper we describe image-based modeling techniques that make possible the creation of photo-realistic computer models of real human faces. The image-based model is built using example views of the face, bypassing the need for any three-dimensional computer graphics models. A learning network is trained to associate each of the example images with a set of pose and expression parameters. For a novel set of parameters, the network synthesizes a novel, intermediate view using a morphing approach. This image-based synthesis paradigm can adequately model both rigid and non-rigid facial movements. We also describe an analysis-by-synthesis algorithm, which is capable of extracting a set of high-level parameters from an image sequence involving facial movement using embedded image-based models. The parameters of the models are perturbed in a local and independent manner for each image until a correspondence-based error metric is minimized. A small sample of experimental results is presented.
---
paper_title: Realistic modeling for facial animation

paper_content:

A major unsolved problem in computer graphics is the construction and animation of realistic human facial models. Traditionally, facial models have been built painstakingly by manual digitization and animated by ad hoc parametrically controlled facial mesh deformations or kinematic approximation of muscle actions. Fortunately, animators are now able to digitize facial geometries through the use of scanning range sensors and animate them through the dynamic simulation of facial tissues and muscles. However, these techniques require considerable user input to construct facial models of individuals suitable for animation. In this paper, we present a methodology for automating this challenging task. Starting with a structured facial mesh, we develop algorithms that automatically construct functional models of the heads of human subjects from laser-scanned range and reflectance data. These algorithms automatically insert contractile muscles at anatomically correct positions within a dynamic skin model and root them in an estimated skull structure with a hinged jaw. They also synthesize functional eyes, eyelids, teeth, and a neck and fit them to the final model. The constructed face may be animated via muscle actuations. In this way, we create the most authentic and functional facial models of individuals available to date and demonstrate their use in facial animation.
---
paper_title: Automatic Single View-Based 3-D Face Synthesis for Unsupervised Multimedia Applications

paper_content:

Various 3-D face synthesis techniques have been proposed and extensively used in many applications. Compared with others, single view-based face synthesis technology allows unsupervised 3-D face reconstruction without any offline operations. Although many algorithms have been published, automatic and robust single view-based 3-D face synthesis still remains unsolved. In contrast to other methods, the single view-based 3-D face synthesis algorithm conducted in this paper enables automated 3-D face synthesis from an arbitrary head-and-shoulder image with the complex background. The developed system first detects the face using Bayesian skin-tone classification based on only the chrominance component, Cr. Based on the detected face, a few salient facial features, such as the corners of the eyebrows and contours of the eyes, mouth, and chin are in turn extracted using variant algorithms, including a dynamic chin extraction mechanism that will be detailed in this paper. Then, face model adaptation consisting of both global and local adaptations is imposed, according to geometric information provided by the extracted facial features. Finally, the 3-D specific face is synthesized using the adapted 3-D face model with a texture map directly derived from the input face image, followed by the implementation of facial animation using this synthesized face.
---
paper_title: Bosphorus database for 3D face analysis

paper_content:

A new 3D face database that includes a rich set of expressions, systematic variation of poses and different types of occlusions is presented in this paper. This database is unique from three aspects: i) the facial expressions are composed of judiciously selected subset of Action Units as well as the six basic emotions, and many actors/actresses are incorporated to obtain more realistic expression data; ii) a rich set of head pose variations are available; and iii) different types of face occlusions are included. Hence, this new database can be a very valuable resource for development and evaluation of algorithms on face recognition under adverse conditions and facial expression analysis as well as for facial expression synthesis.
---
paper_title: Face Image Synthesis and Interpretation Using 3D Illumination-Based AAM Models

paper_content:

One of the more exciting and unsolved problems in computer vision nowadays is automatic, fast and full interpretation of face images under variable conditions of lighting and pose. Interpretation is the inference of knowledge from an image. This knowledge covers relevant information, such as 3D shape and albedo, both related to the identity, but also information about physical factors which affect appearance of faces, such as pose and lighting. Interpretation of faces not only should be limited to retrieve the aforementioned pieces of information, but also, it should be capable of synthesizing novel facial images in which some of these pieces of information have been modified. This kind of interpretation can be achieved by using the paradigm known as analysis by synthesis, see Figure 1. Ideally, an approach based on analysis by synthesis, should consist of a generative facial parametric model that codes all the sources of appearance variation separately and independently, and an optimization algorithm which systematically varies the model parameters until the synthetic image produced by the model is as similar as possible to the test image, also called input image. A full interpretation approach should include the recovery of 3D shape, 3D pose, albedo and lighting from a single face image which exhibits any possible combination of these sources of variation. Active appearance models, or simply AAMs (Cootes et al. (2001); Edwards et al. (1998); Matthews & Baker (2004)), with respect to other approaches, represent a fast alternative to perform face interpretation using the analysis by synthesis paradigm. Texture and shape, are attributes modeled by AAMs by using statistic tools such as principal components analysis or shortly PCA. However, the apparent texture of a face is an implicit combination of lighting and albedo. The separation process of these two attributes is not an easy task within the context of sparse models, like AAMs. AAMs use a sparse set of vertices which outline the shape. Texture is interpolated over that shape. In fact, a detailed dense set of surface normals, which is not available in AAMs, is required to perform the separation of lighting and albedo. On the other hand, texture and shape variation among human faces is relatively small when uniform lighting is considered. AAMs take advantage of this fact by supposing a constant relationship between changes of appearance and the variation of the model parameters producing those changes. This approximately constant relationship is a constant gradient which is used for performing fast fitting to input images. However, for most purposes, lighting is not uniform, and a proper separation of albedo and lighting becomes necessary. In a similar way as is texture variation in uniform lighting, albedo variation among human 4
---
paper_title: Active acquisition of 3D shape for moving objects

paper_content:

An active 3D acquisition system is presented that projects a simple pattern of squares on a scene and views it from a different angle. This paper describes how the observed pattern can be extracted from the image data. The underlying algorithm automatically detects the lines and crossings of the projected pattern in the image. Experiments show that the algorithm is robust and provides accurate three-dimensional (3D) reconstructions. Its one-shot operation principle enables the system to retrieve the shape of moving objects.
---
paper_title: Towards an Example-Based Image Compression Architecture for Video-Conferencing

paper_content:

This paper consists of two major parts. First, we present the outline of a simple approach to very-low bandwidth video-conferencing system relying on an example-based hierarchical image compression scheme. In particular, we discuss the use of example images as a model, the number of required examples, faces as a class of semi-rigid objects, a hierarchical model based on decomposition into different time-scales, and the decomposition of face images into patches of interest. In the second part, we present several algorithms for image processing and animation as well as experimental evaluations. Among the original contributions of this paper is an automatic algorithm for pose estimation and normalization. We also review and compare different algorithms for finding the nearest neighbors in a database for a new input as well as a generalized algorithm for blending patches of interest in order to synthesize new images. Finally, we outline the possible integration of several algorithms to illustrate a simple model-based video-conference system.
---
paper_title: Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models

paper_content:

An approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions is presented. The approach exploits a sophisticated generative model of the human face originally developed for realistic facial animation. The face model which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physics-based synthetic facial tissue and a set of anatomically motivated facial muscle actuators. The estimation of dynamical facial muscle contractions from video sequences of expressive human faces is considered. An estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images is developed. The technique estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions. >
---
paper_title: Model-Based and Image-Based Methods for Facial Image Synthesis , Analysis and Recognition

paper_content:

We review several model-based and image-based methods that we have developed for analyzing, synthesizing, and recognizing facial images. Our model-based methods include a sophisticated, functional model of the human face/head, which incorporates a biomechanical tissue model with embedded muscle actuators, and techniques for applying it to computer animation and expression estimation in video. Our image-based methods include tensor-faces, a nonlinear (multilinear) representation for facial image ensembles that disentangles pose, illumination, and expression effects to improve facial recognition.
---
paper_title: Low-dimensional procedure for the characterization of human faces

paper_content:

A method is presented for the representation of (pictures of) faces. Within a specified framework the representation is ideal. This results in the characterization of a face, to within an error bound, by a relatively low-dimensional vector. The method is illustrated in detail by the use of an ensemble of pictures taken for this purpose.
---
paper_title: Fake fur rendering

paper_content:

A probabilistic lighting model is presented for thin coats of fur over skin. Previous methods for rendering furry objects and creatures have addressed the case where individual strands or tufts of hair may be resolvable at the pixel level. These methods are often computationally intensive. However, a large class of realworld cases where individual hairs are much smaller than the size of a pixel can be addressed using a probabilistic model for the expected value of reflected light within a small surface area. Under the assumption that hair parameters are slowly varying across the skin, lighting calculations are performed on a reference hair with prefiltered parameters. The reflected light from individual hairs and from the skin below is blended using the expectation of a ray striking a hair in that area as the opacity of the fur coating. Approximations for hair-to-hair shadowing and hair-to-skin shadowing can be made using the same hit-expectation model. Our system can be implemented in existing commercial surfacerendering software at a much lower computational cost than typical resolvable-hair methods. CR
---
paper_title: Accurate face models from uncalibrated and ill-lit video sequences

paper_content:

We propose a face reconstruction technique that produces models that not only look good when texture mapped, but are also metrically accurate. Our method is designed to work with short uncalibrated video or movie sequences, even when the lighting is poor resulting in specularities and shadows that complicate the algorithm's task. Our approach relies on optimizing the shape parameters of a sophisticated PCA based model given pairwise image correspondences as input. All that is required is enough relative motion between camera and subject so that we can derive structure from motion. By matching the results against laser scanning data, we will show that its precision is excellent and can be predicted as a junction of the number and quality of the correspondences. This is important if one wishes to obtain the appropriate compromise between processing speed and quality of the results. Furthermore, our method is in fact not specific to faces and could equally be applied to any shape for which a shape model controlled with relatively small number of parameters exists.
---
paper_title: Active acquisition of 3D shape for moving objects

paper_content:

An active 3D acquisition system is presented that projects a simple pattern of squares on a scene and views it from a different angle. This paper describes how the observed pattern can be extracted from the image data. The underlying algorithm automatically detects the lines and crossings of the projected pattern in the image. Experiments show that the algorithm is robust and provides accurate three-dimensional (3D) reconstructions. Its one-shot operation principle enables the system to retrieve the shape of moving objects.
---
paper_title: A statistical model for synthesis of detailed facial geometry

paper_content:

Detailed surface geometry contributes greatly to the visual realism of 3D face models. However, acquiring high-resolution face geometry is often tedious and expensive. Consequently, most face models used in games, virtual reality, or computer vision look unrealistically smooth. In this paper, we introduce a new statistical technique for the analysis and synthesis of small three-dimensional facial features, such as wrinkles and pores. We acquire high-resolution face geometry for people across a wide range of ages, genders, and races. For each scan, we separate the skin surface details from a smooth base mesh using displaced subdivision surfaces. Then, we analyze the resulting displacement maps using the texture analysis/synthesis framework of Heeger and Bergen, adapted to capture statistics that vary spatially across a face. Finally, we use the extracted statistics to synthesize plausible detail on face meshes of arbitrary subjects. We demonstrate the effectiveness of this method in several applications, including analysis of facial texture in subjects with different ages and genders, interpolation between high-resolution face scans, adding detail to low-resolution face scans, and adjusting the apparent age of faces. In all cases, we are able to re-produce fine geometric details consistent with those observed in high resolution scans.
---
paper_title: Image-Based BRDF Measurement Including Human Skin

paper_content:

We present a new image-based process for measuring the bidirectional reflectance of homogeneous surfaces rapidly, completely, and accurately. For simple sample shapes (spheres and cylinders) the method requires only a digital camera and a stable light source. Adding a 3D scanner allows a wide class of curved near-convex objects to be measured. With measurements for a variety of materials from paints to human skin, we demonstrate the new method's ability to achieve high resolution and accuracy over a large domain of illumination and reflection directions. We verify our measurements by tests of internal consistency and by comparison against measurements made using a gonioreflectometer.
---
paper_title: Modeling and Rendering for Realistic Facial Animation

paper_content:

Rendering realistic faces and facial expressions requires good models for the reflectance of skin and the motion of the face. We describe a system for modeling, animating, and rendering a face using measured data for geometry, motion, and reflectance, which realistically reproduces the appearance of a particular person’s face and facial expressions. Because we build a complete model that includes geometry and bidirectional reflectance, the face can be rendered under any illumination and viewing conditions. Our face modeling system creates structured face models with correspondences across different faces, which provide a foundation for a variety of facial animation operations.
---
paper_title: Face Re-Lighting from a Single Image under Harsh Lighting Conditions

paper_content:

In this paper, we present a new method to change the illumination condition of a face image, with unknown face geometry and albedo information. This problem is particularly difficult when there is only one single image of the subject available and it was taken under a harsh lighting condition. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using spherical harmonic representation. However, the approximation error can be large under harsh lighting conditions thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion based framework that uses a Markov Random Field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to harsh lighting conditions, but insensitive to partial occlusions as well. The performance of our framework is demonstrated through various experimental results, including the improvement to the face recognition rate under harsh lighting conditions.
---
paper_title: A Measurement-Based Skin Reflectance Model for Face Rendering and Editing

paper_content:

We present a novel skin reflectance model for faces and its application to face appearance editing. We decompose the high-dimensional bidirectional scattering surface reflectance distribution function (BSSRDF) of skin into components that can be estimated from measured data. Our model is intuitive, amenable to interactive rendering, and easy to edit. High-quality renderings come close to reproducing real photographs. We have measured 3D face geometry, skin reflectance, and subsurface scattering for a large group of people using custom-built devices and fit the data to our model. The analysis of the reflectance data reveals variations according to subject age, race, gender and external factors (heat, cold, makeup, etc.) We derive a low-dimensional model using non-negative matrix factorization (NMF) that spans the space of skin reflectance in our database. A user can define meaningful parameters in this space - such as race, gender, and age - and change the overall appearance of a person (e.g., making a Caucasian face look more Asian) or change local features (e.g., adding moles, freckles or hair follicles). SIGGRAPH 2005
---
paper_title: Reconstructing High Quality Face-Surfaces using Model Based Stereo

paper_content:

We present a novel model based stereo system, which accurately extracts the 3D shape and pose of faces from multiple images taken simultaneously. Extracting the 3D shape from images is important in areas such as pose-invariant face recognition and image manipulation. The method is based on a 3D morphable face model learned from a database of facial scans. The use of a strong face prior allows us to extract high precision surfaces from stereo data of faces, where traditional correlation based stereo methods fail because of the mostly textureless input images. The method uses two or more uncalibrated images of arbitrary baseline, estimating calibration and shape simultaneously. Results using two and three input images are presented. We replace the lighting and albedo estimation of a monocular method with the use of stereo information, making the system more accurate and robust. We evaluate the method using ground truth data and the standard PIE image dataset. A comparison with the state of the art monocular system shows that the new method has a significantly higher accuracy.
---
paper_title: Analysis of human faces using a measurement-based skin reflectance model

paper_content:

We have measured 3D face geometry, skin reflectance, and subsurface scattering using custom-built devices for 149 subjects of varying age, gender, and race. We developed a novel skin reflectance model whose parameters can be estimated from measurements. The model decomposes the large amount of measured skin data into a spatially-varying analytic BRDF, a diffuse albedo map, and diffuse subsurface scattering. Our model is intuitive, physically plausible, and -- since we do not use the original measured data -- easy to edit as well. High-quality renderings come close to reproducing real photographs. The analysis of the model parameters for our sample population reveals variations according to subject age, gender, skin type, and external factors (e.g., sweat, cold, or makeup). Using our statistics, a user can edit the overall appearance of a face (e.g., changing skin type and age) or change small-scale features using texture synthesis (e.g., adding moles and freckles). We are making the collected statistics publicly available to the research community for applications in face synthesis and analysis.
---
paper_title: Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions

paper_content:

In this paper, we present a new method to modify the appearance of a face image by manipulating the illumination condition, when the face geometry and albedo information is unknown. This problem is particularly difficult when there is only a single image of the subject available. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using a spherical harmonic representation. Moreover, morphable models are statistical ensembles of facial properties such as shape and texture. In this paper, we integrate spherical harmonics into the morphable model framework by proposing a 3D spherical harmonic basis morphable model (SHBMM). The proposed method can represent a face under arbitrary unknown lighting and pose simply by three low-dimensional vectors, i.e., shape parameters, spherical harmonic basis parameters, and illumination coefficients, which are called the SHBMM parameters. However, when the image was taken under an extreme lighting condition, the approximation error can be large, thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion-based framework that uses a Markov random field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to extreme lighting conditions, but also insensitive to partial occlusions. The performance of our framework is demonstrated through various experimental results, including the improved rates for face recognition under extreme lighting conditions.
---
paper_title: Resynthesizing facial animation through 3D model-based tracking

paper_content:

Given video footage of a person's face, we present new techniques to automatically recover the face position and the facial expression from each frame in the video sequence. A 3D face model is fitted to each frame using a continuous optimization technique. Our model is based on a set of 3D face models that are linearly combined using 3D morphing. Our method has the advantages over previous techniques of fitting directly a realistic 3-dimensional face model and of recovering parameters that can be used directly in an animation system. We also explore many applications, including performance-driven animation (applying the recovered position and expression of the face to a synthetic character to produce an animation that mimics the input video), relighting the face, varying the camera position, and adding facial ornaments such as tattoos and scars.
---
paper_title: Synthesizing realistic facial expressions from photographs

paper_content:

We present new techniques for creating photorealistic textured 3D facial models from photographs of a human subject, and for creating smooth transitions between different facial expressions by morphing between these different models. Starting from several uncalibrated views of a human subject, we employ a user-assisted technique to recover the camera poses corresponding to the views as well as the 3D coordinates of a sparse set of chosen locations on the subject's face. A scattered data interpolation technique is then used to deform a generic face mesh to fit the particular geometry of the subject's face. Having recovered the camera poses and the facial geometry, we extract from the input images one or more texture maps for the model. This process is repeated for several facial expressions of a particular subject. To generate transitions between these facial expressions we use 3D shape morphing between the corresponding face models, while at the same time blending the corresponding textures. Using our technique, we have been able to generate highly realistic face models and natural looking animations.
---
paper_title: Trainable videorealistic speech animation

paper_content:

We describe how to create with machine learning techniques a generative, videorealistic, and speech animation module. A human subject is first recorded using a videocamera as he/she utters a pre-determined speech corpus. After processing the corpus automatically, a visual speech module is learned from the data that is capable of synthesizing the human subject's mouth uttering entirely novel utterances that were not recorded in the original video. The synthesized utterance is re-composited onto a background sequence, which contains natural head and eye movement. The final output is videorealistic in the sense that it looks like a video camera recording of the subject. At run time, the input to the system can be either real audio sequences or synthetic audio produced by a text-to-speech system, as long as they have been phonetically aligned.
---
paper_title: Video Rewrite: driving visual speech with audio

paper_content:

Video Rewrite uses existing footage to create automatically new video of a person mouthing words that she did not speak in the original footage. This technique is useful in movie dubbing, for example, where the movie sequence can be modified to sync the actors’ lip motions to the new soundtrack. Video Rewrite automatically labels the phonemes in the training data and in the new audio track. Video Rewrite reorders the mouth images in the training footage to match the phoneme sequence of the new audio track. When particular phonemes are unavailable in the training footage, Video Rewrite selects the closest approximations. The resulting sequence of mouth images is stitched into the background footage. This stitching process automatically corrects for differences in head position and orientation between the mouth images and the background footage. Video Rewrite uses computer-vision techniques to track points on the speaker’s mouth in the training footage, and morphing techniques to combine these mouth gestures into the final video sequence. The new video combines the dynamics of the original actor’s articulations with the mannerisms and setting dictated by the background footage. Video Rewrite is the first facial-animation system to automate all the labeling and assembly tasks required to resync existing footage to a new soundtrack.
---
paper_title: Geometry-driven photorealistic facial expression synthesis

paper_content:

Expression mapping (also called performance driven animation) has been a popular method for generating facial animations. A shortcoming of this method is that it does not generate expression details such as the wrinkles due to skin deformations. In this paper, we provide a solution to this problem. We have developed a geometry-driven facial expression synthesis system. Given feature point positions (the geometry) of a facial expression, our system automatically synthesizes a corresponding expression image that includes photorealistic and natural looking expression details. Due to the difficulty of point tracking, the number of feature points required by the synthesis system is, in general, more than what is directly available from a performance sequence. We have developed a technique to infer the missing feature point motions from the tracked subset by using an example-based approach. Another application of our system is expression editing where the user drags feature points while the system interactively generates facial expressions with skin deformation details.
---
paper_title: A virtual environment and model of the eye for surgical simulation

paper_content:

An anatomically detailed 3-D computer graphic model of the eye and surrounding face within a virtual environment has been implemented for use in a surgical simulator. The simulator forms part of a teleoperated micro-surgical robotic system being developed for eye surgery. The model has been designed to both visually and mechanically simulate features of the human eye by coupling computer graphic realism with finite element analysis. ::: The paper gives an overview of the system with emphasis on the graphical modelling techniques and a computationally efficient framework for representing anatomical details of the eye and for finite element analysis of the mechanical properties. Examples of realistic images coupled to large deformation finite element model of the cornea are presented. These images can be rendered sufficiently fast for the virtual reality application.
---
paper_title: Fake fur rendering

paper_content:

A probabilistic lighting model is presented for thin coats of fur over skin. Previous methods for rendering furry objects and creatures have addressed the case where individual strands or tufts of hair may be resolvable at the pixel level. These methods are often computationally intensive. However, a large class of realworld cases where individual hairs are much smaller than the size of a pixel can be addressed using a probabilistic model for the expected value of reflected light within a small surface area. Under the assumption that hair parameters are slowly varying across the skin, lighting calculations are performed on a reference hair with prefiltered parameters. The reflected light from individual hairs and from the skin below is blended using the expectation of a ray striking a hair in that area as the opacity of the fur coating. Approximations for hair-to-hair shadowing and hair-to-skin shadowing can be made using the same hit-expectation model. Our system can be implemented in existing commercial surfacerendering software at a much lower computational cost than typical resolvable-hair methods. CR
---
paper_title: A practical model for hair mutual interactions

paper_content:

Hair exhibits strong anisotropic dynamic properties which demand distinct dynamic models for single strands and hair-hair interactions. While a single strand can be modeled as a multibody open chain expressed in generalized coordinates, modeling hair-hair interactions is a more difficult problem. A dynamic model for this purpose is proposed based on a sparse set of guide strands. Long range connections among the strands are modeled as breakable static links formulated as nonreversible positional springs. Dynamic hair-to-hair collision is solved with the help of auxiliary triangle strips among nearby strands. Adaptive guide strands can be generated and removed on the fly to dynamically control the accuracy of a simulation. A high-quality dense hair model can be obtained at the end by transforming and interpolating the sparse guide strands. Fine imagery of the final dense model is rendered by considering both primary scattering and self-shadowing inside the hair volume which is modeled as being partially translucent.
---
paper_title: Super-helices for predicting the dynamics of natural hair

paper_content:

Simulating human hair is recognized as one of the most difficult tasks in computer animation. In this paper, we show that the Kirchhoff equations for dynamic, inextensible elastic rods can be used for accurately predicting hair motion. These equations fully account for the nonlinear behavior of hair strands with respect to bending and twisting. We introduce a novel deformable model for solving them: each strand is represented by a Super-Helix, i.e., a piecewise helical rod which is animated using the principles of Lagrangian mechanics. This results in a realistic and stable simulation, allowing large time steps. Our second contribution is an in-depth validation of the Super-Helix model, carried out through a series of experiments based on the comparison of real and simulated hair motions. We show that our model efficiently handles a wide range of hair types with a high level of realism.
---
paper_title: Rendering fur with three dimensional textures

paper_content:

We present a method for rendering scenes with fine detail via an object called a texel, a rendering primitive inspired by volume densities mixed with anisotropic lighting models. This technique solves a long outstanding problem in image synthesis: the rendering of furry surfaces.
---
paper_title: Image-Based Hair Capture by Inverse Lighting

paper_content:

We introduce an image-based method for modeling a specific subject's hair. The principle of the approach is to study the variations of hair illumination under controlled illumination. The use of a stationary viewpoint and the assumption that the subject is still allows us to work with perfectly registered images: all pixels in an image sequence represent the same portion of the hair, and the particular illumination profile observed at each pixel can be used to infer the missing degree of directional information. This is accomplished by synthesizing reflection profiles using a hair reflectance model, for a number of candidate directions at each pixel, and choosing the orientation that provides the best profile match. Our results demonstrate the potential of this approach, by effectively reconstructing accurate hair strands that are well highlighted by a particular light source movement.
---
paper_title: Efficient multiple scattering in hair using spherical harmonics

paper_content:

Previous research has shown that a global multiple scattering simulation is needed to achieve physically realistic renderings of hair, particularly light-colored hair with low absorption. However, previous methods have either sacrificed accuracy or have been too computationally expensive for practical use. In this paper we describe a physically based, volumetric rendering method that computes multiple scattering solutions, including directional effects, much faster than previous accurate methods. Our two-pass method first traces light paths through a volumetric representation of the hair, contributing power to a 3D grid of spherical harmonic coefficients that store the directional distribution of scattered radiance everywhere in the hair volume. Then, in a ray tracing pass, multiple scattering is computed by integrating the stored radiance against the scattering functions of visible fibers using an efficient matrix multiplication. Single scattering is computed using conventional direct illumination methods. In our comparisons the new method produces quality similar to that of the best previous methods, but computes multiple scattering more than 10 times faster.
---
paper_title: Realistic avatar eye and head animation using a neurobiological model of visual attention

paper_content:

We describe a neurobiological model of visual attention and eye/head movements in primates, and its application to the automatic animation of a realistic virtual human head watching an unconstrained variety of visual inputs. The bottom-up (image-based) attention model is based on the known neurophysiology of visual processing along the occipito-parietal pathway of the primate brain, while the eye/head movement model is derived from recordings in freely behaving Rhesus monkeys. The system is successful at autonomously saccading towards and tracking salient targets in a variety of video clips, including synthetic stimuli, real outdoors scenes and gaming console outputs. The resulting virtual human eye/head animation yields realistic rendering of the simulation results, both suggesting applicability of this approach to avatar animation and reinforcing the plausibility of the neural model.
---
paper_title: Modeling realistic virtual hairstyles

paper_content:

The author presents an effective method for modeling realistic curly hairstyles, taking into account both artificial hairstyling processes and natural curliness. The result is a detailed geometric model of hairs that can be rendered and animated via existing methods. Our technique exploits the analogy between hairs and a vector field; it interactively and efficiently models global and local hair flows by superimposing procedurally defined vector field primitives that have local influence. Usually only a very small number of vector field primitives are needed to model a complicated hairstyle. An initial model of hair strands is extracted from the superimposed vector fields by tracing their field lines. Random natural or artificial curliness can be added to the initial model through a parametric hair offset function with a randomized distribution of parameters over the scalp. Techniques for shearing and clustering are also designed to improve the overall appearance of the hair model. Our technique has been successfully applied to generate a variety of realistic hairstyles with different curliness and length distributions.
---
paper_title: Modeling hair from multiple views

paper_content:

In this paper, we propose a novel image-based approach to model hair geometry from images taken at multiple viewpoints. Unlike previous hair modeling techniques that require intensive user interactions or rely on special capturing setup under controlled illumination conditions, we use a handheld camera to capture hair images under uncontrolled illumination conditions. Our multi-view approach is natural and flexible for capturing. It also provides inherent strong and accurate geometric constraints to recover hair models.In our approach, the hair fibers are synthesized from local image orientations. Each synthesized fiber segment is validated and optimally triangulated from all visible views. The hair volume and the visibility of synthesized fibers can also be reliably estimated from multiple views. Flexibility of acquisition, little user interaction, and high quality results of recovered complex hair models are the key advantages of our method.
---
paper_title: Hair rendering by jittering and pseudo shadow

paper_content:

Proposes two properties of hair that we believe are essential to realistic hair rendering. To satisfy these two properties of hair, we introduce the "jittering and pseudo-hair model". The jittering and pseudo-shadows are achieved through the use of a shadow factor. Experimental results show that, with the incorporation of these two properties, hair images look more natural than images do without these properties, thus proving that these two properties do indeed improve the realism of hair images.
---
paper_title: Simulating multiple scattering in hair using a photon mapping approach

paper_content:

Simulating multiple scattering correctly is important for accurate rendering of hair. However, a volume of hair is a difficult scene to simulate because scattering from an individual fiber is very structured and forward directed, and because the radiance distributions that arise from many such scattering events remain quite directional. For these reasons, previous methods cannot compute accurate images substantially faster than Monte Carlo path tracing.This paper proposes a new physically accurate method for rendering hair that is based on previous volumetric photon mapping methods. The first pass generates a photon map by tracing particles through the hair geometry, depositing them along paths rather than at scattering events. The second pass ray traces the hair, computing direct illumination and looking up indirect radiance in the photon map. Photons are stored and looked up in 5D position-direction space to allow for the very directional radiance distributions that occur in hair. Together with a new radiance caching method for fibers, our method simulates difficult scattering problems in hair efficiently and with low noise.The new algorithm is validated against path tracing and also compared with a photograph of light scattering in real hair.
---
paper_title: Capture of hair geometry from multiple images

paper_content:

Hair is a major feature of digital characters. Unfortunately, it has a complex geometry which challenges standard modeling tools. Some dedicated techniques exist, but creating a realistic hairstyle still takes hours. Complementary to user-driven methods, we here propose an image-based approach to capture the geometry of hair.The novelty of this work is that we draw information from the scattering properties of the hair that are normally considered a hindrance. To do so, we analyze image sequences from a fixed camera with a moving light source. We first introduce a novel method to compute the image orientation of the hairs from their anisotropic behavior. This method is proven to subsume and extend existing work while improving accuracy. This image orientation is then raised into a 3D orientation by analyzing the light reflected by the hair fibers. This part relies on minimal assumptions that have been proven correct in previous work.Finally, we show how to use several such image sequences to reconstruct the complete hair geometry of a real person. Results are shown to illustrate the fidelity of the captured geometry to the original hair. This technique paves the way for a new approach to digital hair generation.
---

Title: Techniques of Facial Synthesis: A Comprehensive Literature Review

Section 1: INTRODUCTION
Description 1: Write about the significance of realistic image synthesis, applications in various fields, and the emergence and importance of facial analysis and synthesis.

Section 2: FACIAL EXPRESSION SYNTHESIS
Description 2: Summarize different approaches for synthesizing facial expressions, including geometric manipulations and image manipulations.

Section 3: Parameterization
Description 3: Discuss techniques for facial expressions that overcome limitations of simple interpolations, including the pros and cons of parameterizations.

Section 4: Physics-based muscle models
Description 4: Elaborate on the categories of physics-based muscle models, describing mass-spring systems, vector representations, layered spring meshes, and the Finite Element Method.

Section 5: Pseudo or simulated muscle
Description 5: Explain pseudo or simulated muscle modeling for realistic results and discuss free-form deformation techniques.

Section 6: Image Manipulations
Description 6: Describe 2D and 3D morphing effects and vascular expressions for facial animation.

Section 7: Model fitting
Description 7: Cover techniques for fitting a predefined wireframe on a face for realistic facial movement and expressions.

Section 8: Expressions using tracking
Description 8: Discuss performance-driven approaches wherein tracked human actors control facial expressions in real-time animations.

Section 9: LITERATURE REVIEW OF FACIAL SYNTHESIS MODELS
Description 9: Review literature on image-based face modeling and animation along five threads: statistical models, geometry modeling, appearance modeling, animation, and the modeling of eyes and hair.

Section 10: CONCLUSION AND FUTURE DIRECTIONS
Description 10: Summarize the current state of face image processing technologies, highlighting future research directions to achieve realism in automated facial expression synthesis.