A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms
---
paper_title: Robust Eye and Pupil Detection Method for Gaze Tracking

paper_content:

Robust and accurate pupil detection is a prerequisite for gaze detection. Hence, we propose a new eye/pupil detection method for gaze detection on a large display. The novelty of our research can be summarized by the following four points. First, in order to overcome the performance limitations of conventional methods of eye detection, such as adaptive boosting (Adaboost) and continuously adaptive mean shift (CAMShift) algorithms, we propose adaptive selection of the Adaboost and CAMShift methods. Second, this adaptive selection is based on two parameters: pixel differences in successive images and matching values determined by CAMShift. Third, a support vector machine (SVM)-based classifier is used with these two parameters as the input, which improves the eye detection performance. Fourth, the center of the pupil within the detected eye region is accurately located by means of circular edge detection, binarization and calculation of the geometric center. The experimental results show that the proposed method can detect the center of the pupil at a speed of approximately 19.4 frames/s with an RMS error of approximately 5.75 pixels, which is superior to the performance of conventional detection methods.
---
paper_title: MobiGaze: development of a gaze interface for handheld mobile devices

paper_content:

Handheld mobile devices that have a touch screen are widely used but are awkward to use with one hand. To solve this problem, we propose MobiGaze, which is a user interface that uses one's gaze (gaze interface) to operate a handheld mobile device. By using stereo cameras, the user's line of sight is detected in 3D, enabling the user to interact with a mobile device by means of his/her gaze. We have constructed a prototype system of MobiGaze that consists of two cameras with IR-LED, a Windows-based notebook PC, and iPod touch. Moreover, we have developed several applications for MobiGaze.
---
paper_title: Eye gaze tracking techniques for interactive applications

paper_content:

This paper presents a review of eye gaze tracking technology and focuses on recent advancements that might facilitate its use in general computer applications. Early eye gaze tracking devices were appropriate for scientific exploration in controlled environments. Although it has been thought for long that they have the potential to become important computer input devices as well, the technology still lacks important usability requirements that hinders its applicability. We present a detailed description of the pupil-corneal reflection technique due to its claimed usability advantages, and show that this method is still not quite appropriate for general interactive applications. Finally, we present several recent techniques for remote eye gaze tracking with improved usability. These new solutions simplify or eliminate the calibration procedure and allow free head motion.
---
paper_title: In the Eye of the Beholder: A Survey of Models for Eyes and Gaze

paper_content:

Despite active research and significant progress in the last 30 years, eye detection and tracking remains challenging due to the individuality of eyes, occlusion, variability in scale, location, and light conditions. Data on eye location and details of eye movements have numerous applications and are essential in face detection, biometric identification, and particular human-computer interaction tasks. This paper reviews current progress and state of the art in video-based eye detection and tracking in order to identify promising techniques as well as issues to be further addressed. We present a detailed review of recent eye models and techniques for eye detection and tracking. We also survey methods for gaze estimation and compare them based on their geometric properties and reported accuracies. This review shows that, despite their apparent simplicity, the development of a general eye detection technique involves addressing many challenges, requires further theoretical developments, and is consequently of interest to many other domains problems in computer vision and beyond.
---
paper_title: Passive driver gaze tracking with active appearance models

paper_content:

Monocular gaze estimation is usually performed by locating the pupils, and the inner and outer eye corners in the image of the driver’s head. Of these feature points, the eye corners are just as important, and perhaps harder to detect, than the pupils. The eye corners are usually found using local feature detectors and trackers. In this paper, we describe a monocular driver gaze tracking system which uses a global head model, specifically an Active Appearance Model (AAM), to track the whole head. From the AAM, the eye corners, eye region, and head pose are robustly extracted and then used to estimate the gaze.
---
paper_title: Robust Head Mounted Wearable Eye Tracking System for Dynamical Calibration

paper_content:

In this work, a new head mounted eye tracking system is presented. Based on computer vision techniques, the system integrates eye images and head movement, in real time, performing a robust gaze point tracking. Nystagmus movements due to vestibulo-ocular reflex are monitored and integrated. The system proposed here is a strongly improved version of a previous platform called HATCAM, which was robust against changes of illumination conditions. The new version, called HAT-Move, is equipped with accurate inertial motion unit to detect the head movement enabling eye gaze even in dynamical conditions. HAT-Move performance is investigated in a group of healthy subjects in both static and dynamic conditions, i.e. when head is kept still or free to move. Evaluation was performed in terms of amplitude of the angular error between the real coordinates of the fixed points and those computed by the system in two experimental setups, specifically, in laboratory settings and in a 3D virtual reality (VR) scenario. The achieved results showed that HAT-Move is able to achieve eye gaze angular error of about 1 degree along both horizontal and vertical directions
---
paper_title: Homography normalization for robust gaze estimation in uncalibrated setups

paper_content:

Homography normalization is presented as a novel gaze estimation method for uncalibrated setups. The method applies when head movements are present but without any requirements to camera calibration or geometric calibration. The method is geometrically and empirically demonstrated to be robust to head pose changes and despite being less constrained than cross-ratio methods, it consistently performs favorably by several degrees on both simulated data and data from physical setups. The physical setups include the use of off-the-shelf web cameras with infrared light (night vision) and standard cameras with and without infrared light. The benefits of homography normalization and uncalibrated setups in general are also demonstrated through obtaining gaze estimates (in the visible spectrum) using only the screen reflections on the cornea.
---
paper_title: EyeTab: model-based gaze estimation on unmodified tablet computers

paper_content:

Despite the widespread use of mobile phones and tablets, hand-held portable devices have only recently been identified as a promising platform for gaze-aware applications. Estimating gaze on portable devices is challenging given their limited computational resources, low quality integrated front-facing RGB cameras, and small screens to which gaze is mapped. In this paper we present EyeTab, a model-based approach for binocular gaze estimation that runs entirely on an unmodified tablet. EyeTab builds on set of established image processing and computer vision algorithms and adapts them for robust and near-realtime gaze estimation. A technical prototype evaluation with eight participants in a normal indoors office setting shows that EyeTab achieves an average gaze estimation accuracy of 6.88° of visual angle at 12 frames per second.
---
paper_title: Towards Gaze Interaction in Immersive Virtual Reality : Evaluation of a Monocular Eye Tracking Set-Up

paper_content:

Of all senses, it is visual perception that is predominantly deluded in Virtual Realities. Yet, the eyes of the observer, despite the fact that they are the fastest perceivable moving body part, have gotten relatively little attention as an interaction modality. A solid integration of gaze, however, provides great opportunities for implicit and explicit human-computer interaction. We present our work on integrating a lightweight head-mounted eye tracking system in a CAVE-like Virtual Reality Set-Up and provide promising data from a user study on the achieved accuracy and latency.
---
paper_title: Towards the development of a standardized performance evaluation framework for eye gaze estimation systems in consumer platforms

paper_content:

There is a need to standardize the performance of eye gaze estimation (EGE) methods in various platforms for human computer interaction (HCI). Because of lack of consistent schemes or protocols for summative evaluation of EGE systems, performance results in this field can neither be compared nor reproduced with any consistency. In contemporary literature, gaze tracking accuracy is measured under non-identical sets of conditions, with variable metrics and most results do not report the impact of system meta-parameters that significantly affect tracking performances. In this work, the diverse nature of these research outcomes and system parameters which affect gaze tracking in different platforms is investigated and their error contributions are estimated quantitatively. Then the concept and development of a performance evaluation framework is proposed- that can define design criteria and benchmark quality measures for the eye gaze research community.
---
paper_title: Eye-gaze systems - An analysis of error sources and potential accuracy in consumer electronics use cases

paper_content:

Several generic CE use cases and corresponding techniques for eye gaze estimation (EGE) are reviewed. The optimal approaches for each use case are determined from a review of recent literature. In addition, the most probable error sources for EGE are determined and the impact of these error sources is quantified. A discussion and analysis of the research outcome is given and future work outlined.
---
paper_title: Detecting eye position and gaze from a single camera and 2 light sources

paper_content:

We introduce a new method for computing the 3D position of an eye and its gaze direction from a single camera and at least two near infra-red light sources. The method is based on the theory of spherical optical surfaces and uses the Gullstrand model of the eye to estimate the positions of the center of the cornea and the center of the pupil in 3D. The direction of gaze can then be computed from the vector connecting these two points. The point of regard can also be computed from the intersection of the direction of gaze with an object in the scene. We have simulated this model using ray traced images of the eye, and obtained very promising results. The major contribution of this new technique over current eye tracking technology is that the system does not require to be calibrated with the user before each user session, and it allows for free head motion.
---
paper_title: From Gaze Control to Attentive Interfaces

paper_content:

Interactive applications that make use of eye tracking have traditionally been based on command-and-control. Applications that make more subtle use of eye gaze have recently become increasingly popular in the domain of attentive interfaces that adapt their behaviour based on the visual attention of the user. We provide a review of the main systems and application domains where this genre of interfaces has been used.
---
paper_title: Limbus/pupil switching for wearable eye tracking under variable lighting conditions

paper_content:

We present a low-cost wearable eye tracker built from off-the-shelf components. Based on the open source openEyes project (the only other similar effort that we are aware of), our eye tracker operates in the visible spectrum and variable lighting conditions. The novelty of our approach rests in automatically switching between tracking the pupil/iris boundary in bright light to tracking the iris/sclera boundary (limbus) in dim light. Additional improvements include a semi-automatic procedure for calibrating the eye and scene cameras, as well as an automatic procedure for initializing the location of the pupil in the first image frame. The system is accurate to two degrees visual angle in both indoor and outdoor environments.
---
paper_title: Eye Gaze for Consumer Electronics: Controlling and commanding intelligent systems.

paper_content:

Over the last several years, there has been much research and investigation on finding new ways to interact with the smart systems that currently form an integral part of our lives. One of the most widely researched fields in human?computer interaction (HCI) has been the use of human eye gaze as an input modality to control and command intelligent systems. For example, gaze-based schemes for hands-free input to computers for text entry/scrolling/pointing was proposed as early as in 1989 for disabled persons [1]. In the field of commercial applications, gaze-based interactions have brought immersive experiences in the world of virtual gaming and multimedia entertainment [2]. Eye gaze is also a significant feature for detecting the attention and intent of an individual. For example, gaze tracking could be implemented in a car to detect driver consciousness or in a smartphone to switch operations by sensing user attentiveness.
---
paper_title: New Solution to the Midas Touch Problem: Identification of Visual Commands Via Extraction of Focal Fixations

paper_content:

Abstract Reliable identification of intentional visual commands is a major problem in the development of eye-movements based user interfaces. This work suggests that the presence of focal visual fixations is indicative of visual commands. Two experiments are described which assessed the effectiveness of this approach in a simple gaze-control interface. Identification accuracy was shown to match that of the commonly used dwell time method. Using focal fixations led to less visual fatigue and higher speed of work. Perspectives of using focal fixations for identification of visual commands in various kinds of eye-movements based interfaces are discussed.
---
paper_title: General theory of remote gaze estimation using the pupil center and corneal reflections

paper_content:

This paper presents a general theory for the remote estimation of the point-of-gaze (POG) from the coordinates of the centers of the pupil and corneal reflections. Corneal reflections are produced by light sources that illuminate the eye and the centers of the pupil and corneal reflections are estimated in video images from one or more cameras. The general theory covers the full range of possible system configurations. Using one camera and one light source, the POG can be estimated only if the head is completely stationary. Using one camera and multiple light sources, the POG can be estimated with free head movements, following the completion of a multiple-point calibration procedure. When multiple cameras and multiple light sources are used, the POG can be estimated following a simple one-point calibration procedure. Experimental and simulation results suggest that the main sources of gaze estimation errors are the discrepancy between the shape of real corneas and the spherical corneal shape assumed in the general theory, and the noise in the estimation of the centers of the pupil and corneal reflections. A detailed example of a system that uses the general theory to estimate the POG on a computer screen is presented.
---
paper_title: In the Eye of the Beholder: A Survey of Models for Eyes and Gaze

paper_content:

Despite active research and significant progress in the last 30 years, eye detection and tracking remains challenging due to the individuality of eyes, occlusion, variability in scale, location, and light conditions. Data on eye location and details of eye movements have numerous applications and are essential in face detection, biometric identification, and particular human-computer interaction tasks. This paper reviews current progress and state of the art in video-based eye detection and tracking in order to identify promising techniques as well as issues to be further addressed. We present a detailed review of recent eye models and techniques for eye detection and tracking. We also survey methods for gaze estimation and compare them based on their geometric properties and reported accuracies. This review shows that, despite their apparent simplicity, the development of a general eye detection technique involves addressing many challenges, requires further theoretical developments, and is consequently of interest to many other domains problems in computer vision and beyond.
---
paper_title: Remote Gaze Tracking System on a Large Display

paper_content:

We propose a new remote gaze tracking system as an intelligent TV interface. Our research is novel in the following three ways: first, because a user can sit at various positions in front of a large display, the capture volume of the gaze tracking system should be greater, so the proposed system includes two cameras which can be moved simultaneously by panning and tilting mechanisms, a wide view camera (WVC) for detecting eye position and an auto-focusing narrow view camera (NVC) for capturing enlarged eye images. Second, in order to remove the complicated calibration between the WVC and NVC and to enhance the capture speed of the NVC, these two cameras are combined in a parallel structure. Third, the auto-focusing of the NVC is achieved on the basis of both the user’s facial width in the WVC image and a focus score calculated on the eye image of the NVC. Experimental results showed that the proposed system can be operated with a gaze tracking accuracy of ±0.737°~±0.775° and a speed of 5~10 frames/s.
---
paper_title: Gaze-based selection of standard-size menu items

paper_content:

With recent advances in eye tracking technology, eye gaze gradually gains acceptance as a pointing modality. Its relatively low accuracy, however, determines the need to use enlarged controls in eye-based interfaces rendering their design rather peculiar. Another factor impairing pointing performance is deficient robustness of an eye tracker's calibration. To facilitate pointing at standard-size menus, we developed a technique that uses dynamic target expansion for on-line correction of the eye tracker's calibration. Correction is based on the relative change in the gaze point location upon the expansion. A user study suggests that the technique affords a dramatic six-fold improvement in selection accuracy. This is traded off against a much smaller reduction in performance speed (39%). The technique is thus believed to contribute to development of universal-access solutions supporting navigation through standard menus by eye gaze alone.
---
paper_title: Towards Accurate Eye Tracker Calibration – Methods and Procedures☆

paper_content:

Abstract Eye movement is a new emerging modality in human computer interfaces. With better access to devices which are able to measure eye movements (so called eye trackers) it becomes accessible even in ordinary environments. However, the first problem that must be faced when working with eye movements is a correct mapping from an output of eye tracker to a gaze point – place where the user is looking at the screen. That is why the work must always be started with calibration of the device. The paper describes the process of calibration, analyses of the possible steps and ways how to simplify this process.
---
paper_title: General theory of remote gaze estimation using the pupil center and corneal reflections

paper_content:

This paper presents a general theory for the remote estimation of the point-of-gaze (POG) from the coordinates of the centers of the pupil and corneal reflections. Corneal reflections are produced by light sources that illuminate the eye and the centers of the pupil and corneal reflections are estimated in video images from one or more cameras. The general theory covers the full range of possible system configurations. Using one camera and one light source, the POG can be estimated only if the head is completely stationary. Using one camera and multiple light sources, the POG can be estimated with free head movements, following the completion of a multiple-point calibration procedure. When multiple cameras and multiple light sources are used, the POG can be estimated following a simple one-point calibration procedure. Experimental and simulation results suggest that the main sources of gaze estimation errors are the discrepancy between the shape of real corneas and the spherical corneal shape assumed in the general theory, and the noise in the estimation of the centers of the pupil and corneal reflections. A detailed example of a system that uses the general theory to estimate the POG on a computer screen is presented.
---
paper_title: Novel Eye Gaze Tracking Techniques Under Natural Head Movement

paper_content:

Most available remote eye gaze trackers have two characteristics that hinder them being widely used as the important computer input devices for human computer interaction. First, they have to be calibrated for each user individually; second, they have low tolerance for head movement and require the users to hold their heads unnaturally still. In this paper, by exploiting the eye anatomy, we propose two novel solutions to allow natural head movement and minimize the calibration procedure to only one time for a new individual. The first technique is proposed to estimate the 3D eye gaze directly. In this technique, the cornea of the eyeball is modeled as a convex mirror. Via the properties of convex mirror, a simple method is proposed to estimate the 3D optic axis of the eye. The visual axis, which is the true 3D gaze direction of the user, can be determined subsequently after knowing the angle deviation between the visual axis and optic axis by a simple calibration procedure. Therefore, the gaze point on an object in the scene can be obtained by simply intersecting the estimated 3D gaze direction with the object. Different from the first technique, our second technique does not need to estimate the 3D eye gaze directly, and the gaze point on an object is estimated from a gaze mapping function implicitly. In addition, a dynamic computational head compensation model is developed to automatically update the gaze mapping function whenever the head moves. Hence, the eye gaze can be estimated under natural head movement. Furthermore, it minimizes the calibration procedure to only one time for a new individual. The advantage of the proposed techniques over the current state of the art eye gaze trackers is that it can estimate the eye gaze of the user accurately under natural head movement, without need to perform the gaze calibration every time before using it. Our proposed methods will improve the usability of the eye gaze tracking technology, and we believe that it represents an important step for the eye tracker to be accepted as a natural computer input device.
---
paper_title: Multiperson Visual Focus of Attention from Head Pose and Meeting Contextual Cues

paper_content:

This paper introduces a novel contextual model for the recognition of people's visual focus of attention (VFOA) in meetings from audio-visual perceptual cues. More specifically, instead of independently recognizing the VFOA of each meeting participant from his own head pose, we propose to jointly recognize the participants' visual attention in order to introduce context-dependent interaction models that relate to group activity and the social dynamics of communication. Meeting contextual information is represented by the location of people, conversational events identifying floor holding patterns, and a presentation activity variable. By modeling the interactions between the different contexts and their combined and sometimes contradictory impact on the gazing behavior, our model allows us to handle VFOA recognition in difficult task-based meetings involving artifacts, presentations, and moving people. We validated our model through rigorous evaluation on a publicly available and challenging data set of 12 real meetings (5 hours of data). The results demonstrated that the integration of the presentation and conversation dynamical context using our model can lead to significant performance improvements.
---
paper_title: A novel 2-D mapping-based remote eye gaze tracking method using two IR light sources

paper_content:

Consumers nowadays demand for more convenient human-device interaction (HDI) in every device they use. As a part of the HDI technologies, remote eye gaze tracking (REGT) is getting more attentions. Among state-of-the-art REGT methods, the ones which utilize four infrared (IR) lights have better accuracy and robustness to head movements. In this paper, a novel glint estimation method is proposed, which enables four-glint-based REGT methods to operate with only two IR light sources. Given two glints obtained from IR light sources attached on the upper or lower corners of the screen, the proposed method estimates the other two virtual glints based on mathematical and geometric principles. Then, the point of gaze (POG) can be estimated using a mapping function established between the four glints and the screen through a user calibration process. Experimental results show that the REGT system using the proposed glint estimation method can achieve highly competitive performance compared with the ones which use four IR light sources.1
---
paper_title: Taxonomic study of polynomial regressions applied to the calibration of video-oculographic systems

paper_content:

Of gaze tracking techniques, video-oculography (VOG) is one of the most attractive because of its versatility and simplicity. VOG systems based on general purpose mapping methods use simple polynomial expressions to estimate a user's point of regard. Although the behaviour of such systems is generally acceptable, a detailed study of the calibration process is needed to facilitate progress in improving accuracy and tolerance to user head movement. To date, there has been no thorough comparative study of how mapping equations affect final system response. After developing a taxonomic classification of calibration functions, we examine over 400,000 models and evaluate the validity of several conventional assumptions. The rigorous experimental procedure employed enabled us to optimize the calibration process for a real VOG gaze tracking system and, thereby, halve the calibration time without detrimental effect on accuracy or tolerance to head movement.
---
paper_title: Mapping the Pupil-Glint Vector to Gaze Coordinates in a Simple Video-Based Eye Tracker

paper_content:

In a video-based eye tracker, the normalized pupil-glint vector changes as the eyes move. Using an appropriate model, the pupil-glint vector can be mapped to gaze coordinates. Using a simple hardware configuration with one camera and one infrared source, several mapping functions – some from literature and some derived here – were compared with one another with respect to the accuracy that could be achieved. The study served to confirm the results of a previous study with another data set and to expand on the possibilities that are considered from the previous study. The data of various participants was examined for trends which led to derivation of a mapping model that proved to be more accurate than all but one model from literature. It was also shown that the best calibration configuration for this hardware setup is one that contains fourteen targets while taking about 20 seconds for the procedure to be completed.
---
paper_title: 2D Gaze Estimation Based on Pupil-Glint Vector Using an Artificial Neural Network

paper_content:

Gaze estimation methods play an important role in a gaze tracking system. A novel 2D gaze estimation method based on the pupil-glint vector is proposed in this paper. First, the circular ring rays location (CRRL) method and Gaussian fitting are utilized for pupil and glint detection, respectively. Then the pupil-glint vector is calculated through subtraction of pupil and glint center fitting. Second, a mapping function is established according to the corresponding relationship between pupil-glint vectors and actual gaze calibration points. In order to solve the mapping function, an improved artificial neural network (DLSR-ANN) based on direct least squares regression is proposed. When the mapping function is determined, gaze estimation can be actualized through calculating gaze point coordinates. Finally, error compensation is implemented to further enhance accuracy of gaze estimation. The proposed method can achieve a corresponding accuracy of 1.29°, 0.89°, 0.52°, and 0.39° when a model with four, six, nine, or 16 calibration markers is utilized for calibration, respectively. Considering error compensation, gaze estimation accuracy can reach 0.36°. The experimental results show that gaze estimation accuracy of the proposed method in this paper is better than that of linear regression (direct least squares regression) and nonlinear regression (generic artificial neural network). The proposed method contributes to enhancing the total accuracy of a gaze tracking system.
---
paper_title: Subpixel eye gaze tracking

paper_content:

This paper addresses the accuracy problem of an eye gaze tracking system. We first analyze the technical barrier for a gaze tracking system to achieve a desired accuracy, and then propose a subpixel tracking method to break this barrier. We present new algorithms for detecting the inner eye corner and the center of an iris at subpixel accuracy, and we apply these new methods in developing a real-time gaze tracking system. Experimental results indicate that the new methods achieve an average accuracy within 1.4/spl deg/ using normal eye image resolutions.
---
paper_title: Implementation of an Eye Gaze Tracking System for the Disabled People

paper_content:

The paper proposes a modified pupil center corneal reflection(PCCR) hardware method to improve the system accuracy. The modified PCCR eye gaze tracking system, a new version of the PCCR eye gaze tracking system supplemented by the relation between IR LED position and the distance from the eye gaze tracking system to the monitor screen, improves the tracking accuracy within one degree. The system also suggests a circuit that can do power control adaptively between the minimum and maximum power. It is confirmed that the system performs well both for indoors and outdoors in spite of the reduced calculation. Besides, very convenient mouse functions are proposed so that can be used on PC with the eye gaze tracking functions. The user group confirmed their performances that result in high level of satisfaction with excellent tracking function. Above all, the paper suggests the adaptive exposure control algorithm for the proposed system which is robust against light. The adaptive exposure control algorithm presents excellent performance over the existing system for both indoors and outdoors even when the calculation is down to one fifth.
---
paper_title: Implicit Calibration of a Remote Gaze Tracker

paper_content:

We describe a system designed to monitor the gaze of a user working naturally at a computer workstation. The system consists of three cameras situated between the keyboard and the monitor. Free head movements are allowed within a three-dimensional volume approximately 40 centimeters in diameter. Two fixed, wide-field "face" cameras equipped with active-illumination systems enable rapid localization of the subject's pupils. A third steerable "eye" camera has a relatively narrow field of view, and acquires the images of the eyes which are used for gaze estimation. Unlike previous approaches which construct an explicit three-dimensional representation of the subject's head and eye, we derive mappings for steering control and gaze estimation using a procedure we call implicit calibration. Implicit calibration is performed by collecting a "training set" of parameters and associated measurements, and solving for a set of coefficients relating the measurements back to the parameters of interest. Preliminary data on three subjects indicate an median gaze estimation error of ap-proximately 0.8 degree.
---
paper_title: A novel approach to 3-d gaze tracking using stereo cameras

paper_content:

A novel approach to three-dimensional (3-D) gaze tracking using 3-D computer vision techniques is proposed in this paper. This method employs multiple cameras and multiple point light sources to estimate the optical axis of user's eye without using any user-dependent parameters. Thus, it renders the inconvenient system calibration process which may produce possible calibration errors unnecessary. A real-time 3-D gaze tracking system has been developed which can provide 30 gaze measurements per second. Moreover, a simple and accurate calibration method is proposed to calibrate the gaze tracking system. Before using the system, each user only has to stare at a target point for a few (2-3) seconds so that the constant angle between the 3-D line of sight and the optical axis can be estimated. The test results of six subjects showed that the gaze tracking system is very promising achieving an average estimation error of under 1/spl deg/.
---
paper_title: Real time eye gaze tracking with Kinect

paper_content:

Traditional gaze tracking systems rely on explicit infrared lights and high resolution cameras to achieve high performance and robustness. These systems, however, require complex setup and thus are restricted in lab research and hard to apply in practice. In this paper, we propose to perform gaze tracking with a consumer level depth sensor (Kinect). Leveraging on Kinect's capability to obtain 3D coordinates, we propose an efficient model-based gaze tracking system. We first build a unified 3D eye model to relate gaze directions and eye features (pupil center, eyeball center, cornea center) through subject-dependent eye parameters. A personal calibration framework is further proposed to estimate the subject-dependent eye parameters. Finally we can perform real time gaze tracking given the 3D coordinates of eye features from Kinect and the subject-dependent eye parameters from personal calibration procedure. Experimental results with 6 subjects prove the effectiveness of the proposed 3D eye model and the personal calibration framework. Furthermore, the gaze tracking system is able to work in real time (20 fps) and with low resolution eye images.
---
paper_title: General theory of remote gaze estimation using the pupil center and corneal reflections

paper_content:

This paper presents a general theory for the remote estimation of the point-of-gaze (POG) from the coordinates of the centers of the pupil and corneal reflections. Corneal reflections are produced by light sources that illuminate the eye and the centers of the pupil and corneal reflections are estimated in video images from one or more cameras. The general theory covers the full range of possible system configurations. Using one camera and one light source, the POG can be estimated only if the head is completely stationary. Using one camera and multiple light sources, the POG can be estimated with free head movements, following the completion of a multiple-point calibration procedure. When multiple cameras and multiple light sources are used, the POG can be estimated following a simple one-point calibration procedure. Experimental and simulation results suggest that the main sources of gaze estimation errors are the discrepancy between the shape of real corneas and the spherical corneal shape assumed in the general theory, and the noise in the estimation of the centers of the pupil and corneal reflections. A detailed example of a system that uses the general theory to estimate the POG on a computer screen is presented.
---
paper_title: User-calibration-free remote eye-gaze tracking system with extended tracking range

paper_content:

A novel general method to extend the tracking range of user-calibration-free remote eye-gaze tracking (REGT) systems that are based on the analysis of stereo-images from multiple cameras is presented. The method consists of two distinct phases. In the brief initial phase, estimates of the center of the pupil and corneal reflections in pairs of stereo-images are used to estimate automatically a set of subject-specific eye parameters. In the second phase, these subject — specific eye parameters are used with estimates of the center of the pupil and corneal reflections in images from any one of the systems' cameras to compute the Point-of-Gaze (PoG). Experiments with a system that includes two cameras show that the tracking range for horizontal gaze directions can be extended from ±23.2° when the two cameras are used as a stereo pair to ±35.5° when the two cameras are used independently to estimate the POG.
---
paper_title: A single camera eye-gaze tracking system with free head motion

paper_content:

Eye-gaze as a form of human machine interface holds great promise for improving the way we interact with machines. Eye-gaze tracking devices that are non-contact, non-restrictive, accurate and easy to use will increase the appeal for including eye-gaze information in future applications. The system we have developed and which we describe in this paper achieves these goals using a single high resolution camera with a fixed field of view. The single camera system has no moving parts which results in rapid reacquisition of the eye after loss of tracking. Free head motion is achieved using multiple glints and 3D modeling techniques. Accuracies of under 1° of visual angle are achieved over a field of view of 14x12x20 cm and over various hardware configurations, camera resolutions and frame rates.
---
paper_title: Calibration-free gaze tracking using a binocular 3D eye model

paper_content:

This paper presents a calibration-free method for estimating the point of gaze (POG) on a display by using two pairs of stereo cameras. By using one pair of cameras and two light sources, the optical axis of the eye and the position of the center of the cornea can be estimated. This estimation is carried out by using a spherical model of the cornea. One pair of cameras is used for the estimation of the optical axis of the left eye, and the other pair is used for the estimation of the optical axis of the right eye. The point of intersection of optical axis with the display is termed the point of the optical axis (POA). The POG is approximately estimated as the midpoint of the line joining POAs of both the eyes with the display. We have developed a prototype system based on this method and demonstrated that the midpoint of POAs was closer to the fiducial point that the user gazed at than each POA.
---
paper_title: Hybrid Method for 3-D Gaze Tracking Using Glint and Contour Features

paper_content:

Glint features have important roles in gaze-tracking systems. However, when the operation range of a gaze-tracking system is enlarged, the performance of glint-feature-based (GFB) approaches will be degraded mainly due to the curvature variation problem at around the edge of the cornea. Although the pupil contour feature may provide complementary information to help estimating the eye gaze, existing methods do not properly handle the cornea refraction problem, leading to inaccurate results. This paper describes a contour-feature-based (CFB) 3-D gaze-tracking method that is compatible to cornea refraction. We also show that both the GFB and CFB approaches can be formulated in a unified framework and, thus, they can be easily integrated. Furthermore, it is shown that the proposed CFB method and the GFB method should be integrated because the two methods provide complementary information that helps to leverage the strength of both features, providing robustness and flexibility to the system. Computer simulations and real experiments show the effectiveness of the proposed approach for gaze tracking.
---
paper_title: Gaze estimation method based on an aspherical model of the cornea: surface of revolution about the optical axis of the eye

paper_content:

A novel gaze estimation method based on a novel aspherical model of the cornea is proposed in this paper. The model is a surface of revolution about the optical axis of the eye. The calculation method is explained on the basis of the model. A prototype system for estimating the point of gaze (POG) has been developed using this method. The proposed method has been found to be more accurate than the gaze estimation method based on a spherical model of the cornea.
---
paper_title: Novel Eye Gaze Tracking Techniques Under Natural Head Movement

paper_content:

Most available remote eye gaze trackers have two characteristics that hinder them being widely used as the important computer input devices for human computer interaction. First, they have to be calibrated for each user individually; second, they have low tolerance for head movement and require the users to hold their heads unnaturally still. In this paper, by exploiting the eye anatomy, we propose two novel solutions to allow natural head movement and minimize the calibration procedure to only one time for a new individual. The first technique is proposed to estimate the 3D eye gaze directly. In this technique, the cornea of the eyeball is modeled as a convex mirror. Via the properties of convex mirror, a simple method is proposed to estimate the 3D optic axis of the eye. The visual axis, which is the true 3D gaze direction of the user, can be determined subsequently after knowing the angle deviation between the visual axis and optic axis by a simple calibration procedure. Therefore, the gaze point on an object in the scene can be obtained by simply intersecting the estimated 3D gaze direction with the object. Different from the first technique, our second technique does not need to estimate the 3D eye gaze directly, and the gaze point on an object is estimated from a gaze mapping function implicitly. In addition, a dynamic computational head compensation model is developed to automatically update the gaze mapping function whenever the head moves. Hence, the eye gaze can be estimated under natural head movement. Furthermore, it minimizes the calibration procedure to only one time for a new individual. The advantage of the proposed techniques over the current state of the art eye gaze trackers is that it can estimate the eye gaze of the user accurately under natural head movement, without need to perform the gaze calibration every time before using it. Our proposed methods will improve the usability of the eye gaze tracking technology, and we believe that it represents an important step for the eye tracker to be accepted as a natural computer input device.
---
paper_title: A single-camera remote eye tracker

paper_content:

Many eye-tracking systems either require the user to keep their head still or involve cameras or other equipment mounted on the user's head. While acceptable for research applications, these limitations make the systems unsatisfactory for prolonged use in interactive applications. Since the goal of our work is to use eye trackers for improved visual communication through gaze guidance [1,2] and for Augmentative and Alternative Communication (AAC) [3], we are interested in less invasive eye tracking techniques.
---
paper_title: Detecting eye position and gaze from a single camera and 2 light sources

paper_content:

We introduce a new method for computing the 3D position of an eye and its gaze direction from a single camera and at least two near infra-red light sources. The method is based on the theory of spherical optical surfaces and uses the Gullstrand model of the eye to estimate the positions of the center of the cornea and the center of the pupil in 3D. The direction of gaze can then be computed from the vector connecting these two points. The point of regard can also be computed from the intersection of the direction of gaze with an object in the scene. We have simulated this model using ray traced images of the eye, and obtained very promising results. The major contribution of this new technique over current eye tracking technology is that the system does not require to be calibrated with the user before each user session, and it allows for free head motion.
---
paper_title: A free-head, simple calibration, gaze tracking system that enables gaze-based interaction

paper_content:

Human eye gaze is a strong candidate to create a new application area based on human-computer interaction. To implement a really practical gaze-based interaction system, gaze detection must be realized without placing any restriction on the user's behavior or comfort. This paper describes a gaze tracking system that offers freehead, simple personal calibration. It does not require the user wear anything on her head, and she can move her head freely. Personal calibration takes only a very short time; the user is asked to look at two markers on the screen. An experiment shows that the accuracy of the implemented system is about 1.0 degrees (view angle).
---
paper_title: Eye gaze tracking using an active stereo head

paper_content:

In the eye gaze tracking problem, the goal is to determine where on a monitor screen a computer user is looking, ie., the gaze point. Existing systems generally have one of two limitations: either the head must remain fixed in front of a stationary camera, or, to allow for head motion, the user must wear an obstructive device. We introduce a 3D eye tracking system where the head motion is allowed without the need for markers or worn devices. We use a pair of stereo systems: a wide angle stereo system detects the face and steers an active narrow FOV stereo system to track the eye at high resolution. For high resolution tracking, the eye is modeled in 3D, including the corneal ball, pupil and fovea. We discuss the calibration of the stereo systems, the eye model, eye detection and tracking, and we close with an evaluation of the accuracy of the estimated gaze point on the monitor.
---
paper_title: 3D eye model-based gaze estimation from a depth sensor

paper_content:

In this paper, we address the 3D eye gaze estimation problem using a low-cost, simple-setup, and non-intrusive consumer depth sensor (Kinect sensor). We present an effective and accurate method based on 3D eye model to estimate the point of gaze of a subject with the tolerance of free head movement. To determine the parameters involved in the proposed eye model, we propose i) an improved convolution-based means of gradients iris center localization method to accurately and efficiently locate the iris center in 3D space; ii) a geometric constraints-based method to estimate the eyeball center under the constraints that all the iris center points are distributed on a sphere originated from the eyeball center and the sizes of two eyeballs of a subject are identical; iii) an effective Kappa angle calculation method based on the fact that the visual axes of both eyes intersect at a same point with the screen plane. The final point of gaze is calculated by using the estimated eye model parameters. We experimentally evaluate our gaze estimation method on five subjects. The experimental results show the good performance of the proposed method with an average estimation accuracy of 3.78°, which outperforms several state-of-the-arts.
---
paper_title: Taxonomic study of polynomial regressions applied to the calibration of video-oculographic systems

paper_content:

Of gaze tracking techniques, video-oculography (VOG) is one of the most attractive because of its versatility and simplicity. VOG systems based on general purpose mapping methods use simple polynomial expressions to estimate a user's point of regard. Although the behaviour of such systems is generally acceptable, a detailed study of the calibration process is needed to facilitate progress in improving accuracy and tolerance to user head movement. To date, there has been no thorough comparative study of how mapping equations affect final system response. After developing a taxonomic classification of calibration functions, we examine over 400,000 models and evaluate the validity of several conventional assumptions. The rigorous experimental procedure employed enabled us to optimize the calibration process for a real VOG gaze tracking system and, thereby, halve the calibration time without detrimental effect on accuracy or tolerance to head movement.
---
paper_title: Towards accurate and robust cross-ratio based gaze trackers through learning from simulation

paper_content:

Cross-ratio (CR) based methods offer many attractive properties for remote gaze estimation using a single camera in an uncalibrated setup by exploiting invariance of a plane projectivity. Unfortunately, due to several simplification assumptions, the performance of CR-based eye gaze trackers decays significantly as the subject moves away from the calibration position. In this paper, we introduce an adaptive homography mapping for achieving gaze prediction with higher accuracy at the calibration position and more robustness under head movements. This is achieved with a learning-based method for compensating both spatially-varying gaze errors and head pose dependent errors simultaneously in a unified framework. The model of adaptive homography is trained offline using simulated data, saving a tremendous amount of time in data collection. We validate the effectiveness of the proposed approach using both simulated and real data from a physical setup. We show that our method compares favorably against other state-of-the-art CR based methods.
---
paper_title: Homography normalization for robust gaze estimation in uncalibrated setups

paper_content:

Homography normalization is presented as a novel gaze estimation method for uncalibrated setups. The method applies when head movements are present but without any requirements to camera calibration or geometric calibration. The method is geometrically and empirically demonstrated to be robust to head pose changes and despite being less constrained than cross-ratio methods, it consistently performs favorably by several degrees on both simulated data and data from physical setups. The physical setups include the use of off-the-shelf web cameras with infrared light (night vision) and standard cameras with and without infrared light. The benefits of homography normalization and uncalibrated setups in general are also demonstrated through obtaining gaze estimates (in the visible spectrum) using only the screen reflections on the cornea.
---
paper_title: Augmenting the robustness of cross-ratio gaze tracking methods to head movement

paper_content:

Remote gaze estimation using a single non-calibrated camera, simple user calibration or calibration free, and robust to head movements are very desirable features of eye tracking systems. Because cross-ratio (CR) is an invariant property of projective geometry, gaze estimation methods that rely on this property have the potential to provide these features, though most current implementations rely on a few simplifications that compromise the performance of the method. In this paper, the CR method for gaze tracking is revisited, and we introduce a new method that explicitly compensates head movements using a simple 3 parameter eye model. The method uses a single non-calibrated camera and requires a simple calibration procedure per user to estimate the eye parameters. We have conducted simulations and experiments with real users that show significant improvements over current state-of-the-art CR methods that do not explicitly compensate for head motion.
---
paper_title: Free head motion eye gaze tracking using a single camera and multiple light sources

paper_content:

One of the main limitations of current remote eye gaze tracking (REGT) techniques is that the user's head must remain within a very limited area in front of the monitor screen. In this paper, we present a free head motion REGT technique. By projecting a known rectangular pattern of lights, the technique estimates the gaze position relative to this rectangle using an invariant property of projective geometry. We carry extensive analysis of similar methods using an eye model to compare their accuracy. Based on these results, we propose a new estimation procedure that compensates the angular difference between the eye visual axis and optical axis. We have developed a real time (30 fps) prototype using a single camera and 5 light sources to generate the light pattern. Experimental results shows that the accuracy of the system is about 1deg of visual angle
---
paper_title: Gaze direction estimation using support vector machine with active appearance model

paper_content:

In recent years, research on human-computer interaction is becoming popular, most of which uses body movements, gestures or eye gaze direction. Until now, gazing estimation is still an active research domain. We propose an efficient method to solve the problem of the eye gaze point. We first locate the eye region by modifying the characteristics of the Active Appearance Model (AAM). Then by employing the Support Vector Machine (SVM), we estimate the five gazing directions through classification. The original 68 facial feature points in AAM are modified into 36 eye feature points. According to the two-dimensional coordinates of feature points, we classify different directions of eye gazing. The modified 36 feature points describe the contour of eyes, iris size, iris location, and the position of pupils. In addition, the resolution of cameras does not affect our method to determine the direction of line of sight accurately. The final results show the independence of classifications, less classification errors, and more accurate estimation of the gazing directions.
---
paper_title: Appearance-Based Gaze Estimation in the Wild

paper_content:

Appearance-based gaze estimation is believed to work well in real-world settings, but existing datasets have been collected under controlled laboratory conditions and methods have been not evaluated across multiple datasets. In this work we study appearance-based gaze estimation in the wild. We present the MPIIGaze dataset that contains 213,659 images we collected from 15 participants during natural everyday laptop use over more than three months. Our dataset is significantly more variable than existing ones with respect to appearance and illumination. We also present a method for in-the-wild appearance-based gaze estimation using multimodal convolutional neural networks that significantly outperforms state-of-the art methods in the most challenging cross-dataset evaluation. We present an extensive evaluation of several state-of-the-art image-based gaze estimation algorithms on three current datasets, including our own. This evaluation provides clear insights and allows us to identify key research challenges of gaze estimation in the wild.
---
paper_title: Manifold Alignment for Person Independent Appearance-Based Gaze Estimation

paper_content:

We show that dually supervised manifold embedding can improve the performance of machine learning based person-independent and thus calibration-free gaze estimation. For this purpose, we perform a manifold embedding for each person in the training dataset and then learn a linear transformation that aligns the individual, person-dependent manifolds. We evaluate the effect of manifold alignment on the recently presented Columbia dataset, where we analyze the influence on 6 regression methods and 8 feature variants. Using manifold alignment, we are able to improve the person-independent gaze estimation performance by up to 31.2 % compared to the best approach without manifold alignment.
---
paper_title: Estimation of eye gaze direction angles based on active appearance models

paper_content:

In this paper we demonstrate efficient methods for continuous estimation of eye gaze angles with application to sign language videos. The difficulty of the task lies on the fact that those videos contain images with low face resolution since they are recorded from distance. First, we proceed to the modeling of face and eyes region by training and fitting Global and Local Active Appearance Models (LAAM). Next, we propose a system for eye gaze estimation based on a machine learning approach. In the first stage of our method, we classify gaze into discrete classes using GMMs that are based either on the parameters of the LAAM, or on HOG descriptors for the eyes region. We also propose a method for computing gaze direction angles from GMM log-likelihoods. We qualitatively and quantitatively evaluate our methods on two sign language databases and compare with a state of the art geometric model of the eye based on LAAM landmarks, which provides an estimate in direction angles. Finally, we further evaluate our framework by getting ground truth data from an eye tracking system Our proposed methods, and especially the GMMs using LAAM parameters, demonstrate high accuracy and robustness even in challenging tasks.
---
paper_title: Local Binary Pattern Histogram features for on-screen eye-gaze direction estimation and a comparison of appearance based methods

paper_content:

Human Computer Interaction (HCI) has become an important focus of both computer science researches and industrial applications. And, on-screen gaze estimation is one of the hottest topics in this rapidly growing field. Eye-gaze direction estimation is a sub-research area of on-screen gaze estimation and the number of studies that focused on the estimation of on-screen gaze direction is limited. Due to this, various appearance-based video-oculography methods are investigated in this work. Firstly, a new dataset is created via user images taken from daylight censored cameras located at computer screen. Then, Local Binary Pattern Histogram (LBPH), which is used in this work for the first time to obtain on-screen gaze direction information, and Principal Component Analysis (PCA) methods are employed to extract image features. And, parameter optimized Support Vector Machine (SVM), Artificial Neural Networks (ANNs) and k-Nearest Neighbor (k-NN) learning methods are adopted in order to estimate on-screen gaze direction. Finally, these methods' abilities to correctly estimate the on-screen gaze direction are compared using the resulting classification accuracies of applied methods and previous works. The best classification accuracy of 96.67% is obtained when using LBPH and SVM method pair which is better than previous works. The results also show that appearance based methods are pretty applicable for estimating on-screen gaze direction.
---
paper_title: Statistical models of appearance for eye tracking and eye-blink detection and measurement

paper_content:

A statistical active appearance model (AAM) is developed to track and detect eye blinking. The model has been designed to be robust to variations of head pose or gaze. In particular we analyze and determine the model parameters which encode the variations caused by blinking. This global model is further extended using a series of sub-models to enable independent modeling and tracking of the two eye regions. Several methods to enable measurement and detection of eye-blink are proposed and evaluated. The results of various tests on different image databases are presented to validate each model.
---
paper_title: Eye detection using discriminatory Haar features and a new efficient SVM

paper_content:

This paper presents an accurate and efficient eye detection method using the discriminatory Haar features (DHFs) and a new efficient support vector machine (eSVM). The DHFs are extracted by applying a discriminating feature extraction (DFE) method to the 2D Haar wavelet transform. The DFE method is capable of extracting multiple discriminatory features for two-class problems based on two novel measure vectors and a new criterion in the whitened principal component analysis (PCA) space. The eSVM significantly improves the computational efficiency upon the conventional SVM for eye detection without sacrificing the generalization performance. Experiments on the Face Recognition Grand Challenge (FRGC) database and the BioID face database show that (i) the DHFs exhibit promising classification capability for eye detection problem; (ii) the eSVM runs much faster than the conventional SVM; and (iii) the proposed eye detection method achieves near real-time eye detection speed and better eye detection performance than some state-of-the-art eye detection methods. A discriminating feature extraction (DFE) method for two-class problems is proposed.The DFE method is applied to derive the discriminatory Haar features (DHFs) for eye detection.An efficient support vector machine (eSVM) is proposed to improve the efficiency of the SVM.An accurate and efficient eye detection method is presented using the DHFs and the eSVM.
---
paper_title: Eye Tracking for Everyone

paper_content:

From scientific research to commercial applications, eye tracking is an important tool across many domains. Despite its range of applications, eye tracking has yet to become a pervasive technology. We believe that we can put the power of eye tracking in everyone's palm by building eye tracking software that works on commodity hardware such as mobile phones and tablets, without the need for additional sensors or devices. We tackle this problem by introducing GazeCapture, the first large-scale dataset for eye tracking, containing data from over 1450 people consisting of almost 2:5M frames. Using GazeCapture, we train iTracker, a convolutional neural network for eye tracking, which achieves a significant reduction in error over previous approaches while running in real time (10–15fps) on a modern mobile device. Our model achieves a prediction error of 1.71cm and 2.53cm without calibration on mobile phones and tablets respectively. With calibration, this is reduced to 1.34cm and 2.12cm. Further, we demonstrate that the features learned by iTracker generalize well to other datasets, achieving state-of-the-art results. The code, data, and models are available at http://gazecapture.csail.mit.edu.
---
paper_title: Estimating 3D Gaze Directions Using Unlabeled Eye Images via Synthetic Iris Appearance Fitting

paper_content:

Estimating three-dimensional (3D) human eye gaze by capturing a single eye image without active illumination is challenging. Although the elliptical iris shape provides a useful cue, existing methods face difficulties in ellipse fitting due to unreliable iris contour detection. These methods may fail frequently especially with low resolution eye images. In this paper, we propose a synthetic iris appearance fitting (SIAF) method that is model-driven to compute 3D gaze direction from iris shape. Instead of fitting an ellipse based on exactly detected iris contour, our method first synthesizes a set of physically possible iris appearances and then optimizes inside this synthetic space to find the best solution to explain the captured eye image. In this way, the solution is highly constrained and guaranteed to be physically feasible. In addition, the proposed advanced image analysis techniques also help the SIAF method be robust to the unreliable iris contour detection. Furthermore, with multiple eye images, we propose a SIAF-joint method that can further reduce the gaze error by half, and it also resolves the binary ambiguity which is inevitable in conventional methods based on simple ellipse fitting.
---
paper_title: Gaze tracking by Binocular Vision and LBP features

paper_content:

In this paper, a new method for eye gaze tracking is proposed under natural head movement. In this method, Local-Binary-Pattern Texture Feature (LBP) is adopted to calculate the eye features according to the characteristic of the eye, and a precise Binocular Vision approach is used to detect the space coordinate of the eye. The combined features of space coordinates and LBP features of the eyes are fed into Support Vector Regression (SVR) to match the gaze mapping function, in the hope of tracking gaze direction under natural head movement. The experimental results prove that the proposed method can determine the gaze direction accurately.
---
paper_title: Appearance-Based Gaze Tracking with Free Head Movement

paper_content:

In this work, we develop an appearance-based gaze tracking system allowing user to move their head freely. The main difficulty of the appearance-based gaze tracking method is that the eye appearance is sensitive to head orientation. To overcome the difficulty, we propose a 3-D gaze tracking method combining head pose tracking and appearance-based gaze estimation. We use a random forest approach to model the neighbor structure of the joint head pose and eye appearance space, and efficiently select neighbors from the collected high dimensional data set. Li-optimization is then used to seek for the best solution for regression from the selected neighboring samples. Experiment results shows that it can provide robust binocular gaze tracking results with less constraints but still provides moderate estimation accuracy of gaze estimation.
---
paper_title: Free head motion eye gaze tracking using a single camera and multiple light sources

paper_content:

One of the main limitations of current remote eye gaze tracking (REGT) techniques is that the user's head must remain within a very limited area in front of the monitor screen. In this paper, we present a free head motion REGT technique. By projecting a known rectangular pattern of lights, the technique estimates the gaze position relative to this rectangle using an invariant property of projective geometry. We carry extensive analysis of similar methods using an eye model to compare their accuracy. Based on these results, we propose a new estimation procedure that compensates the angular difference between the eye visual axis and optical axis. We have developed a real time (30 fps) prototype using a single camera and 5 light sources to generate the light pattern. Experimental results shows that the accuracy of the system is about 1deg of visual angle
---
paper_title: Eye-gaze tracking system based on particle swarm optimization and BP neural network

paper_content:

In order to enhance the practicability and accuracy of the eye-gaze tracking system, a new type low pixel eye feature point location method is adopted. This method can accurately extract the eye-gaze features, namely iris centre point and canthus points when the image pickup requirements are low. The eye-gaze tracking method based on particle swarm optimization (PSO) BP neural network is raised, to capture pictures of eyes under the same environment, and a regression model where the connection weights and threshold values are optimized by PSO algorithm is built via BP network. This method is free of the inherent defects of BP network. This method requires only a common camera and normal illumination intensity rather than high-standard hardware, which greatly cuts the restrictive requirements for the system hardware and thus enhances the system practicability. The experiment results show that PSO-BP model is of higher robustness and accuracy than BP model, and is of higher recognition rate and can effectively enhances the eye-gaze tracking accuracy.
---
paper_title: Real-time eye gaze direction classification using convolutional neural network

paper_content:

Estimation eye gaze direction is useful in various human-computer interaction tasks. Knowledge of gaze direction can give valuable information regarding users point of attention. Certain patterns of eye movements known as eye accessing cues are reported to be related to the cognitive processes in the human brain. We propose a real-time framework for the classification of eye gaze direction and estimation of eye accessing cues. In the first stage, the algorithm detects faces using a modified version of the Viola-Jones algorithm. A rough eye region is obtained using geometric relations and facial landmarks. The eye region obtained is used in the subsequent stage to classify the eye gaze direction. A convolutional neural network is employed in this work for the classification of eye gaze direction. The proposed algorithm was tested on Eye Chimera database and found to outperform state of the art methods. The computational complexity of the algorithm is very less in the testing phase. The algorithm achieved an average frame rate of 24 fps in the desktop environment.
---
paper_title: Eye-gaze tracking system by haar cascade classifier

paper_content:

Human can quickly and effortlessly focus on a few most interesting points in an image. Different observers tend to have the same fixations towards the same scene. In order to predict observer's fixations, eye gaze information can be used to reveal human attention and interest. This paper presents a real-time eye gaze tracking system. Haar cascade classifier is used to calculate the position of eye gaze based on the rectangular features of human eye. Then this position is adopted to match the space coordinates of screen representing where an observer is looking. The experimental results from different kinds of scenes validate the effectiveness of our system.
---
paper_title: Driver gaze tracker using deformable template matching

paper_content:

In driver assistance system, human eye gaze direction is an important feature described some driver's situation such as distraction and fatigue. This paper proposes a method to track driver's gaze direction by using deformable template matching. The method is divided into three steps: first, identifying the face area. Second, localizing the eye area. Finally, combining the eye region model and sight algorithm to determine the driver's gaze direction. Experimental results show that this method can effectively identify human's eye and track gaze motion in a real-time running basis.
---
paper_title: Eye Tracking by Template Matching using an Automatic Codebook Generation Scheme

paper_content:

We present an eye tracking algorithm which is robust against variations in scale, orientation and changes of eye appearances, such as eye blinking. The locations of the eye regions in the different frames are found using template matching. The method is kept invariant for rotations and scale by exploiting temporal information and by using a codebook of eye templates the method is robust against changes in eye region appearances. The entries of the codebook are generated automatically during the tracking of the eye regions and eventually represent a distinct set of eye appearances. Classification into different eye gestures, like blinking or different gaze directions, seems possible using these automatically learned patterns.
---
paper_title: A 2 D eye gaze estimation system with low-resolution webcam images

paper_content:

In this article, a low-cost system for 2D eye gaze estimation with low-resolution webcam images is presented. Two algorithms are proposed for this purpose, one for the eye-ball detection with stable approximate pupil-center and the other one for the eye movements' direction detection. Eyeball is detected using deformable angular integral search by minimum intensity (DAISMI) algorithm. Deformable template-based 2D gaze estimation (DTBGE) algorithm is employed as a noise filter for deciding the stable movement decisions. While DTBGE employs binary images, DAISMI employs gray-scale images. Right and left eye estimates are evaluated separately. DAISMI finds the stable approximate pupil-center location by calculating the mass-center of eyeball border vertices to be employed for initial deformable template alignment. DTBGE starts running with initial alignment and updates the template alignment with resulting eye movements and eyeball size frame by frame. The horizontal and vertical deviation of eye movements through eyeball size is considered as if it is directly proportional with the deviation of cursor movements in a certain screen size and resolution. The core advantage of the system is that it does not employ the real pupil-center as a reference point for gaze estimation which is more reliable against corneal reflection. Visual angle accuracy is used for the evaluation and benchmarking of the system. Effectiveness of the proposed system is presented and experimental results are shown.
---
paper_title: GazePointer: A real time mouse pointer control implementation based on eye gaze tracking

paper_content:

The field of Human-Computer Interaction (HCI) has witnessed a tremendous growth in the past decade. The advent of tablet PCs and cell phones allowing touch-based control has been hailed warmly. The researchers in this field have also explored the potential of `eye-gaze' as a possible means of interaction. Some commercial solutions have already been launched, but they are as yet expensive and offer limited usability. This paper strives to present a low cost real time system for eye-gaze based human-computer interaction.
---
paper_title: Reducing shoulder-surfing by using gaze-based password entry

paper_content:

Shoulder-surfing -- using direct observation techniques, such as looking over someone's shoulder, to get passwords, PINs and other sensitive personal information -- is a problem that has been difficult to overcome. When a user enters information using a keyboard, mouse, touch screen or any traditional input device, a malicious observer may be able to acquire the user's password credentials. We present EyePassword, a system that mitigates the issues of shoulder surfing via a novel approach to user input. With EyePassword, a user enters sensitive input (password, PIN, etc.) by selecting from an on-screen keyboard using only the orientation of their pupils (i.e. the position of their gaze on screen), making eavesdropping by a malicious observer largely impractical. We present a number of design choices and discuss their effect on usability and security. We conducted user studies to evaluate the speed, accuracy and user acceptance of our approach. Our results demonstrate that gaze-based password entry requires marginal additional time over using a keyboard, error rates are similar to those of using a keyboard and subjects preferred the gaze-based password entry approach over traditional methods.
---
paper_title: Cheap and Easy PIN Entering Using Eye Gaze

paper_content:

PINs are one of the most popular methods to perform simple and fast user authentication.PIN stands for Personal Identification Number, which may have any number of digits or even letters.Nevertheless, 4-digit PIN is the most common and is used for instance in ATMs or cellular phones.The main advantage of the PIN is that it is easy to remember and fast to enter. There are, however,some drawbacks. One of them - addressed in this paper - is a possibility to steal PIN by a techniquecalled `shoulder surfing'. To avoid such problems a novel method of the PIN entering was proposed.Instead of using a numerical keyboard, the PIN may be entered by eye gazes, which is a hands-free,easy and robust technique.
---
paper_title: Evaluation of the Potential of Gaze Input for Game Interaction

paper_content:

To evaluate the potential of gaze input for game interaction, we used two tasks commonly found in video game control, target acquisition and target tracking, in a set of two experiments. In the first experiment, we compared the target acquisition and target tracking performance of two eye trackers with four other input devices. Gaze input had a similar performance to the mouse for big targets, and better performance than a joystick, a device often used in gaming. In the second experiment, we compared target acquisition performance using either gaze or mouse for pointing, and either a mouse button or an EMG switch for clicking. The hands-free gaze-EMG input combination was faster than the mouse while maintaining a similar error rate. Our results suggest that there is a potential for gaze input in game interaction, given a sufficiently accurate and responsive eye tracker and a well-designed interface.
---
paper_title: Evaluation of eye gaze interaction

paper_content:

Eye gaze interaction can provide a convenient and natural addition to user-computer dialogues. We have previously reported on our interaction techniques using eye gaze [10]. While our techniques seemed useful in demonstration, we now investigate their strengths and weaknesses in a controlled setting. In this paper, we present two experiments that compare an interaction technique we developed for object selection based on a where a person is looking with the most commonly used selection method using a mouse. We find that our eye gaze interaction technique is faster than selection with a mouse. The results show that our algorithm, which makes use of knowledge about how the eyes behave, preserves the natural quickness of the eye. Eye gaze interaction is a reasonable addition to computer interaction and is convenient in situations where it is important to use the hands for other tasks. It is particularly beneficial for the larger screen workspaces and virtual environments of the future, and it will become increasingly practical as eye tracker technology matures.
---
paper_title: Eye-gaze tracking system based on particle swarm optimization and BP neural network

paper_content:

In order to enhance the practicability and accuracy of the eye-gaze tracking system, a new type low pixel eye feature point location method is adopted. This method can accurately extract the eye-gaze features, namely iris centre point and canthus points when the image pickup requirements are low. The eye-gaze tracking method based on particle swarm optimization (PSO) BP neural network is raised, to capture pictures of eyes under the same environment, and a regression model where the connection weights and threshold values are optimized by PSO algorithm is built via BP network. This method is free of the inherent defects of BP network. This method requires only a common camera and normal illumination intensity rather than high-standard hardware, which greatly cuts the restrictive requirements for the system hardware and thus enhances the system practicability. The experiment results show that PSO-BP model is of higher robustness and accuracy than BP model, and is of higher recognition rate and can effectively enhances the eye-gaze tracking accuracy.
---
paper_title: Manual and gaze input cascaded (MAGIC) pointing

paper_content:

This work explores a new direction in utilizing eye gaze forcomputer input. Gaze tracking has long been considered as analternative or potentially superior pointing method for computerinput. We believe that many fundamental limitations exist withtraditional gaze pointing. In particular, it is unnatural tooverload a perceptual channel such as vision with a motor controltask. We therefore propose an alternative approach, dubbed MAGIC(Manual And Gaze Input Cascaded) pointing. With such an approach,pointing appears to the user to be a manual task, used for finemanipulation and selection. However, a large portion of the cursormovement is eliminated by warping the cursor to the eye gaze area,which encompasses the target. Two specific MAGIC pointingtechniques, one conservative and one liberal, were designed,analyzed, and implemented with an eye tracker we developed. Theywere then tested in a pilot study. This early- stage explorationshowed that the MAGIC pointing techniques might offer manyadvantages, including reduced physical effort and fatigue ascompared to traditional manual pointing, greater accuracy andnaturalness than traditional gaze pointing, and possibly fasterspeed than manual pointing. The pros and cons of the two techniquesare discussed in light of both performance data and subjectivereports.
---
paper_title: Increasing the security of gaze-based cued-recall graphical passwords using saliency masks

paper_content:

With computers being used ever more ubiquitously in situations where privacy is important, secure user authentication is a central requirement. Gaze-based graphical passwords are a particularly promising means for shoulder-surfing-resistant authentication, but selecting secure passwords remains challenging. In this paper, we present a novel gaze-based authentication scheme that makes use of cued-recall graphical passwords on a single image. In order to increase password security, our approach uses a computational model of visual attention to mask those areas of the image that are most likely to attract visual attention. We create a realistic threat model for attacks that may occur in public settings, such as filming the user's interaction while drawing money from an ATM. Based on a 12-participant user study, we show that our approach is significantly more secure than a standard image-based authentication and gaze-based 4-digit PIN entry.
---
paper_title: Head and gaze dynamics in visual attention and context learning

paper_content:

Future intelligent environments and systems may need to interact with humans while simultaneously analyzing events and critical situations. Assistive living, advanced driver assistance systems, and intelligent command-and-control centers are just a few of these cases where human interactions play a critical role in situation analysis. In particular, the behavior or body language of the human subject may be a strong indicator of the context of the situation. In this paper we demonstrate how the interaction of a human observer's head pose and eye gaze behaviors can provide significant insight into the context of the event. Such semantic data derived from human behaviors can be used to help interpret and recognize an ongoing event. We present examples from driving and intelligent meeting rooms to support these conclusions, and demonstrate how to use these techniques to improve contextual learning.
---
paper_title: Using Eye Gaze Patterns to Identify User Tasks

paper_content:

Users of today’s desktop interface often suffer from interruption overload. Our research seeks to develop an attention manager that mitigates the disruptive effects of interruptions by identifying moments of low mental workload in a user’s task sequence. To develop such a system, however, we need effective mechanisms to identify user tasks in real-time. In this paper, we show how eye gaze patterns may be used to identify user tasks. We also show that gaze patterns can indicate usability issues of an interface as well as the mental workload that the interface induces on a user. Our results can help inform the design of an attention manager and may lead to new methods to evaluate user interfaces.
---
paper_title: Robust Eye and Pupil Detection Method for Gaze Tracking

paper_content:

Robust and accurate pupil detection is a prerequisite for gaze detection. Hence, we propose a new eye/pupil detection method for gaze detection on a large display. The novelty of our research can be summarized by the following four points. First, in order to overcome the performance limitations of conventional methods of eye detection, such as adaptive boosting (Adaboost) and continuously adaptive mean shift (CAMShift) algorithms, we propose adaptive selection of the Adaboost and CAMShift methods. Second, this adaptive selection is based on two parameters: pixel differences in successive images and matching values determined by CAMShift. Third, a support vector machine (SVM)-based classifier is used with these two parameters as the input, which improves the eye detection performance. Fourth, the center of the pupil within the detected eye region is accurately located by means of circular edge detection, binarization and calculation of the geometric center. The experimental results show that the proposed method can detect the center of the pupil at a speed of approximately 19.4 frames/s with an RMS error of approximately 5.75 pixels, which is superior to the performance of conventional detection methods.
---
paper_title: Gaze tracking system at a distance for controlling IPTV

paper_content:

Gaze tracking is used for detecting the position that a user is looking at. In this research, a new gaze-tracking system and method are proposed to control a large-screen TV at a distance. This research is novel in the following four ways as compared to previous work: First, this is the first system for gaze tracking on a large-screen TV at a distance. Second, in order to increase convenience, the user's eye is captured by a remote gaze-tracking camera not requiring a user to wear any device. Third, without the complicated calibrations among the screen, the camera and the eye coordinates, the gaze position on the TV screen is obtained by using a simple 2D method based on a geometric transform with pupil center and four cornea specular reflections. Fourth, by using a near-infrared (NIR) passing filter on the camera and NIR illuminators, the pupil region becomes distinctive in the input image irrespective of the change in the environmental visible light. Experimental results showed that the proposed system could be used as a new interface for controlling a TV with a 60-inch-wide screen (16:9).
---
paper_title: Driver gaze tracker using deformable template matching

paper_content:

In driver assistance system, human eye gaze direction is an important feature described some driver's situation such as distraction and fatigue. This paper proposes a method to track driver's gaze direction by using deformable template matching. The method is divided into three steps: first, identifying the face area. Second, localizing the eye area. Finally, combining the eye region model and sight algorithm to determine the driver's gaze direction. Experimental results show that this method can effectively identify human's eye and track gaze motion in a real-time running basis.
---
paper_title: A High Speed Eye Tracking System with Robust Pupil Center Estimation Algorithm

paper_content:

This paper presents a new high-speed head- mounted binocular eye position measurement system using infrared lighting and image processing technology. Current eye tracking systems either run on-line at a lower speed, do the processing off-line, or use dedicated hardware to reach high on-line processing rates. The eye position measurement system we developed only uses a general-purpose computing system and off-the-shelf imaging devices. The binocular system can provide on-line horizontal and vertical measurement at a speed of 150 Hz on a desktop system with a 3 GHz Pentium processor. We report a novel dual-mode software system and a two-step processing algorithm to increase the processing rate. A symmetric mass center algorithm is designed to provide more accurate measurements when the pupil area is partially occluded. Using synthetic eye images, we show that our algorithm performs consistently better than some of the widely used algorithms in industry.
---
paper_title: Eye movement driven head-mounted camera: it looks where the eyes look

paper_content:

A first proof of concept was developed for a head-mounted video camera system that is continuously aligned with the user's orientation of gaze. Eye movements are tracked by video-oculography and used as signals to drive servo motors that rotate the camera. Thus, the sensorimotor output of a biological system for the control of eye movements - evolved over millions of years s used to move an artificial eye. All the capabilities of multi-sensory processing for eye, head, and surround motions are detected by the vestibular, visual, and somatosensory systems and used to drive a technical camera system. A camera guided in this way mimics the natural exploration of a visual scene and acquires video sequences from the perspective of a mobile user, while the vestibulo-ocular reflex naturally stabilizes the "gaze-in-space" of the camera during head movements and locomotion. Various applications in health care, industry, and research are conceivable.
---
paper_title: Exploring natural eye-gaze-based interaction for immersive virtual reality

paper_content:

Eye tracking technology in a head-mounted display has undergone rapid advancement in recent years, making it possible for researchers to explore new interaction techniques using natural eye movements. This paper explores three novel eye-gaze-based interaction techniques: (1) Duo-Reticles, eye-gaze selection based on eye-gaze and inertial reticles, (2) Radial Pursuit, cluttered-object selection that takes advantage of smooth pursuit, and (3) Nod and Roll, head-gesture-based interaction based on the vestibulo-ocular reflex. In an initial user study, we compare each technique against a baseline condition in a scenario that demonstrates its strengths and weaknesses.
---
paper_title: Eye Tracking by Template Matching using an Automatic Codebook Generation Scheme

paper_content:

We present an eye tracking algorithm which is robust against variations in scale, orientation and changes of eye appearances, such as eye blinking. The locations of the eye regions in the different frames are found using template matching. The method is kept invariant for rotations and scale by exploiting temporal information and by using a codebook of eye templates the method is robust against changes in eye region appearances. The entries of the codebook are generated automatically during the tracking of the eye regions and eventually represent a distinct set of eye appearances. Classification into different eye gestures, like blinking or different gaze directions, seems possible using these automatically learned patterns.
---
paper_title: Binocular eye-tracking for the control of a 3D immersive multimedia user interface

paper_content:

In this paper, we present an innovative approach to design a gazecontrolled Multimedia User Interface for modern, immersive headsets. The wide-spread availability of consumer grade Virtual Reality Head Mounted Displays such as the Oculus RiftTM transformed VR to a commodity available for everyday use. However, Virtual Environments require new paradigms of User Interfaces, since standard 2D interfaces are designed to be viewed from a static vantage point only, e.g. the computer screen. Additionally, traditional input methods such as the keyboard and mouse are hard to manipulate when the user wears a Head Mounted Display. We present a 3D Multimedia User Interface based on eye-tracking and develop six applications which cover commonly operated actions of everyday computing such as mail composing and multimedia viewing. We perform a user study to evaluate our system by acquiring both quantitative and qualitative data. The study indicated that users make less type errors while operating the eye-controlled interface compared to using the standard keyboard during immersive viewing. Subjects stated that they enjoyed the eye-tracking 3D interface more than the keyboard/mouse combination.
---
paper_title: Robust Head Mounted Wearable Eye Tracking System for Dynamical Calibration

paper_content:

In this work, a new head mounted eye tracking system is presented. Based on computer vision techniques, the system integrates eye images and head movement, in real time, performing a robust gaze point tracking. Nystagmus movements due to vestibulo-ocular reflex are monitored and integrated. The system proposed here is a strongly improved version of a previous platform called HATCAM, which was robust against changes of illumination conditions. The new version, called HAT-Move, is equipped with accurate inertial motion unit to detect the head movement enabling eye gaze even in dynamical conditions. HAT-Move performance is investigated in a group of healthy subjects in both static and dynamic conditions, i.e. when head is kept still or free to move. Evaluation was performed in terms of amplitude of the angular error between the real coordinates of the fixed points and those computed by the system in two experimental setups, specifically, in laboratory settings and in a 3D virtual reality (VR) scenario. The achieved results showed that HAT-Move is able to achieve eye gaze angular error of about 1 degree along both horizontal and vertical directions
---
paper_title: Head-mounted binocular gaze detection for selective visual recognition systems

paper_content:

Abstract For an effective and efficient visual recognition system, the region-of-interest extraction of users is one of most important image processing duties. Generally, the position of the pupil in an eyeball is directly related to the user's interest in an input visual image. However, when using a monocular eye tracking system, it is not easy to discern an accurate position of the region-of-interest of the user in a three-dimensional space. In this paper, an eye tracking system based on binocular gaze detection is presented. The proposed system is designed as a wearable device with three mini-cameras and two hot-mirrors for users to see through. The two eye monitoring system acquires the pupil images of both eyes through the hot-mirrors illuminated by infrared LEDs; the front-view camera acquires the image visible to the user. The experiment results show that the proposed system improves upon the region-of-interest localization error of the users in both the horizontal and vertical directions for the front-view images acquired by the user. Through the proposed system, it became possible to accurately extract the user's region-of-interest, and so can be used to improve the image processing speed through focused information processing for the region-of-interest and narrow the selective information acquisition needed for users.
---
paper_title: 3D gaze tracking method using Purkinje images on eye optical model and pupil

paper_content:

Gaze tracking is to detect the position a user is looking at. Most research on gaze estimation has focused on calculating the X, Y gaze position on a 2D plane. However, as the importance of stereoscopic displays and 3D applications has increased greatly, research into 3D gaze estimation of not only the X, Y gaze position, but also the Z gaze position has gained attention for the development of next-generation interfaces. In this paper, we propose a new method for estimating the 3D gaze position based on the illuminative reflections (Purkinje images) on the surface of the cornea and lens by considering the 3D optical structure of the human eye model. This research is novel in the following four ways compared with previous work. First, we theoretically analyze the generated models of Purkinje images based on the 3D human eye model for 3D gaze estimation. Second, the relative positions of the first and fourth Purkinje images to the pupil center, inter-distance between these two Purkinje images, and pupil size are used as the features for calculating the Z gaze position. The pupil size is used on the basis of the fact that pupil accommodation happens according to the gaze positions in the Z direction. Third, with these features as inputs, the final Z gaze position is calculated using a multi-layered perceptron (MLP). Fourth, the X, Y gaze position on the 2D plane is calculated by the position of the pupil center based on a geometric transform considering the calculated Z gaze position. Experimental results showed that the average errors of the 3D gaze estimation were about 0.961 (0.48 cm) on the X-axis, 1.601 (0.77 cm) on the Y-axis, and 4.59 cm along the Z-axis in 3D space. & 2011 Elsevier Ltd. All rights reserved.
---
paper_title: Estimating 3-D Point-of-Regard in a Real Environment Using a Head-Mounted Eye-Tracking System

paper_content:

Unlike conventional portable eye-tracking methods that estimate the position of the mounted camera using 2-D image coordinates, the techniques that are proposed here present richer information about person's gaze when moving over a wide area. They also include visualizing scanpaths when the user with a head-mounted device makes natural head movements. We employ a Visual SLAM technique to estimate the head pose and extract environmental information. When the person's head moves, the proposed method obtains a 3-D point-of-regard. Furthermore, scanpaths can be appropriately overlaid on image sequences to support quantitative analysis. Additionally, a 3-D environment is employed to detect objects of focus and to visualize an attention map.
---
paper_title: Wearable Reading Assist System: Augmented Reality Document Combining Document Retrieval and Eye Tracking

paper_content:

We present a new system that assists people's reading activity by combining a wearable eye tracker, a see-through head mounted display, and an image based document retrieval engine. An image based document retrieval engine is used for identification of the reading document, whereas an eye tracker is used to detect which part of the document the reader is currently reading. The reader can refer to the glossary of the latest viewed key word by looking at the see-through head mounted display. This novel document reading assist application, which is the integration of a document retrieval system into an everyday reading scenario for the first time, enriches people's reading life. In this paper, we i) investigate the performance of the state-of-the-art image based document retrieval method using a wearable camera, ii) propose a method for identification of the word the reader is attendant, and iii) conduct pilot studies for evaluation of the system in this reading context. The results show the potential of a document retrieval system in combination with a gaze based user-oriented system.
---
paper_title: Design and implementation of an interactive HMD for wearable AR system

paper_content:

In this work, we present an interactive optical see-through HMD (head-mounted device) which makes use of a user's gaze information for the interaction in the AR (augmented reality) environment. In particular, we propose a method to employ a user's half-blink information for more efficient interaction. As the interaction is achieved by using a user's eye gaze and half-blink information, the proposed system can provide more efficient computing environment. In addition, the proposed system can be quite helpful to those who have difficulties in using conventional interaction methods which use hands or feet. The experimental results present the robustness and efficiency of the proposed system.
---
paper_title: Demo of FaceVR: real-time facial reenactment and eye gaze control in virtual reality

paper_content:

We introduce FaceVR, a novel method for gaze-aware facial reenactment in the Virtual Reality (VR) context. The key component of FaceVR is a robust algorithm to perform real-time facial motion capture of an actor who is wearing a head-mounted display (HMD), as well as a new data-driven approach for eye tracking from monocular videos. In addition to these face reconstruction components, FaceVR incorporates photo-realistic re-rendering in real time, thus allowing artificial modifications of face and eye appearances. For instance, we can alter facial expressions, change gaze directions, or remove the VR goggles in realistic re-renderings. In a live setup with a source and a target actor, we apply these newly-introduced algorithmic components. We assume that the source actor is wearing a VR device, and we capture his facial expressions and eye movement in real-time. For the target video, we mimic a similar tracking process; however, we use the source input to drive the animations of the target video, thus enabling gaze-aware facial reenactment. To render the modified target video on a stereo display, we augment our capture and reconstruction process with stereo data. In the end, FaceVR produces compelling results for a variety of applications, such as gaze-aware facial reenactment, reenactment in virtual reality, removal of VR goggles, and re-targeting of somebody's gaze direction in a video conferencing call.
---
paper_title: openEyes: a low-cost head-mounted eye-tracking solution

paper_content:

Eye tracking has long held the promise of being a useful methodology for human computer interaction. However, a number of barriers have stood in the way of the integration of eye tracking into everyday applications, including the intrusiveness, robustness, availability, and price of eye-tracking systems. To lower these barriers, we have developed the openEyes system. The system consists of an open-hardware design for a digital eye tracker that can be built from low-cost off-the-shelf components, and a set of open-source software tools for digital image capture, manipulation, and analysis in eye-tracking applications. We expect that the availability of this system will facilitate the development of eye-tracking applications and the eventual integration of eye tracking into the next generation of everyday human computer interfaces. We discuss the methods and technical challenges of low-cost eye tracking as well as the design decisions that produced our current system.
---
paper_title: Gaze tracking based on active appearance model and multiple support vector regression on mobile devices

paper_content:

Gaze tracking technology is a convenient interfacing method for mobile devices. Most previous studies used a large-sized desktop or head-mounted display. In this study, we propose a novel gaze tracking method using an active appearance model (AAM) and multiple support vector regression (SVR) on a mobile device. Our research has four main contributions. First, in calculating the gaze position, the amount of facial rotation and translation based on four feature values is computed using facial feature points detected by AAM. Second, the amount of eye rotation based on two feature values is computed for measuring eye gaze position. Third, to compensate for the fitting error of an AAM in facial rotation, we use the adaptive discrete Kalman filter (DKF), which applies a different velocity of state transition matrix to the facial feature points. Fourth, we obtain gaze position on a mobile device based on multiple SVR by separating the rotation and translation of face and eye rotation. Experimental results show that the root mean square (rms) gaze error is 36.94 pixels on the 4.5-in. screen of a mobile device with a screen resolution of 800×600 pixels.
---
paper_title: Ultra-low-cost 3D gaze estimation: an intuitive high information throughput compliment to direct brain–machine interfaces

paper_content:

Eye movements are highly correlated with motor intentions and are often retained by patients with serious motor deficiencies. Despite this, eye tracking is not widely used as control interface for movement in impaired patients due to poor signal interpretation and lack of control flexibility. We propose that tracking the gaze position in 3D rather than 2D provides a considerably richer signal for human machine interfaces by allowing direct interaction with the environment rather than via computer displays. We demonstrate here that by using mass-produced video-game hardware, it is possible to produce an ultra-low-cost binocular eye-tracker with comparable performance to commercial systems, yet 800 times cheaper. Our head-mounted system has 30 USD material costs and operates at over 120 Hz sampling rate with a 0.5‐1 degree of visual angle resolution. We perform 2D and 3D gaze estimation, controlling a real-time volumetric cursor essential for driving complex user interfaces. Our approach yields an information throughput of 43 bits s −1 , more than ten times that of invasive and semi-invasive brain‐machine interfaces (BMIs) that are vastly more expensive. Unlike many BMIs our system yields effective real-time closed loop control of devices (10 ms latency), after just ten minutes of training, which we demonstrate through a novel BMI benchmark—the control of the video arcade game ‘Pong’. (Some figures may appear in colour only in the online journal)
---
paper_title: Gaze estimation based on head movements in virtual reality applications using deep learning

paper_content:

Gaze detection in Virtual Reality systems is mostly performed using eye-tracking devices. The coordinates of the sight, as well as other data regarding the eyes, are used as input values for the applications. While this trend is becoming more and more popular in the interaction design of immersive systems, most visors do not come with an embedded eye-tracker, especially those that are low cost and maybe based on mobile phones. We suggest implementing an innovative gaze estimation system into virtual environments as a source of information regarding users intentions. We propose a solution based on a combination of the features of the images and the movement of the head as an input of a Deep Convolutional Neural Network capable of inferring the 2D gaze coordinates in the imaging plane.
---
paper_title: Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras

paper_content:

Properly calibrating an optical see-through head-mounted display (OST-HMD) and maintaining a consistent calibration over time can be a very challenging task. Automated methods need an accurate model of both the OST-HMD screen and the user's constantly changing eye-position to correctly project virtual information. While some automated methods exist, they often have restrictions, including fixed eye-cameras that cannot be adjusted for different users.To address this problem, we have developed a method that automatically determines the position of an adjustable eye-tracking camera and its unconstrained position relative to the display. Unlike methods that require a fixed pose between the HMD and eye camera, our framework allows for automatic calibration even after adjustments of the camera to a particular individual's eye and even after the HMD moves on the user's face. Using two sets of IR-LEDs rigidly attached to the camera and OST-HMD frame, we can calculate the correct projection for different eye positions in real time and changes in HMD position within several frames. To verify the accuracy of our method, we conducted two experiments with a commercial HMD by calibrating a number of different eye and camera positions. Ground truth was measured through markers on both the camera and HMD screens, and we achieve a viewing accuracy of 1.66 degrees for the eyes of 5 different experiment participants.
---
paper_title: ModulAR: Eye-Controlled Vision Augmentations for Head Mounted Displays

paper_content:

In the last few years, the advancement of head mounted display technology and optics has opened up many new possibilities for the field of Augmented Reality. However, many commercial and prototype systems often have a single display modality, fixed field of view, or inflexible form factor. In this paper, we introduce Modular Augmented Reality (ModulAR), a hardware and software framework designed to improve flexibility and hands-free control of video see-through augmented reality displays and augmentative functionality. To accomplish this goal, we introduce the use of integrated eye tracking for on-demand control of vision augmentations such as optical zoom or field of view expansion. Physical modification of the device's configuration can be accomplished on the fly using interchangeable camera-lens modules that provide different types of vision enhancements. We implement and test functionality for several primary configurations using telescopic and fisheye camera-lens systems, though many other customizations are possible. We also implement a number of eye-based interactions in order to engage and control the vision augmentations in real time, and explore different methods for merging streams of augmented vision into the user's normal field of view. In a series of experiments, we conduct an in depth analysis of visual acuity and head and eye movement during search and recognition tasks. Results show that methods with larger field of view that utilize binary on/off and gradual zoom mechanisms outperform snapshot and sub-windowed methods and that type of eye engagement has little effect on performance.
---
paper_title: Real-time nonintrusive monitoring and prediction of driver fatigue

paper_content:

This paper describes a real-time online prototype driver-fatigue monitor. It uses remotely located charge-coupled-device cameras equipped with active infrared illuminators to acquire video images of the driver. Various visual cues that typically characterize the level of alertness of a person are extracted in real time and systematically combined to infer the fatigue level of the driver. The visual cues employed characterize eyelid movement, gaze movement, head movement, and facial expression. A probabilistic model is developed to model human fatigue and to predict fatigue based on the visual cues obtained. The simultaneous use of multiple visual cues and their systematic combination yields a much more robust and accurate fatigue characterization than using a single visual cue. This system was validated under real-life fatigue conditions with human subjects of different ethnic backgrounds, genders, and ages; with/without glasses; and under different illumination conditions. It was found to be reasonably robust, reliable, and accurate in fatigue characterization.
---
paper_title: Driver fatigue alarm based on eye detection and gaze estimation

paper_content:

The driver assistant system has attracted much attention as an essential component of intelligent transportation systems. One task of driver assistant system is to prevent the drivers from fatigue. For the fatigue detection it is natural that the information about eyes should be utilized. The driver fatigue can be divided into two types, one is the sleep with eyes close and another is the sleep with eyes open. Considering that the fatigue detection is related with the prior knowledge and probabilistic statistics, the dynamic Bayesian network is used as the analysis tool to perform the reasoning of fatigue. Two kinds of experiments are performed to verify the system effectiveness, one is based on the video got from the laboratory and another is based on the video got from the real driving situation. Ten persons participate in the test and the experimental result is that, in the laboratory all the fatigue events can be detected, and in the practical vehicle the detection ratio is about 85%. Experiments show that in most of situations the proposed system works and the corresponding performance is satisfying.
---
paper_title: Analyzing the relationship between head pose and gaze to model driver visual attention

paper_content:

Monitoring driver behavior is crucial in the design of advanced driver assistance systems (ADAS) that can detect driver actions, providing necessary warnings when not attentive to driving tasks. The visual attention of a driver is an important aspect to consider, as most driving tasks require visual resources. Previous work has investigated algorithms to detect driver visual attention by tracking the head or eye movement. While tracking pupil can give an accurate direction of visual attention, estimating gaze on vehicle environment is a challenging problem due to changes in illumination, head rotations, and occlusions (e.g. hand, glasses). Instead, this paper investigates the use of the head pose as a coarse estimate of the driver visual attention. The key challenge is the non-trivial relation between head and eye movements while glancing to a target object, which depends on the driver, the underlying cognitive and visual demand, and the environment. First, we evaluate the performance of a state-of-the-art head pose detection algorithm over natural driving recordings, which are compared with ground truth estimations derived from AprilTags attached to a headband. Then, the study proposes regression models to estimate the drivers' gaze based on the head position and orientation, which are built with data from natural driving recordings. The proposed system achieves high accuracy over the horizontal direction, but moderate/low performance over the vertical direction. We compare results while our participants were driving, and when the vehicle was parked.
---
paper_title: Driver Gaze Tracking and Eyes Off the Road Detection System

paper_content:

Distracted driving is one of the main causes of vehicle collisions in the United States. Passively monitoring a driver's activities constitutes the basis of an automobile safety system that can potentially reduce the number of accidents by estimating the driver's focus of attention. This paper proposes an inexpensive vision-based system to accurately detect Eyes Off the Road (EOR). The system has three main components: 1) robust facial feature tracking; 2) head pose and gaze estimation; and 3) 3-D geometric reasoning to detect EOR. From the video stream of a camera installed on the steering wheel column, our system tracks facial features from the driver's face. Using the tracked landmarks and a 3-D face model, the system computes head pose and gaze direction. The head pose estimation algorithm is robust to nonrigid face deformations due to changes in expressions. Finally, using a 3-D geometric analysis, the system reliably detects EOR.
---
paper_title: Real-time eye detection and tracking for driver observation under various light conditions

paper_content:

Eye tracking is one of the key technologies for future driver assistance systems since human eyes contain much information about the driver's condition such as gaze, attention level, and fatigue level. Thus, nonintrusive methods for eye detection and tracking are important for many applications of vision-based driver-automotive interaction. One problem common to many eye tracking methods proposed so far is their sensitivity to lighting condition change. This tends to significantly limit their scope for automotive applications. In this paper we present a new realtime eye detection and tracking method that works under variable and realistic lighting conditions. By combining imaging by using IR light and appearance-based object recognition techniques, our method can robustly track eyes even when the pupils are not very bright due to significant external illumination interferences. The appearance model is incorporated in both eye detection and tracking via the use of a support vector machine and mean shift tracking. Our experimental results show the feasibility of our approach and the validity for the method is extended for drivers wearing sunglasses.
---
paper_title: Recognition of a Driver's Gaze for Vehicle Headlamp Control

paper_content:

In this paper, we propose a novel method for gaze recognition of a driver coping with rotation of a driver's face. Frontal face images and left half profile images were separately trained using the Viola-Jones (V-J) algorithm to produce classifiers that can detect faces. The right half profile can be detected by mirroring the entire image when neither a frontal face nor a left half profile was detected. As an initial step, this method was used to simultaneously detect the driver's face. Then, we applied a regressional version of linear discriminant analysis (LDAr) to the detected facial region to extract important features for classification. Finally, these features were used to classify the driver's gaze in seven directions. In the feature extraction step, LDAr tries to find features that maximize the ratio of interdistances among samples with large differences in the target value to those with small differences in the target value. Therefore, the resultant features are more fitted to regression problems than conventional feature extraction methods. In addition to LDAr, in this paper, a 2-D extension of LDAr is also developed and used as a feature extraction method for gaze recognition. The experimental results show that the proposed method achieves a good gaze recognition rate under various rotation angles of a driver's head, resulting in a reliable headlamp control performance.
---
paper_title: Real-time system for monitoring driver vigilance

paper_content:

In this paper we present a non-intrusive prototype computer vision system for real-time monitoring driver's vigilance. It is based on a hardware system, for real time acquisition of driver's images using an active IR illuminator, and their software implementation for monitoring some visual behaviors that characterize a driver's level of vigilance. These are the eyelid movements and the pose face. The system has been tested with different sequences recorded on night and day driving conditions in a motorway and with different users. We show some experimental results and some conclusions about the performance of the system.
---
paper_title: Eye-Gaze Tracking Method Driven by Raspberry PI Applicable in Automotive Traffic Safety

paper_content:

This paper comes as a response to the fact that, lately, more and more accidents are caused by people who fall asleep at the wheel. Eye tracking is one of the most important aspects in driver assistance systems since human eyes hold much in-formation regarding the driver's state, like attention level, gaze and fatigue level. The number of times the subject blinks will be taken into account for identification of the subject's drowsiness. Also the direction of where the user is looking will be estimated according to the location of the user's eye gaze. The developed algorithm was implemented on a Raspberry Pi board in order to create a portable system. The main determination of this project is to conceive an active eyetracking based system, which focuses on the drowsiness detection amongst fatigue related deficiencies in driving.
---
paper_title: Real-Time Gaze Estimator Based on Driver's Head Orientation for Forward Collision Warning System

paper_content:

This paper presents a vision-based real-time gaze zone estimator based on a driver's head orientation composed of yaw and pitch. Generally, vision-based methods are vulnerable to the wearing of eyeglasses and image variations between day and night. The proposed method is novel in the following four ways: First, the proposed method can work under both day and night conditions and is robust to facial image variation caused by eyeglasses because it only requires simple facial features and not specific features such as eyes, lip corners, and facial contours. Second, an ellipsoidal face model is proposed instead of a cylindrical face model to exactly determine a driver's yaw. Third, we propose new features-the normalized mean and the standard deviation of the horizontal edge projection histogram-to reliably and rapidly estimate a driver's pitch. Fourth, the proposed method obtains an accurate gaze zone by using a support vector machine. Experimental results from 200 000 images showed that the root mean square errors of the estimated yaw and pitch angles are below 7 under both daylight and nighttime conditions. Equivalent results were obtained for drivers with glasses or sunglasses, and 18 gaze zones were accurately estimated using the proposed gaze estimation method.
---
paper_title: Robust and continuous estimation of driver gaze zone by dynamic analysis of multiple face videos

paper_content:

Analysis of driver's head behavior is an integral part of driver monitoring system. Driver's coarse gaze direction or gaze zone is a very important cue in understanding driver- state. Many existing gaze zone estimators are, however, limited to single camera perspectives, which are vulnerable to occlu- sions of facial features from spatially large head movements away from the frontal pose. Non-frontal glances away from the driving direction, though, are of special interest as interesting events, critical to driver safety, occur during those times. In this paper, we present a distributed camera framework for gaze zone estimation using head pose dynamics to operate robustly and continuously even during large head movements. For experimental evaluations, we collected a dataset from naturalistic on-road driving in urban streets and freeways. A human expert provided the gaze zone ground truth using all vision information including eyes and surround context. Our emphasis is to understand the efficacy of the head pose dynamic information in predicting eye-gaze-based zone ground truth. We conducted several experiments in designing the dynamic features and compared the performance against static head pose based approach. Analyses show that dynamic information significantly improves the results.
---
paper_title: Owl and Lizard: Patterns of Head Pose and Eye Pose in Driver Gaze Classification

paper_content:

Accurate, robust, inexpensive gaze tracking in the car can help keep a driver safe by facilitating the more effective study of how to improve (1) vehicle interfaces and (2) the design of future Advanced Driver Assistance Systems. In this paper, we estimate head pose and eye pose from monocular video using methods developed extensively in prior work and ask two new interesting questions. First, how much better can we classify driver gaze using head and eye pose versus just using head pose? Second, are there individual-specific gaze strategies that strongly correlate with how much gaze classification improves with the addition of eye pose information? We answer these questions by evaluating data drawn from an on-road study of 40 drivers. The main insight of the paper is conveyed through the analogy of an "owl" and "lizard" which describes the degree to which the eyes and the head move when shifting gaze. When the head moves a lot ("owl"), not much classification improvement is attained by estimating eye pose on top of head pose. On the other hand, when the head stays still and only the eyes move ("lizard"), classification accuracy increases significantly from adding in eye pose. We characterize how that accuracy varies between people, gaze strategies, and gaze regions.
---
paper_title: Estimating Gaze Direction of Vehicle Drivers Using a Smartphone Camera

paper_content:

Many automated driver monitoring technologies have been proposed to enhance vehicle and road safety. Most existing solutions involve the use of specialized embedded hardware, primarily in high-end automobiles. This paper explores driver assistance methods that can be implemented on mobile devices such as a consumer smartphone, thus offering a level of safety enhancement that is more widely accessible. Specifically, the paper focuses on estimating driver gaze direction as an indicator of driver attention. Input video frames from a smartphone camera facing the driver are first processed through a coarse head pose direction. Next, the locations and scales of face parts, namely mouth, eyes, and nose, define a feature descriptor that is supplied to an SVM gaze classifier which outputs one of 8 common driver gaze directions. A key novel aspect is an in-situ approach for gathering training data that improves generalization performance across drivers, vehicles, smartphones, and capture geometry. Experimental results show that a high accuracy of gaze direction estimation is achieved for four scenarios with different drivers, vehicles, smartphones and camera locations.
---
paper_title: Real-Time Detection of Driver Cognitive Distraction Using Support Vector Machines

paper_content:

As use of in-vehicle information systems (IVISs) such as cell phones, navigation systems, and satellite radios has increased, driver distraction has become an important and growing safety concern. A promising way to overcome this problem is to detect driver distraction and adapt in-vehicle systems accordingly to mitigate such distractions. To realize this strategy, this paper applied support vector machines (SVMs), which is a data mining method, to develop a real-time approach for detecting cognitive distraction using drivers' eye movements and driving performance data. Data were collected in a simulator experiment in which ten participants interacted with an IVIS while driving. The data were used to train and test both SVM and logistic regression models, and three different model characteristics were investigated: how distraction was defined, which data were input to the model, and how the input data were summarized. The results show that the SVM models were able to detect driver distraction with an average accuracy of 81.1%, outperforming more traditional logistic regression models. The best performing model (96.1% accuracy) resulted when distraction was defined using experimental conditions (i.e., IVIS drive or baseline drive), the input data were comprised of eye movement and driving measures, and these data were summarized over a 40-s window with 95% overlap of windows. These results demonstrate that eye movements and simple measures of driving performance can be used to detect driver distraction in real time. Potential applications of this paper include the design of adaptive in-vehicle systems and the evaluation of driver distraction
---
paper_title: Eye-Gaze Tracking Analysis of Driver Behavior While Interacting With Navigation Systems in an Urban Area

paper_content:

With the advent of global positioning system technology, smart phones are used as portable navigation systems. Guidelines that ensure driving safety while using conventional on-board navigation systems have already been published but do not extend to portable navigation systems; therefore, this study focused on the analysis of the eye-gaze tracking of drivers interacting with portable navigation systems in an urban area. Combinations of different display sizes and positions of portable navigation systems were adopted by 20 participants in a driving simulator experiment. An expectation maximum algorithm was proposed to classify the measured eye-gaze points; furthermore, three measures of glance frequency, glance time, and total glance time as a percentage were calculated. The results indicated that the convenient display position with a small visual angle can provide a significantly shorter glance time but a significantly higher glance frequency; however, the small-size display will bring on significantly longer glance time that may result in the increasing of visual distraction for drivers. The small-size portable display received significantly lower scores for subjective evaluation of acceptability and fatigue; moreover, the small-size portable display on the conventional built-in position received significantly lower subjective evaluation scores than that of the big-size one on the upper side of the dashboard. In addition, it indicated an increased risk of rear-end collision that the proportion of time that the time-to-collision was less than 1 s was significantly shorter for the portable navigation than that of traditional on-board one.
---
paper_title: Head pose and gaze direction tracking for detecting a drowsy driver

paper_content:

This paper proposes a system that uses gaze direction tracking and head pose estimation to detect drowsiness of a driver. Head pose is estimated by calculating optic flow of the facial features, which are acquired with a corner detection algorithm. Analysis of the driver's head behavior leads to three moving components: nodding, shaking, and tilting. To track the gaze direction of the driver, we trace the center point of the pupil using CDF analysis and estimate the frequency of eye-movement.
---
paper_title: Usability evaluation of eye tracking on an unmodified common tablet

paper_content:

This paper describes the design, implementation, and usability evaluation of a neural network based eye tracking system on an unmodified common tablet and discusses the challenges and implications of neural networks as an eye tracking component on a mobile platform. We objectively and subjectively evaluate the usability and performance tradeoffs of calibration, one of the fundamental components of eye tracking. The described system obtained an average spatial accuracy of 3.95° and an average temporal resolution of 0.65 Hz during trials. Results indicate that an increased neural network training set may be utilized to increase spatial accuracy, at the cost of greater physical effort and fatigue.
---
paper_title: Eye gesture recognition on portable devices

paper_content:

Hand-held portable devices have received only little attention as a platform in the eye tracking community so far. This is mainly due to their -- until recently -- limited sensing capabilities and processing power. In this work-in-progress paper we present the first prototype eye gesture recognition system for portable devices that does not require any additional equipment. The system combines techniques from image processing, computer vision and pattern recognition to detect eye gestures in the video recorded using the built-in front-facing camera. In a five-participant user study we show that our prototype can recognise four different continuous eye gestures in near real-time with an average accuracy of 60% on an Android-based smartphone (17.6% false positives) and 67.3% on a laptop (5.9% false positives). This initial result is promising and underlines the potential of eye tracking and eye-based interaction on portable devices.
---
paper_title: EyePhone: activating mobile phones with your eyes

paper_content:

As smartphones evolve researchers are studying new techniques to ease the human-mobile interaction. We propose EyePhone, a novel "hand-free" interfacing system capable of driving mobile applications/functions using only the user's eyes movement and actions (e.g., wink). EyePhone tracks the user's eye movement across the phone's display using the camera mounted on the front of the phone; more specifically, machine learning algorithms are used to: i) track the eye and infer its position on the mobile phone display as a user views a particular application; and ii) detect eye blinks that emulate mouse clicks to activate the target application under view. We present a prototype implementation of EyePhone on a Nokia N810, which is capable of tracking the position of the eye on the display, mapping this positions to an application that is activated by a wink. At no time does the user have to physically touch the phone display.
---
paper_title: A system for tracking gaze on handheld devices

paper_content:

Many of the current gaze-tracking systems require that a subject’s head be stabilized and that the interface be fixed to a table. This article describes a prototype system for tracking gaze on the screen of mobile, handheld devices. The proposed system frees the user and the interface from previous constraints, allowing natural freedom of movement within the operational envelope of the system. The method is software-based, and integrates a commercial eye-tracking device (EyeLink I) with a magnetic positional tracking device (Polhemus FASTRAK). The evaluation of the system shows that it is capable of producing valid data with adequate accuracy.
---
paper_title: MobiGaze: development of a gaze interface for handheld mobile devices

paper_content:

Handheld mobile devices that have a touch screen are widely used but are awkward to use with one hand. To solve this problem, we propose MobiGaze, which is a user interface that uses one's gaze (gaze interface) to operate a handheld mobile device. By using stereo cameras, the user's line of sight is detected in 3D, enabling the user to interact with a mobile device by means of his/her gaze. We have constructed a prototype system of MobiGaze that consists of two cameras with IR-LED, a Windows-based notebook PC, and iPod touch. Moreover, we have developed several applications for MobiGaze.
---
paper_title: Integrating eye tracking and motion sensor on mobile phone for interactive 3D display

paper_content:

In this paper, we propose an eye tracking and gaze estimation system for mobile phone. We integrate an eye detector, ::: cornereye center and iso-center to improve pupil detection. The optical flow information is used for eye tracking. We ::: develop a robust eye tracking system that integrates eye detection and optical-flow based image tracking. In addition, we ::: further incorporate the orientation sensor information from the mobile phone to improve the eye tracking for accurate ::: gaze estimation. We demonstrate the accuracy of the proposed eye tracking and gaze estimation system through ::: experiments on some public video sequences as well as videos acquired directly from mobile phone.
---
paper_title: Visible-spectrum remote eye tracker for gaze communication

paper_content:

Many approaches have been proposed to create an eye tracker based on visible-spectrum. These efforts provide a possibility to create inexpensive eye tracker capable to operate outdoor. Although the resulted tracking accuracy is acceptable for a visible-spectrum head-mounted eye tracker, there are many limitations of these approaches to create a remote eye tracker. In this study, we propose a high-accuracy remote eye tracker that uses visible-spectrum imaging and several gaze communication interfaces suited to the tracker. The gaze communication interfaces are designed to assist people with motor disability. Our results show that the proposed eye tracker achieved an average accuracy of 0.77° and a frame rate of 28 fps with a personal computer. With a tablet device, the proposed eye tracker achieved an average accuracy of 0.82° and a frame rate of 25 fps. The proposed gaze communication interfaces enable users to type a complete sentence containing eleven Japanese characters in about a minute.
---
paper_title: Rapid object detection using a boosted cascade of simple features

paper_content:

This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the "integral image" which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers. The third contribution is a method for combining increasingly more complex classifiers in a "cascade" which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. The cascade can be viewed as an object specific focus-of-attention mechanism which unlike previous approaches provides statistical guarantees that discarded regions are unlikely to contain the object of interest. In the domain of face detection the system yields detection rates comparable to the best previous systems. Used in real-time applications, the detector runs at 15 frames per second without resorting to image differencing or skin color detection.
---
paper_title: Smart Tablet Monitoring by a Real-Time Head Movement and Eye Gestures Recognition System

paper_content:

Different research works are studying to integrate new ways of interaction with mobiles devices such as smartphone and tablets to provide a natural and easy mode of communication to people. In this paper we propose a new system to monitoring tablets through head motions and eye-gaze gestures recognition. This system is able to open browser application with simples' motions by the head and the eyes. For the face detection, we use a Viola and Jones technique, and for the eyes detection and tracking we used Haar classifier to the detection part and the template matching for the tracking part. We develop this system on android based tablet. Through the front-facing camera the system capture a real-time streaming video on which we implement the detection and recognition module. We test the system on 5 persons and the experiment results shows that this system is robust and it is invariant for the lightness and the moving state of the user. As known the tablet's resources are limited, in consequence the response time of our system is not satisfied for the real time condition. That is why we apply the cloud computing to solve this problem.
---
paper_title: Improving mobile device interaction by eye tracking analysis

paper_content:

This paper describes a non-intrusive eyetracking tool for mobile devices by using images acquired by the front camera of the iPhone and iPod Touch. By tracking and interpreting the user's gaze to the smartphone's screen coordinates the user can interact with the device by using a more natural and spotaneous way. The application uses a Haar classifier based detection module for identifying the eyes in the acquired images and subsequently the CAMSHIFT algorithm to find and track the eyes movement and detect the user's gaze. The performance of the proposed tool was evaluated by testing the system on 16 users and the results shown that in about 79% of the times it was able to detect correctly the users' gaze.
---
paper_title: EyeTab: model-based gaze estimation on unmodified tablet computers

paper_content:

Despite the widespread use of mobile phones and tablets, hand-held portable devices have only recently been identified as a promising platform for gaze-aware applications. Estimating gaze on portable devices is challenging given their limited computational resources, low quality integrated front-facing RGB cameras, and small screens to which gaze is mapped. In this paper we present EyeTab, a model-based approach for binocular gaze estimation that runs entirely on an unmodified tablet. EyeTab builds on set of established image processing and computer vision algorithms and adapts them for robust and near-realtime gaze estimation. A technical prototype evaluation with eight participants in a normal indoors office setting shows that EyeTab achieves an average gaze estimation accuracy of 6.88° of visual angle at 12 frames per second.
---
paper_title: Exploiting Eye Tracking for Smartphone Authentication

paper_content:

Traditional user authentication methods using passcode or finger movement on smartphones are vulnerable to shoulder surfing attack, smudge attack, and keylogger attack. These attacks are able to infer a passcode based on the information collection of user’s finger movement or tapping input. As an alternative user authentication approach, eye tracking can reduce the risk of suffering those attacks effectively because no hand input is required. However, most existing eye tracking techniques are designed for large screen devices. Many of them depend on special hardware like high resolution eye tracker and special process like calibration, which are not readily available for smartphone users. In this paper, we propose a new eye tracking method for user authentication on a smartphone. It utilizes the smartphone’s front camera to capture a user’s eye movement trajectories which are used as the input of user authentication. No special hardware or calibration process is needed. We develop a prototype and evaluate its effectiveness on an Android smartphone. We recruit a group of volunteers to participate in the user study. Our evaluation results show that the proposed eye tracking technique achieves very high accuracy in user authentication.
---
paper_title: Interacting with the Computer using Gaze Gestures

paper_content:

This paper investigates novel ways to direct computers by eye gaze. Instead of using fixations and dwell times, this work focuses on eye motion, in particular gaze gestures. Gaze gestures are insensitive to accuracy problems and immune against calibration shift. A user study indicates that users are able to perform complex gaze gestures intentionally and investigates which gestures occur unintentionally during normal interaction with the computer. Further experiments show how gaze gestures can be integrated into working with standard desktop applications and controlling media devices.
---
paper_title: New Solution to the Midas Touch Problem: Identification of Visual Commands Via Extraction of Focal Fixations

paper_content:

Abstract Reliable identification of intentional visual commands is a major problem in the development of eye-movements based user interfaces. This work suggests that the presence of focal visual fixations is indicative of visual commands. Two experiments are described which assessed the effectiveness of this approach in a simple gaze-control interface. Identification accuracy was shown to match that of the commonly used dwell time method. Using focal fixations led to less visual fatigue and higher speed of work. Perspectives of using focal fixations for identification of visual commands in various kinds of eye-movements based interfaces are discussed.
---
paper_title: Towards the development of a standardized performance evaluation framework for eye gaze estimation systems in consumer platforms

paper_content:

There is a need to standardize the performance of eye gaze estimation (EGE) methods in various platforms for human computer interaction (HCI). Because of lack of consistent schemes or protocols for summative evaluation of EGE systems, performance results in this field can neither be compared nor reproduced with any consistency. In contemporary literature, gaze tracking accuracy is measured under non-identical sets of conditions, with variable metrics and most results do not report the impact of system meta-parameters that significantly affect tracking performances. In this work, the diverse nature of these research outcomes and system parameters which affect gaze tracking in different platforms is investigated and their error contributions are estimated quantitatively. Then the concept and development of a performance evaluation framework is proposed- that can define design criteria and benchmark quality measures for the eye gaze research community.
---
paper_title: New Solution to the Midas Touch Problem: Identification of Visual Commands Via Extraction of Focal Fixations

paper_content:

Abstract Reliable identification of intentional visual commands is a major problem in the development of eye-movements based user interfaces. This work suggests that the presence of focal visual fixations is indicative of visual commands. Two experiments are described which assessed the effectiveness of this approach in a simple gaze-control interface. Identification accuracy was shown to match that of the commonly used dwell time method. Using focal fixations led to less visual fatigue and higher speed of work. Perspectives of using focal fixations for identification of visual commands in various kinds of eye-movements based interfaces are discussed.
---
paper_title: Eye and gaze tracking for interactive graphic display

paper_content:

This paper describes a computer vision system based on active IR illumination for real-time gaze tracking for interactive graphic display. Unlike most of the existing gaze tracking techniques, which often require assuming a static head to work well and require a cumbersome calibration process for each person, our gaze tracker can perform robust and accurate gaze estimation without calibration and under rather significant head movement. This is made possible by a new gaze calibration procedure that identifies the mapping from pupil parameters to screen coordinates using generalized regression neural networks (GRNNs). With GRNNs, the mapping does not have to be an analytical function and head movement is explicitly accounted for by the gaze mapping function. Furthermore, the mapping function can generalize to other individuals not used in the training. To further improve the gaze estimation accuracy, we employ a hierarchical classification scheme that deals with the classes that tend to be misclassified. This leads to a 10% improvement in classification error. The angular gaze accuracy is about 5° horizontally and 8° vertically. The effectiveness of our gaze tracker is demonstrated by experiments that involve gaze-contingent interactive graphic display.
---
paper_title: Robust remote gaze estimation method based on multiple geometric transforms

paper_content:

The remote gaze estimation (RGE) technique has been widely used as a natural interface in consumer electronic devices for decades. Although outstanding outcomes on RGE have been recently reported in the literature, tracking gaze under large head movements is still an unsolved problem. General RGE methods estimate a user’s point of gaze (POG) using a mapping function representing the relationship between several infrared light sources and their corresponding corneal reflections (CRs) in the eye image. However, the minimum number of available CRs required for a valid POG estimation cannot be satisfied in those methods because the CRs often tend to be distorted or disappeared inevitably under the unconstrained eye and head movements. To overcome this problem, a multiple-transform-based method is proposed. In the proposed method, through three different geometric transform-based normalization processes, several nonlinear mapping functions are simultaneously obtained in the calibration process and then used to estimate the POG. The geometric transforms and mapping functions can be alternatively employed according to the number of available CRs even under large head movement. Experimental results on six subjects demonstrate the effectiveness of the proposed method.
---
paper_title: Implicit Calibration of a Remote Gaze Tracker

paper_content:

We describe a system designed to monitor the gaze of a user working naturally at a computer workstation. The system consists of three cameras situated between the keyboard and the monitor. Free head movements are allowed within a three-dimensional volume approximately 40 centimeters in diameter. Two fixed, wide-field "face" cameras equipped with active-illumination systems enable rapid localization of the subject's pupils. A third steerable "eye" camera has a relatively narrow field of view, and acquires the images of the eyes which are used for gaze estimation. Unlike previous approaches which construct an explicit three-dimensional representation of the subject's head and eye, we derive mappings for steering control and gaze estimation using a procedure we call implicit calibration. Implicit calibration is performed by collecting a "training set" of parameters and associated measurements, and solving for a set of coefficients relating the measurements back to the parameters of interest. Preliminary data on three subjects indicate an median gaze estimation error of ap-proximately 0.8 degree.
---
paper_title: Free head motion eye gaze tracking using a single camera and multiple light sources

paper_content:

One of the main limitations of current remote eye gaze tracking (REGT) techniques is that the user's head must remain within a very limited area in front of the monitor screen. In this paper, we present a free head motion REGT technique. By projecting a known rectangular pattern of lights, the technique estimates the gaze position relative to this rectangle using an invariant property of projective geometry. We carry extensive analysis of similar methods using an eye model to compare their accuracy. Based on these results, we propose a new estimation procedure that compensates the angular difference between the eye visual axis and optical axis. We have developed a real time (30 fps) prototype using a single camera and 5 light sources to generate the light pattern. Experimental results shows that the accuracy of the system is about 1deg of visual angle
---
paper_title: A single-camera remote eye tracker

paper_content:

Many eye-tracking systems either require the user to keep their head still or involve cameras or other equipment mounted on the user's head. While acceptable for research applications, these limitations make the systems unsatisfactory for prolonged use in interactive applications. Since the goal of our work is to use eye trackers for improved visual communication through gaze guidance [1,2] and for Augmentative and Alternative Communication (AAC) [3], we are interested in less invasive eye tracking techniques.
---
paper_title: Adaptive Linear Regression for Appearance-Based Gaze Estimation

paper_content:

We investigate the appearance-based gaze estimation problem, with respect to its essential difficulty in reducing the number of required training samples, and other practical issues such as slight head motion, image resolution variation, and eye blinking. We cast the problem as mapping high-dimensional eye image features to low-dimensional gaze positions, and propose an adaptive linear regression (ALR) method as the key to our solution. The ALR method adaptively selects an optimal set of sparsest training samples for the gaze estimation via l ::: 1 ::: -optimization. In this sense, the number of required training samples is significantly reduced for high accuracy estimation. In addition, by adopting the basic ALR objective function, we integrate the gaze estimation, subpixel alignment and blink detection into a unified optimization framework. By solving these problems simultaneously, we successfully handle slight head motion, image resolution variation and eye blinking in appearance-based gaze estimation. We evaluated the proposed method by conducting experiments with multiple users and variant conditions to verify its effectiveness.
---
paper_title: A free-head, simple calibration, gaze tracking system that enables gaze-based interaction

paper_content:

Human eye gaze is a strong candidate to create a new application area based on human-computer interaction. To implement a really practical gaze-based interaction system, gaze detection must be realized without placing any restriction on the user's behavior or comfort. This paper describes a gaze tracking system that offers freehead, simple personal calibration. It does not require the user wear anything on her head, and she can move her head freely. Personal calibration takes only a very short time; the user is asked to look at two markers on the screen. An experiment shows that the accuracy of the implemented system is about 1.0 degrees (view angle).
---
paper_title: Towards the development of a standardized performance evaluation framework for eye gaze estimation systems in consumer platforms

paper_content:

There is a need to standardize the performance of eye gaze estimation (EGE) methods in various platforms for human computer interaction (HCI). Because of lack of consistent schemes or protocols for summative evaluation of EGE systems, performance results in this field can neither be compared nor reproduced with any consistency. In contemporary literature, gaze tracking accuracy is measured under non-identical sets of conditions, with variable metrics and most results do not report the impact of system meta-parameters that significantly affect tracking performances. In this work, the diverse nature of these research outcomes and system parameters which affect gaze tracking in different platforms is investigated and their error contributions are estimated quantitatively. Then the concept and development of a performance evaluation framework is proposed- that can define design criteria and benchmark quality measures for the eye gaze research community.
---
paper_title: Towards the development of a standardized performance evaluation framework for eye gaze estimation systems in consumer platforms

paper_content:

There is a need to standardize the performance of eye gaze estimation (EGE) methods in various platforms for human computer interaction (HCI). Because of lack of consistent schemes or protocols for summative evaluation of EGE systems, performance results in this field can neither be compared nor reproduced with any consistency. In contemporary literature, gaze tracking accuracy is measured under non-identical sets of conditions, with variable metrics and most results do not report the impact of system meta-parameters that significantly affect tracking performances. In this work, the diverse nature of these research outcomes and system parameters which affect gaze tracking in different platforms is investigated and their error contributions are estimated quantitatively. Then the concept and development of a performance evaluation framework is proposed- that can define design criteria and benchmark quality measures for the eye gaze research community.
---

Title: A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms
Section 1: INTRODUCTION
Description 1: Introduce the advancements in eye gaze tracking technology, historical context, applications, and challenges in performance evaluation across different platforms.

Section 2: Types of eye movements studied
Description 2: Discuss the various types of eye movements studied in eye gaze research, including fixations, saccades, scanpath, gaze duration, and pupil size and blink.

Section 3: Basic setup and method used for eye gaze estimation
Description 3: Explain the typical setup for video-based eye gaze tracking systems and the common methodologies used for eye detection and gaze estimation.

Section 4: Calibration
Description 4: Describe the calibration process for eye gaze tracking systems, including the generalized structure and model of the human eye and the calibration routines.

Section 5: Correspondence of eye gaze with head positions
Description 5: Discuss how head orientation affects gaze tracking and the methods to compensate for head movement.

Section 6: Estimation of gaze tracking accuracy
Description 6: Explain the metrics and methods used to estimate the accuracy of gaze tracking systems.

Section 7: EYE GAZE ESTIMATION ALGORITHMS
Description 7: Review and categorize various eye gaze tracking algorithms, including 2D regression, 3D model-based, cross-ratio, appearance-based, and shape-based methods.

Section 8: USER PLATFORMS IMPLEMENTING GAZE TRACKING
Description 8: Describe the implementation of gaze tracking across different user platforms: desktop systems, TV and large display panels, head-mounted setups, automotive, and handheld devices.

Section 9: Diversity of gaze estimation performance metrics in different user platforms
Description 9: Discuss the variety of performance metrics used in gaze estimation research and the inconsistency in their reporting formats.

Section 10: Platform specific factors affecting usability of gaze tracking systems
Description 10: Analyze the factors affecting the practical performance of gaze tracking systems across different platforms and the impact of various error sources.

Section 11: Need and rationale for developing comprehensive performance evaluation strategies for gaze estimation systems
Description 11: Highlight the necessity for standardized performance evaluation strategies and the current gaps in the assessment of gaze tracking systems.

Section 12: Concept of a performance evaluation framework for gaze estimation systems
Description 12: Propose a framework for evaluating gaze tracking systems' performance, including standardized experiments for testing under various conditions.

Section 13: Methodology
Description 13: Outline the methodology of the proposed experimental framework, detailing the steps involved in evaluating the performance of gaze tracking systems.

Section 14: Studying dynamic eye movement characteristics
Description 14: Discuss plans for studying dynamic eye movements using the proposed framework, focusing on smooth pursuits in addition to fixations.