Object Acquisition and Selection in Human Computer Interaction Systems: A Review
---
paper_title: A human-robot interface based on electrooculography

paper_content:

Design and implementation of an electrooculography based gaze-controlled robotic system is presented. The robot system consists of signal acquisition, pattern recognition, control strategy and robot motion modules. The user's eye gaze movements are reconstructed from electrooculogram (EOG) signals, which are recorded from the face in real time. The eye movement patterns, e.g., saccades, fixation and blinks are detected from the raw eye gaze movement data by a pattern recognition module. The control strategy module interprets the user's intention from the eye movement patterns based on predefined protocols. A horizontally mounted robot, emulating the skeletomuscular configuration of the human arm, is controlled by the robot motion control module to execute the interpreted user intention. The performance results of two control strategies are discussed.
---
paper_title: Eye-based user interfaces: Some recent projects

paper_content:

We are currently witnessing an increasing attention to issues related to Accessibility, which should eliminate, or at least reduce, the distance between disabled people and technology. However, particularly for severely-impaired persons, there are still many challenges that must be overcome. In this paper we present eye tracking as a valuable support for disability in the accomplishment of hands-free tasks. Moreover, we stress the potentials of eye-based interfaces to enhance the user-machine interaction process in “traditional” activities based on keyboard and mouse. Through the description of some of the projects we have recently developed at the University of Pavia, we will show how interfaces based on eye tracking can be really helpful in different contexts of use.
---
paper_title: Design of virtual keyboard using blink control method for the severely disabled

paper_content:

In this paper, a human-machine interface with the concept of ''blink control'' is proposed. The human-machine interface is applied to an assistive device, namely ''blink scanning keyboard'', which is designed specifically for the severely physical disabled and people suffering from motor neuron diseases or severe cerebral palsy. The pseudo electromyography (EMG) signal of blinking eyes could be acquired by wearing a Bluetooth headset with one sensor on the forehead and the other three on the left ear of the user. Through a conscious blink, a clear and immediate variation will be formed in the pseudo EMG signal from the users' forehead. The occurrence of this variation in pseudo EMG signal could be detected and filtered by the algorithms proposed in this paper, acting like a trigger to activate the functions integrated in the scanning keyboard. The severely physical and visual disabled then can operate the proposed design by simply blinking their eyes, thus communicating with outside world.
---
paper_title: Comparing mouse and MAGIC pointing for moving target acquisition

paper_content:

Moving target acquisition is a challenging and manually stressful task if performed using an all-manual, pointer-based interaction technique like mouse interaction, especially if targets are small, move fast, and are visible on screen only for a limited time. The MAGIC pointing interaction approach combines the precision of manual, pointer-based interaction with the speed and little manual stress of eye pointing. In this contribution, a pilot study with twelve participants on moving target acquisition is presented using an abstract experimental task derived from a video analysis scenario. Mouse input, conservative MAGIC pointing and MAGIC button are compared considering acquisition time, error rate, and user satisfaction. Although none of the participants had used MAGIC pointing before, eight participants voted for MAGIC button being their favorite technique; participants performed with only slightly higher mean acquisition time and error rate than with the familiar mouse input. Conservative MAGIC pointing was preferred by three participants; however, mean acquisition time and error rate were significantly worse than with mouse input.
---
paper_title: Influences of dwell time and cursor control on the performance in gaze driven typing

paper_content:

In gaze controlled computer interfaces the dwell time is often used as selection criterion. But this solution comes along with several problems, especially in the temporal domain: Eye movement studies on scene perception could demonstrate that fixations of different durations serve different purposes and should therefore be differentiated. The use of dwell time for selection implies the need to distinguish intentional selections from merely per-ceptual processes, described as the Midas touch problem. Moreover, the feedback of the actual own eye position has not yet been addressed to systematic studies in the context of usability in gaze based computer interaction. We present research on the usability of a simple eye typing set up. Different dwell time and eye position feedback configurations were tested. Our results indicate that smoothing raw eye position and temporal delays in visual feedback enhance the system's functionality and usability. Best overall performance was obtained with a dwell time of 500 ms.
---
paper_title: A Myographic-based HCI Solution Proposal for Upper Limb Amputees☆

paper_content:

Abstract Interaction plays a fundamental role as it sets bridges between humans and computers. However, people with disability are prevented to use computers by the ordinary means, due to physical or intellectual impairments. Thus, the human-computer interaction (HCI) research area has been developing solutions to improve the technological accessibility of impaired people, by enhancing computers and similar devices with the necessary means to attend to the different disabilities, thereby contributing to reduce digital exclusion. Within the aforementioned scope, this paper presents an interaction solution for upper limb amputees, supported on a myographic gesture-control device named Myo. This device is an emergent wearable technology, which consists in a muscle-sensitive bracelet. It transmits myographic and inertial data, susceptible of being converted into actions for interaction purposes (e.g. clicking or moving a mouse cursor). Although being a gesture control armband, Myo can also be used in the legs, as was ascertained through some preliminary tests with users. Both data types (myographic and inertial) remain to be transmitted and are available to be converted into gestures. A general architecture, a use case diagram and the two main functional modules specification are presented. These will guide the future implementation of the proposed Myo-based HCI solution, which is intended to be a solid contribution for the interaction between upper limb amputees and computers.
---
paper_title: Face and Eyes mouse for ALS Patients

paper_content:

The objective of this project is to develop the software to detect the Amyotrophic Lateral Sclerosis (ALS) patient's face and eyes movement, so that the patient is able to use computer as normal people. The developed system is called Face and Eyes mouse for ALS Patients or FEMA. The system is divided into 2 parts, client part and system part. The client part is ALS patients who are able to use mouse and keyboard simulation by using only their head and eyes' movement. Another part is FEMA system which will analyze their actions and return the action result to the patient. In addition, there is the keyboard simulation which provides the similar function as the real keyboard. From the experimental results, the precision rate of all functions except move up and move down is more than 80%. We also found that the important factors of the detection result are brightness and camera quality which are most sensitive to the detection accuracy.
---
paper_title: EyePoint: practical pointing and selection using gaze and keyboard

paper_content:

We present a practical technique for pointing and selection using a combination of eye gaze and keyboard triggers. EyePoint uses a two-step progressive refinement process fluidly stitched together in a look-press-look-release action, which makes it possible to compensate for the accuracy limitations of the current state-of-the-art eye gaze trackers. While research in gaze-based pointing has traditionally focused on disabled users, EyePoint makes gaze-based pointing effective and simple enough for even able-bodied users to use for their everyday computing tasks. As the cost of eye gaze tracking devices decreases, it will become possible for such gaze-based techniques to be used as a viable alternative for users who choose not to use a mouse depending on their abilities, tasks and preferences.
---
paper_title: An Eye-Gaze Tracking and Human Computer Interface System for People with ALS and other Locked-in Diseases

paper_content:

Eye tracking is one of the most natural ways for people with amyotrophic lateral sclerosis and other locked-in and paralysis diseases to communicate. The majority of existing eye-tracking computer input systems use cameras to capture images of the user's eyes to track pupil movements. Most camera-based systems are expensive and not user-friendly. This paper proposes an eye-tracking system called Eyel.ive that uses discrete infrared sensors and emitters as the input device. The eye-tracking and ca libration algorithms classify eye positions into six categories, namely looking up, looking down, looking left, looking right, looking straight ahead (i.e. middle direction), and eye s closed. A graphical user interface optimized for EyeLive is also developed. It divides the screen into a nine-cell grid and uses a hierarchical selection approach for text input. EyeLive's hardware, eye-tracking algorithm, calibration, and user inter face arc compared to those of existing eye-tracking systems. The advantage s of the EyeLive system, such as low cost, user friendliness, and eye strain reduction, are discussed. The performance of the system in classifying eye positions is experimentally tested with eight health y individuals. The results show a 5.6% error rate in classifying eye positions in live directions and a 0% error rate in classifying closed eyes. Additional experiments show that the average typing speed is 1.95 words/min for a novice user and 2.9 1 words /min for an experience d user. The tradeoffs of lower typing speed versus other advantages comparing to other systems are explained.
---
paper_title: The Camera Mouse: visual tracking of body features to provide computer access for people with severe disabilities

paper_content:

The "Camera Mouse" system has been developed to provide computer access for people with severe disabilities. The system tracks the computer user's movements with a video camera and translates them into the movements of the mouse pointer on the screen. Body features such as the tip of the user's nose or finger can be tracked. The visual tracking algorithm is based on cropping an online template of the tracked feature from the current image frame and testing where this template correlates in the subsequent frame. The location of the highest correlation is interpreted as the new location of the feature in the subsequent frame. Various body features are examined for tracking robustness and user convenience. A group of 20 people without disabilities tested the Camera Mouse and quickly learned how to use it to spell out messages or play games. Twelve people with severe cerebral palsy or traumatic brain injury have tried the system, nine of whom have shown success. They interacted with their environment by spelling out messages and exploring the Internet.
---
paper_title: BlinkWrite: efficient text entry using eye blinks

paper_content:

In this paper, a new text entry system is proposed, implemented, and evaluated. BlinkWrite provides a communication gateway for cognitively able motor-impaired individuals who cannot use a traditional eye-tracking system. In contrast to most hands-free systems, BlinkWrite allows text to be entered and corrected using a single input modality: blinks. The system was implemented using a scanning ambiguous keyboard, a new form of scanning keyboard that allows English text to be entered in less than two scanning intervals per character. In a user study, 12 participants entered text using the system with three settings for scanning interval: 1,000, 850, and 700 ms. An average text entry rate of 4.8 wpm was observed with accuracy >97%. The highest average text entry rate was achieved with the scanning interval of 850 ms.
---
paper_title: Twenty years of eye typing: systems and design issues

paper_content:

Eye typing provides a means of communication for severely handicapped people, even those who are only capable of moving their eyes. This paper considers the features, functionality and methods used in the eye typing systems developed in the last twenty years. Primary concerned with text production, the paper also addresses other communication related issues, among them customization and voice output.
---
paper_title: A new interaction technique involving eye gaze tracker and scanning system

paper_content:

This paper presents a new input interaction system for people with severe disabilities by combining eye gaze tracking and single switch scanning interaction techniques. The system is faster than only scanning based systems while more comfortable to use than existing eye gaze tracking based systems. We reported results from a couple of user studies that show the new system is equally fast compared to existing eye tracking systems that does not involve scanning, participants with no prior experience with eye tracking based system could learn using this new system successfully within 10 minutes but it demands higher mental effort in comparison to another new modality of interaction- a gesture based system.
---
paper_title: Gazing with pEYEs: towards a universal input for various applications

paper_content:

Various interfaces for gaze control (which are recommended due to certain requirements of controlling a machine by gaze) have already been developed. One problem, especially for novice users, is that respective interfaces all look different and require different steps to use. As a means to unify interfaces for gaze control, pie menus are suggested. Such pEYEs allow for universal input in various applications usable by novices and by experts. We present two examples for pEYE interfaces; one eye-typing application and one desktop navigation. Observations in user studies indicate effective and efficient performance and a large acceptance.
---
paper_title: Eye-S: a full-screen input modality for pure eye-based communication

paper_content:

To date, several eye input methods have been developed, which, however, are usually designed for specific purposes (e.g. typing) and require dedicated graphical interfaces. In this paper we present Eye-S, a system that allows general input to be provided to the computer through a pure eye-based approach. Thanks to the "eye graffiti" communication style adopted, the technique can be used both for writing and for generating other kinds of commands. In Eye-S, letters and general eye gestures are created through sequences of fixations on nine areas of the screen, which we call hotspots. Being usually not visible, such sensitive regions do not interfere with other applications, that can therefore exploit all the available display space.
---
paper_title: Designing gaze gestures for gaming: an investigation of performance

paper_content:

To enable people with motor impairments to use gaze control to play online games and take part in virtual communities, new interaction techniques are needed that overcome the limitations of dwell clicking on icons in the games interface. We have investigated gaze gestures as a means of achieving this. We report the results of an experiment with 24 participants that examined performance differences between different gestures. We were able to predict the effect on performance of the numbers of legs in the gesture and the primary direction of eye movement in a gesture. We also report the outcomes of user trials in which 12 experienced gamers used the gaze gesture interface to play World of Warcraft. All participants were able to move around and engage other characters in fighting episodes successfully. Gestures were good for issuing specific commands such as spell casting, and less good for continuous control of movement compared with other gaze interaction techniques we have developed.
---
paper_title: No clicks, no problem: using cursor movements to understand and improve search

paper_content:

Understanding how people interact with search engines is important in improving search quality. Web search engines typically analyze queries and clicked results, but these actions provide limited signals regarding search interaction. Laboratory studies often use richer methods such as gaze tracking, but this is impractical at Web scale. In this paper, we examine mouse cursor behavior on search engine results pages (SERPs), including not only clicks but also cursor movements and hovers over different page regions. We: (i) report an eye-tracking study showing that cursor position is closely related to eye gaze, especially on SERPs; (ii) present a scalable approach to capture cursor movements, and an analysis of search result examination behavior evident in these large-scale cursor data; and (iii) describe two applications (estimating search result relevance and distinguishing good from bad abandonment) that demonstrate the value of capturing cursor data. Our findings help us better understand how searchers use cursors on SERPs and can help design more effective search systems. Our scalable cursor tracking method may also be useful in non-search settings.
---
paper_title: An Eye-Gaze Tracking and Human Computer Interface System for People with ALS and other Locked-in Diseases

paper_content:

Eye tracking is one of the most natural ways for people with amyotrophic lateral sclerosis and other locked-in and paralysis diseases to communicate. The majority of existing eye-tracking computer input systems use cameras to capture images of the user's eyes to track pupil movements. Most camera-based systems are expensive and not user-friendly. This paper proposes an eye-tracking system called Eyel.ive that uses discrete infrared sensors and emitters as the input device. The eye-tracking and ca libration algorithms classify eye positions into six categories, namely looking up, looking down, looking left, looking right, looking straight ahead (i.e. middle direction), and eye s closed. A graphical user interface optimized for EyeLive is also developed. It divides the screen into a nine-cell grid and uses a hierarchical selection approach for text input. EyeLive's hardware, eye-tracking algorithm, calibration, and user inter face arc compared to those of existing eye-tracking systems. The advantage s of the EyeLive system, such as low cost, user friendliness, and eye strain reduction, are discussed. The performance of the system in classifying eye positions is experimentally tested with eight health y individuals. The results show a 5.6% error rate in classifying eye positions in live directions and a 0% error rate in classifying closed eyes. Additional experiments show that the average typing speed is 1.95 words/min for a novice user and 2.9 1 words /min for an experience d user. The tradeoffs of lower typing speed versus other advantages comparing to other systems are explained.
---
paper_title: The Camera Mouse: visual tracking of body features to provide computer access for people with severe disabilities

paper_content:

The "Camera Mouse" system has been developed to provide computer access for people with severe disabilities. The system tracks the computer user's movements with a video camera and translates them into the movements of the mouse pointer on the screen. Body features such as the tip of the user's nose or finger can be tracked. The visual tracking algorithm is based on cropping an online template of the tracked feature from the current image frame and testing where this template correlates in the subsequent frame. The location of the highest correlation is interpreted as the new location of the feature in the subsequent frame. Various body features are examined for tracking robustness and user convenience. A group of 20 people without disabilities tested the Camera Mouse and quickly learned how to use it to spell out messages or play games. Twelve people with severe cerebral palsy or traumatic brain injury have tried the system, nine of whom have shown success. They interacted with their environment by spelling out messages and exploring the Internet.
---
paper_title: Portable tongue-supported human computer interaction system design and implementation

paper_content:

 Abstract—Tongue supported human-computer interaction (TSHCI) systems can help critically ill patients interact with both computers and people. These systems can be particularly useful for patients suffering injuries above C7 on their spinal vertebrae. Despite recent successes in their application, several limitations restrict performance of existing TSHCI systems and discourage their use in real life situations. This paper proposes a low-cost, less-intrusive, portable and easy to use design for implementing a TSHCI system. Two applications of the proposed system are reported. Design considerations and performance of the proposed system are also presented.
---
paper_title: Detection of gaze direction by using improved eye-tracking technique

paper_content:

Among the various techniques used for the detection of gaze direction, the procedure based on determining pupil or iris position using video camera connected to a computer deserves special attention. Gaze follows the movement of an item - “a target point” on the PC screen, the video camera captures the images of the eye and the PC determines the gaze position; this appears as “a map” on which the positions detected show as points around the target point. It is essential for the result to consist in a stable image or a map for gaze direction to be recognized by the PC. This paper presents a procedure for achieving stable cursor movement using low-pass filtering. It discusses the design, achievement and test results of a digital low-pass filter used in order to ensure the stableness of the point on the map that corresponds to gaze direction.
---
paper_title: Gaze-based interaction: A 30 year retrospective

paper_content:

Abstract Gaze-based interaction is reviewed, categorized within a taxonomy that splits interaction into four forms, namely diagnostic (off-line measurement), active (selection, look to shoot), passive (foveated rendering, a.k.a. gaze-contingent displays), and expressive (gaze synthesis). Diagnostic interaction is the mainstay of eye-tracked applications, including training or assessment of expertise, and is possibly the longest standing use of gaze due to its mainly offline requirements. Diagnostic analysis of gaze is still very much in demand, especially in training situations such as flight or surgery training. Active interaction is rooted in the desire to use the eyes to point and click, with gaze gestures growing in popularity. Passive interaction is the manipulation of scene elements in response to gaze direction, e.g., to improve frame rate. Expressive eye movement is drawn from its synthesis, which can make use of a procedural (stochastic) model of eye motion driven by goal-oriented tasks such as reading. In discussing each form of interaction, seminal results and recent advancements are reviewed, highlighting outstanding research problems. The survey paper extends an invited proceedings contribution to VS-Games 2017.
---
paper_title: A wireless EOG-based Human Computer Interface

paper_content:

Human Computer Interface (HCI) is becoming a hot area of research. It can be developed for the disabled and many other application fields. Biopentials existed in human body, such as Electroencephalogram (EEG), Electromyogram (EMG), Electrooculogram (EOG) are used to be inputs of the HCI system. In this paper, the design scheme of a wireless EOG-based Human Computer Interfaces is presented. The detailed system diagram is shown in the paper. It consists of EOG signal acquisition, EOG filter and amplifier, ARM microcontroller and Zigbee wireless module. Also, the methods of eye gestures recognition are discussed.
---
paper_title: A novel approach for Human Computer Interface based on eye movements for disabled people

paper_content:

Several researches have been carried out in recent years for developing Human Computer Interface (HCI). Human Computer Interface as an assistive technology helps the people with motor disabilities and who can't move their arms. Eye tracking techniques can be used for the communication of these people. In this paper, image based eye tracking technique is used for interaction. The aim of this paper is to help the disabled people to use the computer efficiently. It is based on con-trolling cursor movements on the screen using only the eyes. The system proposed in this paper uses a low resolution webcam which is attached to a wearable glass frame, for eye tracking. The images obtained from the webcam are processed to locate the iris center. This is used to estimate the eye gaze. The cursor is moved in the desired direction on the screen based on the estimated eye gaze. The selection of the tabs is also carried out by blinking. The proposed system shows good performance in good lighting conditions.
---
paper_title: Interacting with the Computer using Gaze Gestures

paper_content:

This paper investigates novel ways to direct computers by eye gaze. Instead of using fixations and dwell times, this work focuses on eye motion, in particular gaze gestures. Gaze gestures are insensitive to accuracy problems and immune against calibration shift. A user study indicates that users are able to perform complex gaze gestures intentionally and investigates which gestures occur unintentionally during normal interaction with the computer. Further experiments show how gaze gestures can be integrated into working with standard desktop applications and controlling media devices.
---
paper_title: Eye/Head Tracking Technology to Improve HCI with iPad Applications

paper_content:

In order to improve human computer interaction (HCI) for people with special needs, this paper presents an alternative form of interaction, which uses the iPad's front camera and eye/head tracking technology. With this functional nature/capability operating in the background, the user can control already developed or new applications for the iPad by moving their eyes and/or head. There are many techniques, which are currently used to detect facial features, such as eyes or even the face itself. Open source bookstores exist for such purpose, such as OpenCV, which enable very reliable and accurate detection algorithms to be applied, such as Haar Cascade using very high-level programming. All processing is undertaken in real time, and it is therefore important to pay close attention to the use of limited resources (processing capacity) of devices, such as the iPad. The system was validated in tests involving 22 users of different ages and characteristics (people with dark and light-colored eyes and with/without glasses). These tests are performed to assess user/device interaction and to ascertain whether it works properly. The system obtained an accuracy of between 60% and 100% in the three test exercises taken into consideration. The results showed that the Haar Cascade had a significant effect by detecting faces in 100% of cases, unlike eyes and the pupil where interference (light and shade) evidenced less effectiveness. In addition to ascertaining the effectiveness of the system via these exercises, the demo application has also helped to show that user constraints need not affect the enjoyment and use of a particular type of technology. In short, the results obtained are encouraging and these systems may continue to be developed if extended and updated in the future.
---
paper_title: Eye-based user interfaces: Some recent projects

paper_content:

We are currently witnessing an increasing attention to issues related to Accessibility, which should eliminate, or at least reduce, the distance between disabled people and technology. However, particularly for severely-impaired persons, there are still many challenges that must be overcome. In this paper we present eye tracking as a valuable support for disability in the accomplishment of hands-free tasks. Moreover, we stress the potentials of eye-based interfaces to enhance the user-machine interaction process in “traditional” activities based on keyboard and mouse. Through the description of some of the projects we have recently developed at the University of Pavia, we will show how interfaces based on eye tracking can be really helpful in different contexts of use.
---
paper_title: Development of EOG-Based Communication System Controlled by Eight-Directional Eye Movements

paper_content:

A communication support interface controlled by eye movements and voluntary eye blink has been developed for disabled individuals with motor paralysis who cannot speak. Horizontal and vertical electro-oculograms were measured using two electrodes attached above and beside the dominant eye and referring to an earlobe electrode and amplified with AC-coupling in order to reduce the unnecessary drift. Eight directional cursor movements and one selected operation were realized by logically combining the two detected channel signals based on threshold setting specific to the individuals. As experimental results using a projected screen keyboard, processing speed was improved to 12.1 letters/min. while the accuracy was 90.4%.
---
paper_title: The Camera Mouse: visual tracking of body features to provide computer access for people with severe disabilities

paper_content:

The "Camera Mouse" system has been developed to provide computer access for people with severe disabilities. The system tracks the computer user's movements with a video camera and translates them into the movements of the mouse pointer on the screen. Body features such as the tip of the user's nose or finger can be tracked. The visual tracking algorithm is based on cropping an online template of the tracked feature from the current image frame and testing where this template correlates in the subsequent frame. The location of the highest correlation is interpreted as the new location of the feature in the subsequent frame. Various body features are examined for tracking robustness and user convenience. A group of 20 people without disabilities tested the Camera Mouse and quickly learned how to use it to spell out messages or play games. Twelve people with severe cerebral palsy or traumatic brain injury have tried the system, nine of whom have shown success. They interacted with their environment by spelling out messages and exploring the Internet.
---
paper_title: System for assisted mobility using eye movements based on electrooculography

paper_content:

This paper describes an eye-control method based on electrooculography (EOG) to develop a system for assisted mobility. One of its most important features is its modularity, making it adaptable to the particular needs of each user according to the type and degree of handicap involved. An eye model based on electroculographic signal is proposed and its validity is studied. Several human-machine interfaces (HMI) based on EOG are commented, focusing our study on guiding and controlling a wheelchair for disabled people, where the control is actually effected by eye movements within the socket. Different techniques and guidance strategies are then shown with comments on the advantages and disadvantages of each one. The system consists of a standard electric wheelchair with an on-board computer, sensors and a graphic user interface run by the computer. On the other hand, this eye-control method can be applied to handle graphical interfaces, where the eye is used as a mouse computer. Results obtained show that this control technique could be useful in multiple applications, such as mobility and communication aid for handicapped persons.
---
paper_title: The Camera Mouse: visual tracking of body features to provide computer access for people with severe disabilities

paper_content:

The "Camera Mouse" system has been developed to provide computer access for people with severe disabilities. The system tracks the computer user's movements with a video camera and translates them into the movements of the mouse pointer on the screen. Body features such as the tip of the user's nose or finger can be tracked. The visual tracking algorithm is based on cropping an online template of the tracked feature from the current image frame and testing where this template correlates in the subsequent frame. The location of the highest correlation is interpreted as the new location of the feature in the subsequent frame. Various body features are examined for tracking robustness and user convenience. A group of 20 people without disabilities tested the Camera Mouse and quickly learned how to use it to spell out messages or play games. Twelve people with severe cerebral palsy or traumatic brain injury have tried the system, nine of whom have shown success. They interacted with their environment by spelling out messages and exploring the Internet.
---
paper_title: Portable tongue-supported human computer interaction system design and implementation

paper_content:

 Abstract—Tongue supported human-computer interaction (TSHCI) systems can help critically ill patients interact with both computers and people. These systems can be particularly useful for patients suffering injuries above C7 on their spinal vertebrae. Despite recent successes in their application, several limitations restrict performance of existing TSHCI systems and discourage their use in real life situations. This paper proposes a low-cost, less-intrusive, portable and easy to use design for implementing a TSHCI system. Two applications of the proposed system are reported. Design considerations and performance of the proposed system are also presented.
---
paper_title: Gazing with pEYEs: towards a universal input for various applications

paper_content:

Various interfaces for gaze control (which are recommended due to certain requirements of controlling a machine by gaze) have already been developed. One problem, especially for novice users, is that respective interfaces all look different and require different steps to use. As a means to unify interfaces for gaze control, pie menus are suggested. Such pEYEs allow for universal input in various applications usable by novices and by experts. We present two examples for pEYE interfaces; one eye-typing application and one desktop navigation. Observations in user studies indicate effective and efficient performance and a large acceptance.
---
paper_title: Alternatives to single character entry and dwell time selection on eye typing

paper_content:

Eye typing could provide motor disabled people a reliable method of communication given that the text entry speed of current interfaces can be increased to allow for fluent communication. There are two reasons for the relatively slow text entry: dwell time selection requires waiting a certain time, and single character entry limits the maximum entry speed. We adopted a typing interface based on hierarchical pie menus, pEYEwrite [Urbina and Huckauf 2007] and included bigram text entry with one single pie iteration. Therefore, we introduced three different bigram building strategies. Moreover, we combined dwell time selection with selection by borders, providing an alternative selection method and extra functionality. In a longitudinal study we compared participants performance during character-by-character text entry with bigram entry and with text entry with bigrams derived by word prediction. Data showed large advantages of the new entry methods over single character text entry in speed and accuracy. Participants preferred selecting by borders, which allowed them faster selections than the dwell time method.
---
paper_title: Eye typing using word and letter prediction and a fixation algorithm

paper_content:

Two eye typing techniques and a fixation algorithm are described. Similar to word prediction, letter prediction chooses three highly probable next letters and highlights them on an onscreen keyboard. Letter prediction proved promising, as it was as good as word prediction, and in some cases better. The fixation algorithm chooses which button to select for eye-over highlighting. It often chooses the desired button even if another button is closer to the fixation location. Error rates were reduced when using the fixation algorithm combined with letter prediction; however, the algorithm was sensitive to the correctness of the first several letters in a word.
---
paper_title: An Eye-Gaze Tracking and Human Computer Interface System for People with ALS and other Locked-in Diseases

paper_content:

Eye tracking is one of the most natural ways for people with amyotrophic lateral sclerosis and other locked-in and paralysis diseases to communicate. The majority of existing eye-tracking computer input systems use cameras to capture images of the user's eyes to track pupil movements. Most camera-based systems are expensive and not user-friendly. This paper proposes an eye-tracking system called Eyel.ive that uses discrete infrared sensors and emitters as the input device. The eye-tracking and ca libration algorithms classify eye positions into six categories, namely looking up, looking down, looking left, looking right, looking straight ahead (i.e. middle direction), and eye s closed. A graphical user interface optimized for EyeLive is also developed. It divides the screen into a nine-cell grid and uses a hierarchical selection approach for text input. EyeLive's hardware, eye-tracking algorithm, calibration, and user inter face arc compared to those of existing eye-tracking systems. The advantage s of the EyeLive system, such as low cost, user friendliness, and eye strain reduction, are discussed. The performance of the system in classifying eye positions is experimentally tested with eight health y individuals. The results show a 5.6% error rate in classifying eye positions in live directions and a 0% error rate in classifying closed eyes. Additional experiments show that the average typing speed is 1.95 words/min for a novice user and 2.9 1 words /min for an experience d user. The tradeoffs of lower typing speed versus other advantages comparing to other systems are explained.
---
paper_title: Object selection in gaze controlled systems: What you don't look at is what you get

paper_content:

Controlling computers using eye movements can provide a fast and efficient alternative to the computer mouse. However, implementing object selection in gaze-controlled systems is still a challenge. Dwell times or fixations on a certain object typically used to elicit the selection of this object show several disadvantages. We studied deviations of critical thresholds by an individual and task-specific adaptation method. This demonstrated an enormous variability of optimal dwell times. We developed an alternative approach using antisaccades for selection. For selection by antisaccades, highlighted objects are copied to one side of the object. The object is selected when fixating to the side opposed to that copy requiring to inhibit an automatic gaze shift toward new objects. Both techniques were compared in a selection task. Two experiments revealed superior performance in terms of errors for the individually adapted dwell times. Antisaccades provide an alternative approach to dwell time selection, but they did not show an improvement over dwell time. We discuss potential improvements in the antisaccade implementation with which antisaccades might become a serious alternative to dwell times for object selection in gaze-controlled systems.
---
paper_title: BlinkWrite: efficient text entry using eye blinks

paper_content:

In this paper, a new text entry system is proposed, implemented, and evaluated. BlinkWrite provides a communication gateway for cognitively able motor-impaired individuals who cannot use a traditional eye-tracking system. In contrast to most hands-free systems, BlinkWrite allows text to be entered and corrected using a single input modality: blinks. The system was implemented using a scanning ambiguous keyboard, a new form of scanning keyboard that allows English text to be entered in less than two scanning intervals per character. In a user study, 12 participants entered text using the system with three settings for scanning interval: 1,000, 850, and 700 ms. An average text entry rate of 4.8 wpm was observed with accuracy >97%. The highest average text entry rate was achieved with the scanning interval of 850 ms.
---
paper_title: A new input system for disabled users involving eye gaze tracker and scanning interface

paper_content:

Purpose – This paper aims to present a new input interaction system for people with severe disabilities. The new system works based on eye gaze tracking and single switch scanning interaction techniques. It combines eye gaze tracking and scanning in a unique way which is faster than only scanning‐based systems while more comfortable to use than only eye gaze tracking‐based systems, which is also supported by a user study.Design/methodology/approach – The aim was to design a new interaction technique combining eye tracking and scanning technique for severely disabled people. A comparative study with eight users involving only an eye tracking system and the system combining eye tracking and scanning was conducted.Findings – The participants rated the new system easier and less strenuous to use than the eye gaze tracking‐based system, while the new system did not slow the speed of interaction.Originality/value – This new method based on eye gaze tracking and single switch scanning interaction techniques has ...
---
paper_title: Design of virtual keyboard using blink control method for the severely disabled

paper_content:

In this paper, a human-machine interface with the concept of ''blink control'' is proposed. The human-machine interface is applied to an assistive device, namely ''blink scanning keyboard'', which is designed specifically for the severely physical disabled and people suffering from motor neuron diseases or severe cerebral palsy. The pseudo electromyography (EMG) signal of blinking eyes could be acquired by wearing a Bluetooth headset with one sensor on the forehead and the other three on the left ear of the user. Through a conscious blink, a clear and immediate variation will be formed in the pseudo EMG signal from the users' forehead. The occurrence of this variation in pseudo EMG signal could be detected and filtered by the algorithms proposed in this paper, acting like a trigger to activate the functions integrated in the scanning keyboard. The severely physical and visual disabled then can operate the proposed design by simply blinking their eyes, thus communicating with outside world.
---
paper_title: BlinkWrite2: an improved text entry method using eye blinks

paper_content:

Areas of design improvements for BlinkWrite, an eye blink text entry system, are examined, implemented, and evaluated. The result, BlinkWrite2, is a text entry system for individuals with severe motor impairment. Since the ability to blink is often preserved, even in severe conditions such as locked-in syndrome, BlinkWrite2 allows text entry and correction with blinks as the only input modality. Advantages of BlinkWrite2 over its predecessor include an increase in text entry speed. In a user evaluation, 12 participants achieved an average text entry rate of 5.3 wpm, representing a 16% increase over BlinkWrite and a 657% increase over the next fastest video-based eye blink text entry system reported in the literature.
---
paper_title: Communication via eye blinks and eyebrow raises: video-based human-computer interfaces

paper_content:

Two video-based human-computer interaction tools are introduced that can activate a binary switch and issue a selection command. "BlinkLink," as the first tool is called, automatically detects a user's eye blinks and accurately measures their durations. The system is intended to provide an alternate input modality to allow people with severe disabilities to access a computer. Voluntary long blinks trigger mouse clicks, while involuntary short blinks are ignored. The system enables communication using "blink patterns:" sequences of long and short blinks which are interpreted as semiotic messages. The second tool, "EyebrowClicker," automatically detects when a user raises his or her eyebrows and then triggers a mouse click. Both systems can initialize themselves, track the eyes at frame rate, and recover in the event of errors. No special lighting is required. The systems have been tested with interactive games and a spelling program. Results demonstrate overall detection accuracy of 95.6% for BlinkLink and 89.0% for EyebrowClicker.
---
paper_title: A Novel Wearable Forehead EOG Measurement System for Human Computer Interfaces

paper_content:

Amyotrophic lateral sclerosis (ALS) patients whose voluntary muscles are paralyzed commonly communicate with the outside world using eye movement. There have been many efforts to support this method of communication by tracking or detecting eye movement. An electrooculogram (EOG), an electro-physiological signal, is generated by eye movements and can be measured with electrodes placed around the eye. In this study, we proposed a new practical electrode position on the forehead to measure EOG signals, and we developed a wearable forehead EOG measurement system for use in Human Computer/Machine interfaces (HCIs/HMIs). Four electrodes, including the ground electrode, were placed on the forehead. The two channels were arranged vertically and horizontally, sharing a positive electrode. Additionally, a real-time eye movement classification algorithm was developed based on the characteristics of the forehead EOG. Three applications were employed to evaluate the proposed system: a virtual keyboard using a modified Bremen BCI speller and an automatic sequential row-column scanner, and a drivable power wheelchair. The mean typing speeds of the modified Bremen brain–computer interface (BCI) speller and automatic row-column scanner were 10.81 and 7.74 letters per minute, and the mean classification accuracies were 91.25% and 95.12%, respectively. In the power wheelchair demonstration, the user drove the wheelchair through an 8-shape course without collision with obstacles.
---
paper_title: A new interaction technique involving eye gaze tracker and scanning system

paper_content:

This paper presents a new input interaction system for people with severe disabilities by combining eye gaze tracking and single switch scanning interaction techniques. The system is faster than only scanning based systems while more comfortable to use than existing eye gaze tracking based systems. We reported results from a couple of user studies that show the new system is equally fast compared to existing eye tracking systems that does not involve scanning, participants with no prior experience with eye tracking based system could learn using this new system successfully within 10 minutes but it demands higher mental effort in comparison to another new modality of interaction- a gesture based system.
---
paper_title: The effect of clicking by smiling on the accuracy of head-mounted gaze tracking

paper_content:

The effect of facial behaviour on gaze tracking accuracy was studied while using a prototype system that integrated head-mounted, video-based gaze tracking and a capacitive facial movement detection for respective pointing and selecting objects in a simple graphical user interface. Experiments were carried out to determine how voluntary smiling movements that were used to indicate clicks affect the accuracy of gaze tracking due to the combination of user eye movement behaviour and the operation of gaze tracking algorithms. The results showed no observable degradation of the gaze tracking accuracy when using voluntary smiling for object selections.
---
paper_title: Iris tracking and blink detection for human-computer interaction using a low resolution webcam

paper_content:

This paper proposes a technique to develop a non-intrusive interface for human-computer interaction based on visual cues of a human eye, captured using a low resolution USB camera. Such an interface can replace a traditional interface like a mouse thus helping the severely disabled to use the computer just as a normal individual would. It makes use of iris tracking and blink detection for this purpose. Considering fairly constant lighting conditions, a modified Hough Transform is used to track the iris at real time. The Between-The-Eyes feature serves as a stable reference point for gaze direction estimation. When mapped to the computer screen, incremental, directional movement of the mouse pointer position provides precise control over its movement. A method based on the projection function is proposed to detect the user's eye blinks, analyze their patterns and duration, interpreting them as appropriate mouse clicks. The system gives an overall accuracy of 87.4 %.
---
paper_title: BlinkWrite: efficient text entry using eye blinks

paper_content:

In this paper, a new text entry system is proposed, implemented, and evaluated. BlinkWrite provides a communication gateway for cognitively able motor-impaired individuals who cannot use a traditional eye-tracking system. In contrast to most hands-free systems, BlinkWrite allows text to be entered and corrected using a single input modality: blinks. The system was implemented using a scanning ambiguous keyboard, a new form of scanning keyboard that allows English text to be entered in less than two scanning intervals per character. In a user study, 12 participants entered text using the system with three settings for scanning interval: 1,000, 850, and 700 ms. An average text entry rate of 4.8 wpm was observed with accuracy >97%. The highest average text entry rate was achieved with the scanning interval of 850 ms.
---
paper_title: Typing with eye-gaze and tooth-clicks

paper_content:

In eye-gaze-based human-computer interfaces, the most commonly used mechanism for generating activation commands (i.e., mouse clicks) is dwell time (DT). While DT can be relatively efficient and easy to use, it is also associated with the possibility of generating unintentional activation commands -- an issue that is known as the Midas' touch problem. To address this problem, we proposed to use a "tooth-clicker" (TC) device as a mechanism for generating activation commands independently of the activity of the eyes. This paper describes a pilot study that verifies the feasibility of using an eye-gaze tracker (EGT) and a TC to type on an on-screen keyboard, and compares the performance of the EGT-TC system with that of the EGT with two different DT thresholds (880 ms and 490 ms). The six subjects that participated in the study were able to attain typing speeds using the EGT-TC system that were slower than but comparable to the typing speeds that they attained using the EGT with the shorter DT threshold.
---
paper_title: Eye-based HCI with Full Specification of Mouse and Keyboard Using Pupil Knowledge in the Gaze Estimation

paper_content:

Eye-based Human Computer Interaction (HCI) with full specification of mouse and keyboard by human eyes only is proposed for, in particular, disabled person and for input device of wearable computing. It utilizes pupil knowledge in gaze estimation process. Existing conventional eye-mouse, gaze-mouse does not robust against various types of users with different features of color, shape and size of eyes and also is affected by illumination changes, users' movement (attitude changes), etc. Using knowledge about features, the influences are eliminated. Also the proposed eye-based HCI system allows simultaneous key input of more than three keys, for instance, "Ctl+Alt+Del" for initiating task manager, left/right click and drag/drop of mouse event capabilities. Although currently commercially available screen keyboard allows simultaneous key input for two keys such as "shit+*", "Alt+*", it does not allow three keys input at once. Such these full specification of mouse events and keyboard functions are available for the proposed HCI system. Experimental results with six different nationalities of users with the different features shows effectiveness of using the knowledge for improvement of gaze estimation accuracy.
---
paper_title: On object selection in gaze controlled environments

paper_content:

In the past twenty years, gaze control has become a reliable alternative input method not only for handicapped users. The selection of objects, however, which is of highest importance and of highest frequency in computer control, requires explicit control not inherent in eye movements. Objects have been therefore usually selected via prolonged fixations (dwell times). Dwell times seemed to be for many years the unique reliable method for selection. In this paper, we review pros and cons of classical selection methods and novel metaphors, which are based on pies and gestures. The focus is on the effectiveness and efficiency of selections. In order to estimate the potential of current suggestions for selection, a basic empirical comparison is recommended.
---
paper_title: Design of virtual keyboard using blink control method for the severely disabled

paper_content:

In this paper, a human-machine interface with the concept of ''blink control'' is proposed. The human-machine interface is applied to an assistive device, namely ''blink scanning keyboard'', which is designed specifically for the severely physical disabled and people suffering from motor neuron diseases or severe cerebral palsy. The pseudo electromyography (EMG) signal of blinking eyes could be acquired by wearing a Bluetooth headset with one sensor on the forehead and the other three on the left ear of the user. Through a conscious blink, a clear and immediate variation will be formed in the pseudo EMG signal from the users' forehead. The occurrence of this variation in pseudo EMG signal could be detected and filtered by the algorithms proposed in this paper, acting like a trigger to activate the functions integrated in the scanning keyboard. The severely physical and visual disabled then can operate the proposed design by simply blinking their eyes, thus communicating with outside world.
---
paper_title: Controlling Mouse through Eyes

paper_content:

User-computer dialogues are typically one-sided, with the benefits from computer to user far greater than that from user to computer. The movement of a user's eyes can provide a convenient, natural, and high-bandwidth source of additional user input, to help redress this one-sidedness. We, therefore investigate the introduction of eye movements as a computer input medium. Our emphasis is on the study of interaction techniques that incorporate eye movements into the user-computer dialogue. In this paper, an advanced approach to man machine interaction is proposed, in which computer vision techniques are used for interpreting user actions.
---
paper_title: Vision-Based Interface: Using Face and Eye Blinking Tracking with Camera

paper_content:

To provide new type of interaction between human-machine, many devices which could substitute input equipments (e.g. mouse, keyboard etc.) have been invented. In this paper, we develop a novel robust camera mouse based on common Webcam which could be used by handicapped, or be used in some special application. We choose common Webcam to build the device in order to ensure the universality of our design. Combination between optical flow and Harris corner detector obtains good result, and an approach based on tracking fix feature point is used to detect eye blinking. The application can be applied to various illuminations even in a house with no lamp but only screen light. At the same time, this method just takes up less than 30% of system resource. Based on our approach we will present a library from which anybody could develop more application.
---
paper_title: Comparison of video-based pointing and selection techniques for hands-free text entry

paper_content:

Video-based human-computer interaction has received increasing interest over the years. However, earlier research has been mainly focusing on technical characteristics of different methods rather than on user performance and experiences in using computer vision technology. This study aims to investigate performance characteristics of novice users and their subjective experiences in typing text with several video-based pointing and selection techniques. In Experiment 1, eye tracking and head tracking were applied for the task of pointing at the keys of a virtual keyboard. The results showed that gaze pointing was significantly faster but also more erroneous technique as compared with head pointing. Self-reported subjective ratings revealed that it was generally better, faster, more pleasant and efficient to type using gaze pointing than head pointing. In Experiment 2, mouth open and brows up facial gestures were utilized for confirming the selection of a given character. The results showed that text entry speed was approximately the same for both selection techniques, while mouth interaction caused significantly fewer errors than brow interaction. Subjective ratings did not reveal any significant differences between the techniques. Possibilities for design improvements are discussed.
---
paper_title: Alternatives to single character entry and dwell time selection on eye typing

paper_content:

Eye typing could provide motor disabled people a reliable method of communication given that the text entry speed of current interfaces can be increased to allow for fluent communication. There are two reasons for the relatively slow text entry: dwell time selection requires waiting a certain time, and single character entry limits the maximum entry speed. We adopted a typing interface based on hierarchical pie menus, pEYEwrite [Urbina and Huckauf 2007] and included bigram text entry with one single pie iteration. Therefore, we introduced three different bigram building strategies. Moreover, we combined dwell time selection with selection by borders, providing an alternative selection method and extra functionality. In a longitudinal study we compared participants performance during character-by-character text entry with bigram entry and with text entry with bigrams derived by word prediction. Data showed large advantages of the new entry methods over single character text entry in speed and accuracy. Participants preferred selecting by borders, which allowed them faster selections than the dwell time method.
---
paper_title: Face and eye tracking for controlling computer functions

paper_content:

This paper presents a novel user interface with computer. We propose eye and face detection via Haar-like feature with Bilateral Total Variation based super-resolution to resize the low resolution eye image from web camera. The center of eye-pair detection is mapped to the mouse coordinates in computer screen. Left click and right click are controlled by blinking left eye and right eye, respectively. Ambient illuminations are fluorescence and natural daylight in day time mode and fluorescence as one light source at night time mode. The system performance are evaluated on measuring sensitivities of face and eye detection across (1) distances between web camera and the user, (2) roll and pitch angles of the head pose, and (3) eye-center coordinates for controlling computer functions.
---
paper_title: Enhancing an Eye-Tracker based Human-Computer Interface with Multi-modal Accessibility Applied for Text Entry

paper_content:

In natural course, human beings usually make use of multisensory modalities for effective communication or efficiently executing day-to-day tasks. For instance, during verbal conversations we make use of voice, eyes, and various body gestures. Also effective human-computer interaction involves hands, eyes, and voice, if available. Therefore by combining multi-sensory modalities, we can make the whole process more natural and ensure enhanced performance even for the disabled users. Towards this end, we have developed a multimodal human-computer interface (HCI) by combining an eyetracker with a soft-switch which may be considered as typically representing another modality. This multi-modal HCI is applied for text entry using a virtual keyboard appropriately designed in-house, facilitating enhanced performance. Our experimental results demonstrate that using multi-modalities for text entry through the virtual keyboard is more efficient and less strenuous than single modality system and also solves the Midas-touch problem, which is inherent in an eye-tracker based HCI system where only dwell time is used for selecting a character.
---
paper_title: A Fitts Law comparison of eye tracking and manual input in the selection of visual targets

paper_content:

We present a Fitts' Law evaluation of a number of eye tracking and manual input devices in the selection of large visual targets. We compared performance of two eye tracking techniques, manual click and dwell time click, with that of mouse and stylus. Results show eye tracking with manual click outperformed the mouse by 16%, with dwell time click 46% faster. However, eye tracking conditions suffered a high error rate of 11.7% for manual click and 43% for dwell time click conditions. After Welford correction eye tracking still appears to outperform manual input, with IPs of 13.8 bits/s for dwell time click, and 10.9 bits/s for manual click. Eye tracking with manual click provides the best tradeoff between speed and accuracy, and was preferred by 50% of participants. Mouse and stylus had IPs of 4.7 and 4.2 respectively. However, their low error rate of 5% makes these techniques more suitable for refined target selection.
---
paper_title: Blink and click

paper_content:

In this project, we have developed the computer assisting tool for persons who have hand moving problems. This software tool will help them to use a computer for those who cannot use mouse and keyboard properly. The tool detects human eyes and face actions to control mouse and keyboard as a pointer and input devices. Our project uses inexpensive equipment, e.g. web cam. Face moving can control mouse pointing position in any directions on a computer screen. Eyes blinking can click a mouse: left or right. However, if face moving is too fast, the tool may not detect the right direction. Another problem, un-intention of eye blinking may make a mouse click.
---
paper_title: Gazing and frowning as a new human--computer interaction technique

paper_content:

The present aim was to study a new technique for human--computer interaction. It combined the use of two modalities, voluntary gaze direction and voluntary facial muscle activation for object pointing and selection. Fourteen subjects performed a series of pointing tasks with the new technique and with a mouse. At short distances the mouse was significantly faster than the new technique. However, there were no statistically significant differences at medium and long distances between the techniques. Fitts' law analyses were performed both by using only error-free trials and using also data including error trials (i.e., effective target width). In all cases both techniques seemed to follow Fitts' law, although for the new technique the effective target width correlation coefficient was smaller R = 0.776 than for the mouse R = 0.991. The regression slopes suggested that at very long distances (i.e., beyond 800 pixels) the new technique might be faster than the mouse. The new technique showed promising results already after a short practice and in the future it could be useful especially for physically challenged persons.
---
paper_title: Communication via eye blinks and eyebrow raises: video-based human-computer interfaces

paper_content:

Two video-based human-computer interaction tools are introduced that can activate a binary switch and issue a selection command. "BlinkLink," as the first tool is called, automatically detects a user's eye blinks and accurately measures their durations. The system is intended to provide an alternate input modality to allow people with severe disabilities to access a computer. Voluntary long blinks trigger mouse clicks, while involuntary short blinks are ignored. The system enables communication using "blink patterns:" sequences of long and short blinks which are interpreted as semiotic messages. The second tool, "EyebrowClicker," automatically detects when a user raises his or her eyebrows and then triggers a mouse click. Both systems can initialize themselves, track the eyes at frame rate, and recover in the event of errors. No special lighting is required. The systems have been tested with interactive games and a spelling program. Results demonstrate overall detection accuracy of 95.6% for BlinkLink and 89.0% for EyebrowClicker.
---
paper_title: EyePoint: practical pointing and selection using gaze and keyboard

paper_content:

We present a practical technique for pointing and selection using a combination of eye gaze and keyboard triggers. EyePoint uses a two-step progressive refinement process fluidly stitched together in a look-press-look-release action, which makes it possible to compensate for the accuracy limitations of the current state-of-the-art eye gaze trackers. While research in gaze-based pointing has traditionally focused on disabled users, EyePoint makes gaze-based pointing effective and simple enough for even able-bodied users to use for their everyday computing tasks. As the cost of eye gaze tracking devices decreases, it will become possible for such gaze-based techniques to be used as a viable alternative for users who choose not to use a mouse depending on their abilities, tasks and preferences.
---
paper_title: Object selection in gaze controlled systems: What you don't look at is what you get

paper_content:

Controlling computers using eye movements can provide a fast and efficient alternative to the computer mouse. However, implementing object selection in gaze-controlled systems is still a challenge. Dwell times or fixations on a certain object typically used to elicit the selection of this object show several disadvantages. We studied deviations of critical thresholds by an individual and task-specific adaptation method. This demonstrated an enormous variability of optimal dwell times. We developed an alternative approach using antisaccades for selection. For selection by antisaccades, highlighted objects are copied to one side of the object. The object is selected when fixating to the side opposed to that copy requiring to inhibit an automatic gaze shift toward new objects. Both techniques were compared in a selection task. Two experiments revealed superior performance in terms of errors for the individually adapted dwell times. Antisaccades provide an alternative approach to dwell time selection, but they did not show an improvement over dwell time. We discuss potential improvements in the antisaccade implementation with which antisaccades might become a serious alternative to dwell times for object selection in gaze-controlled systems.
---
paper_title: A Myographic-based HCI Solution Proposal for Upper Limb Amputees☆

paper_content:

Abstract Interaction plays a fundamental role as it sets bridges between humans and computers. However, people with disability are prevented to use computers by the ordinary means, due to physical or intellectual impairments. Thus, the human-computer interaction (HCI) research area has been developing solutions to improve the technological accessibility of impaired people, by enhancing computers and similar devices with the necessary means to attend to the different disabilities, thereby contributing to reduce digital exclusion. Within the aforementioned scope, this paper presents an interaction solution for upper limb amputees, supported on a myographic gesture-control device named Myo. This device is an emergent wearable technology, which consists in a muscle-sensitive bracelet. It transmits myographic and inertial data, susceptible of being converted into actions for interaction purposes (e.g. clicking or moving a mouse cursor). Although being a gesture control armband, Myo can also be used in the legs, as was ascertained through some preliminary tests with users. Both data types (myographic and inertial) remain to be transmitted and are available to be converted into gestures. A general architecture, a use case diagram and the two main functional modules specification are presented. These will guide the future implementation of the proposed Myo-based HCI solution, which is intended to be a solid contribution for the interaction between upper limb amputees and computers.
---
paper_title: TOU-AR:Touchable Interface for Interactive Interaction in Augmented Reality Environment

paper_content:

Touchable interface is one of the future interfaces that can be implemented at any medium such as water, table or even sand. The word multi touch refers to the ability to distinguish between two or more fingers touching a touch-sensing surface, such as a touch screen or a touch pad. This interface is provided tracking the area by using depth camera and projected the interface into the medium. This interface is widely used in augmented reality environment. User will project the particular interface into real world medium and user hand will be tracked simultaneously when touching the area. User can interact in more freely ways and as natural as human did in their daily life
---
paper_title: Iris tracking and blink detection for human-computer interaction using a low resolution webcam

paper_content:

This paper proposes a technique to develop a non-intrusive interface for human-computer interaction based on visual cues of a human eye, captured using a low resolution USB camera. Such an interface can replace a traditional interface like a mouse thus helping the severely disabled to use the computer just as a normal individual would. It makes use of iris tracking and blink detection for this purpose. Considering fairly constant lighting conditions, a modified Hough Transform is used to track the iris at real time. The Between-The-Eyes feature serves as a stable reference point for gaze direction estimation. When mapped to the computer screen, incremental, directional movement of the mouse pointer position provides precise control over its movement. A method based on the projection function is proposed to detect the user's eye blinks, analyze their patterns and duration, interpreting them as appropriate mouse clicks. The system gives an overall accuracy of 87.4 %.
---
paper_title: BlinkWrite: efficient text entry using eye blinks

paper_content:

In this paper, a new text entry system is proposed, implemented, and evaluated. BlinkWrite provides a communication gateway for cognitively able motor-impaired individuals who cannot use a traditional eye-tracking system. In contrast to most hands-free systems, BlinkWrite allows text to be entered and corrected using a single input modality: blinks. The system was implemented using a scanning ambiguous keyboard, a new form of scanning keyboard that allows English text to be entered in less than two scanning intervals per character. In a user study, 12 participants entered text using the system with three settings for scanning interval: 1,000, 850, and 700 ms. An average text entry rate of 4.8 wpm was observed with accuracy >97%. The highest average text entry rate was achieved with the scanning interval of 850 ms.
---
paper_title: Real-time eye blink and wink detection for object selection in HCI systems

paper_content:

This paper presents an approach for real-time detection of three types of eye blinks: eye blink (blinking both eyes simultaneously), left and right winks. The process of blink detection has been divided into four parts viz. face localization in facial images acquired through a video camera, eye pair localization, pixels’ motion analysis using optical flow technique, and classification of eye blinks. Blink detection has been performed using a video camera and MATLAB software with image processing and computer vision toolbox. The algorithm takes about 60 ms time for processing a frame and 250 ms time for confirmation and classification of the detected blink. An experiment was conducted to evaluate the performance of the proposed approach in which 10 users voluntarily participated. The performance of the proposed method has been tested under two lighting conditions: natural lighting conditions and controlled lighting conditions. Also, the performance has been tested by varying the distance of the user from the camera. Here, it is observed that the system gives best performance when used under controlled lighting conditions and the user sitting at a distance of about 0.5 m. Accuracy of the proposed approach has been found to be 96, 92 and 88% for detection of eye blink, left wink and right wink, respectively. The proposed method has also been tested on ZJU dataset where it has given precision, detection accuracy and false alarm rate of values 94.11, 91.2 and 1.54%, respectively. The proposed system has been used and evaluated for performing various mouse analogous functions using eye blinks and winks. It has given an accuracy of 90, 80 and 90% in performing left click, double click, and right click operations, respectively.
---
paper_title: Face and Eyes mouse for ALS Patients

paper_content:

The objective of this project is to develop the software to detect the Amyotrophic Lateral Sclerosis (ALS) patient's face and eyes movement, so that the patient is able to use computer as normal people. The developed system is called Face and Eyes mouse for ALS Patients or FEMA. The system is divided into 2 parts, client part and system part. The client part is ALS patients who are able to use mouse and keyboard simulation by using only their head and eyes' movement. Another part is FEMA system which will analyze their actions and return the action result to the patient. In addition, there is the keyboard simulation which provides the similar function as the real keyboard. From the experimental results, the precision rate of all functions except move up and move down is more than 80%. We also found that the important factors of the detection result are brightness and camera quality which are most sensitive to the detection accuracy.
---
paper_title: Blink and click

paper_content:

In this project, we have developed the computer assisting tool for persons who have hand moving problems. This software tool will help them to use a computer for those who cannot use mouse and keyboard properly. The tool detects human eyes and face actions to control mouse and keyboard as a pointer and input devices. Our project uses inexpensive equipment, e.g. web cam. Face moving can control mouse pointing position in any directions on a computer screen. Eyes blinking can click a mouse: left or right. However, if face moving is too fast, the tool may not detect the right direction. Another problem, un-intention of eye blinking may make a mouse click.
---
paper_title: Blink and wink detection for mouse pointer control

paper_content:

A Human-Computer Interaction (HCI) system that is designed for individuals with severe disabilities to simulate control of a traditional computer mouse is introduced. The camera-based system monitors a user's eyes and allows the user to simulate clicking the mouse using voluntary blinks and winks. For users who can control head movements and can wink with one eye while keeping their other eye visibly open, the system allows complete use of a typical mouse, including moving the pointer, left and right clicking, double clicking, and click-and-dragging. For users who cannot wink but can blink voluntarily the system allows the user to perform left clicks, the most common and useful mouse action. The system does not require any training data to distinguish open eyes versus closed eyes. Eye classification is accomplished online during real-time interactions. The system had an accuracy of 8027/8306 = 96.6% in classifying sub-images with open or closed eyes and successfully allows the users to simulate a traditional computer mouse.
---
paper_title: BlinkWrite: efficient text entry using eye blinks

paper_content:

In this paper, a new text entry system is proposed, implemented, and evaluated. BlinkWrite provides a communication gateway for cognitively able motor-impaired individuals who cannot use a traditional eye-tracking system. In contrast to most hands-free systems, BlinkWrite allows text to be entered and corrected using a single input modality: blinks. The system was implemented using a scanning ambiguous keyboard, a new form of scanning keyboard that allows English text to be entered in less than two scanning intervals per character. In a user study, 12 participants entered text using the system with three settings for scanning interval: 1,000, 850, and 700 ms. An average text entry rate of 4.8 wpm was observed with accuracy >97%. The highest average text entry rate was achieved with the scanning interval of 850 ms.
---
paper_title: BlinkWrite2: an improved text entry method using eye blinks

paper_content:

Areas of design improvements for BlinkWrite, an eye blink text entry system, are examined, implemented, and evaluated. The result, BlinkWrite2, is a text entry system for individuals with severe motor impairment. Since the ability to blink is often preserved, even in severe conditions such as locked-in syndrome, BlinkWrite2 allows text entry and correction with blinks as the only input modality. Advantages of BlinkWrite2 over its predecessor include an increase in text entry speed. In a user evaluation, 12 participants achieved an average text entry rate of 5.3 wpm, representing a 16% increase over BlinkWrite and a 657% increase over the next fastest video-based eye blink text entry system reported in the literature.
---
paper_title: Eye typing using word and letter prediction and a fixation algorithm

paper_content:

Two eye typing techniques and a fixation algorithm are described. Similar to word prediction, letter prediction chooses three highly probable next letters and highlights them on an onscreen keyboard. Letter prediction proved promising, as it was as good as word prediction, and in some cases better. The fixation algorithm chooses which button to select for eye-over highlighting. It often chooses the desired button even if another button is closer to the fixation location. Error rates were reduced when using the fixation algorithm combined with letter prediction; however, the algorithm was sensitive to the correctness of the first several letters in a word.
---
paper_title: System for assisted mobility using eye movements based on electrooculography

paper_content:

This paper describes an eye-control method based on electrooculography (EOG) to develop a system for assisted mobility. One of its most important features is its modularity, making it adaptable to the particular needs of each user according to the type and degree of handicap involved. An eye model based on electroculographic signal is proposed and its validity is studied. Several human-machine interfaces (HMI) based on EOG are commented, focusing our study on guiding and controlling a wheelchair for disabled people, where the control is actually effected by eye movements within the socket. Different techniques and guidance strategies are then shown with comments on the advantages and disadvantages of each one. The system consists of a standard electric wheelchair with an on-board computer, sensors and a graphic user interface run by the computer. On the other hand, this eye-control method can be applied to handle graphical interfaces, where the eye is used as a mouse computer. Results obtained show that this control technique could be useful in multiple applications, such as mobility and communication aid for handicapped persons.
---
paper_title: The effect of clicking by smiling on the accuracy of head-mounted gaze tracking

paper_content:

The effect of facial behaviour on gaze tracking accuracy was studied while using a prototype system that integrated head-mounted, video-based gaze tracking and a capacitive facial movement detection for respective pointing and selecting objects in a simple graphical user interface. Experiments were carried out to determine how voluntary smiling movements that were used to indicate clicks affect the accuracy of gaze tracking due to the combination of user eye movement behaviour and the operation of gaze tracking algorithms. The results showed no observable degradation of the gaze tracking accuracy when using voluntary smiling for object selections.
---
paper_title: Typing with eye-gaze and tooth-clicks

paper_content:

In eye-gaze-based human-computer interfaces, the most commonly used mechanism for generating activation commands (i.e., mouse clicks) is dwell time (DT). While DT can be relatively efficient and easy to use, it is also associated with the possibility of generating unintentional activation commands -- an issue that is known as the Midas' touch problem. To address this problem, we proposed to use a "tooth-clicker" (TC) device as a mechanism for generating activation commands independently of the activity of the eyes. This paper describes a pilot study that verifies the feasibility of using an eye-gaze tracker (EGT) and a TC to type on an on-screen keyboard, and compares the performance of the EGT-TC system with that of the EGT with two different DT thresholds (880 ms and 490 ms). The six subjects that participated in the study were able to attain typing speeds using the EGT-TC system that were slower than but comparable to the typing speeds that they attained using the EGT with the shorter DT threshold.
---
paper_title: Designing gaze gestures for gaming: an investigation of performance

paper_content:

To enable people with motor impairments to use gaze control to play online games and take part in virtual communities, new interaction techniques are needed that overcome the limitations of dwell clicking on icons in the games interface. We have investigated gaze gestures as a means of achieving this. We report the results of an experiment with 24 participants that examined performance differences between different gestures. We were able to predict the effect on performance of the numbers of legs in the gesture and the primary direction of eye movement in a gesture. We also report the outcomes of user trials in which 12 experienced gamers used the gaze gesture interface to play World of Warcraft. All participants were able to move around and engage other characters in fighting episodes successfully. Gestures were good for issuing specific commands such as spell casting, and less good for continuous control of movement compared with other gaze interaction techniques we have developed.
---
paper_title: Eye typing in application: A comparison of two systems with ALS patients

paper_content:

A variety of eye typing systems has been developed during the last decades. Such systems can provide support for people who lost the ability to communicate, e.g. patients suffering from motor neuron diseases such as amyotrophic lateral sclerosis (ALS). In the current retrospective analysis, two eye typing applications were tested (EyeGaze, GazeTalk) by ALS patients (N = 4) in order to analyze objective performance measures and subjective ratings. An advantage of the EyeGaze system was found for most of the evaluated criteria. The results are discussed in respect of the special target population and in relation to requirements of eye tracking devices.
---
paper_title: Eye-based user interfaces: Some recent projects

paper_content:

We are currently witnessing an increasing attention to issues related to Accessibility, which should eliminate, or at least reduce, the distance between disabled people and technology. However, particularly for severely-impaired persons, there are still many challenges that must be overcome. In this paper we present eye tracking as a valuable support for disability in the accomplishment of hands-free tasks. Moreover, we stress the potentials of eye-based interfaces to enhance the user-machine interaction process in “traditional” activities based on keyboard and mouse. Through the description of some of the projects we have recently developed at the University of Pavia, we will show how interfaces based on eye tracking can be really helpful in different contexts of use.
---
paper_title: Vision-Based Interface: Using Face and Eye Blinking Tracking with Camera

paper_content:

To provide new type of interaction between human-machine, many devices which could substitute input equipments (e.g. mouse, keyboard etc.) have been invented. In this paper, we develop a novel robust camera mouse based on common Webcam which could be used by handicapped, or be used in some special application. We choose common Webcam to build the device in order to ensure the universality of our design. Combination between optical flow and Harris corner detector obtains good result, and an approach based on tracking fix feature point is used to detect eye blinking. The application can be applied to various illuminations even in a house with no lamp but only screen light. At the same time, this method just takes up less than 30% of system resource. Based on our approach we will present a library from which anybody could develop more application.
---
paper_title: Using the Optical Flow to Implement a Relative Virtual Mouse Controlled by Head Movements

paper_content:

The following paper introduces the work conducted to create a relative virtual mouse based on the interpretation of head movements and face gesture through a low cost camera and the optical flow of the images. This virtual device is designed specifically as an alternative non- contact pointer for people with mobility impairments in the upper extremities and reduced head control. The proposed virtual device was compared with a conventional mouse, a touchpad and a digital joystick. Validation results show performances close to a digital joystick but far away from a conventional mouse.
---
paper_title: Communication via eye blinks and eyebrow raises: video-based human-computer interfaces

paper_content:

Two video-based human-computer interaction tools are introduced that can activate a binary switch and issue a selection command. "BlinkLink," as the first tool is called, automatically detects a user's eye blinks and accurately measures their durations. The system is intended to provide an alternate input modality to allow people with severe disabilities to access a computer. Voluntary long blinks trigger mouse clicks, while involuntary short blinks are ignored. The system enables communication using "blink patterns:" sequences of long and short blinks which are interpreted as semiotic messages. The second tool, "EyebrowClicker," automatically detects when a user raises his or her eyebrows and then triggers a mouse click. Both systems can initialize themselves, track the eyes at frame rate, and recover in the event of errors. No special lighting is required. The systems have been tested with interactive games and a spelling program. Results demonstrate overall detection accuracy of 95.6% for BlinkLink and 89.0% for EyebrowClicker.
---
paper_title: Blink and wink detection for mouse pointer control

paper_content:

A Human-Computer Interaction (HCI) system that is designed for individuals with severe disabilities to simulate control of a traditional computer mouse is introduced. The camera-based system monitors a user's eyes and allows the user to simulate clicking the mouse using voluntary blinks and winks. For users who can control head movements and can wink with one eye while keeping their other eye visibly open, the system allows complete use of a typical mouse, including moving the pointer, left and right clicking, double clicking, and click-and-dragging. For users who cannot wink but can blink voluntarily the system allows the user to perform left clicks, the most common and useful mouse action. The system does not require any training data to distinguish open eyes versus closed eyes. Eye classification is accomplished online during real-time interactions. The system had an accuracy of 8027/8306 = 96.6% in classifying sub-images with open or closed eyes and successfully allows the users to simulate a traditional computer mouse.
---
paper_title: Efficient Eye Blink Detection Method for disabled- helping domain

paper_content:

In this paper, we present a real time method based on some video and image processing algorithms for eye blink detection. The motivation of this research is the need of disabling who cannot control the calls with human mobile interaction directly without the need of hands. A Haar Cascade Classifier is applied for face and eye detection for getting eye and facial axis information. In addition, the same classifier is used based on Haar- like features to find out the relationship between the eyes and the facial axis for positioning the eyes. An efficient eye tracking method is proposed which uses the position of detected face. Finally, an eye blinking detection based on eyelids state (close or open) is used for controlling android mobile phones. The method is used with and without smoothing filter to show the improvement of detection accuracy. The application is used in real time for studying the effect of light and distance between the eyes and the mobile device in order to evaluate the accuracy detection and overall accuracy of the system. Test results show that our proposed method provides a 98% overall accuracy and 100% detection accuracy for a distance of 35 cm and an artificial light.
---
paper_title: A new interaction technique involving eye gaze tracker and scanning system

paper_content:

This paper presents a new input interaction system for people with severe disabilities by combining eye gaze tracking and single switch scanning interaction techniques. The system is faster than only scanning based systems while more comfortable to use than existing eye gaze tracking based systems. We reported results from a couple of user studies that show the new system is equally fast compared to existing eye tracking systems that does not involve scanning, participants with no prior experience with eye tracking based system could learn using this new system successfully within 10 minutes but it demands higher mental effort in comparison to another new modality of interaction- a gesture based system.
---
paper_title: Eye/Head Tracking Technology to Improve HCI with iPad Applications

paper_content:

In order to improve human computer interaction (HCI) for people with special needs, this paper presents an alternative form of interaction, which uses the iPad's front camera and eye/head tracking technology. With this functional nature/capability operating in the background, the user can control already developed or new applications for the iPad by moving their eyes and/or head. There are many techniques, which are currently used to detect facial features, such as eyes or even the face itself. Open source bookstores exist for such purpose, such as OpenCV, which enable very reliable and accurate detection algorithms to be applied, such as Haar Cascade using very high-level programming. All processing is undertaken in real time, and it is therefore important to pay close attention to the use of limited resources (processing capacity) of devices, such as the iPad. The system was validated in tests involving 22 users of different ages and characteristics (people with dark and light-colored eyes and with/without glasses). These tests are performed to assess user/device interaction and to ascertain whether it works properly. The system obtained an accuracy of between 60% and 100% in the three test exercises taken into consideration. The results showed that the Haar Cascade had a significant effect by detecting faces in 100% of cases, unlike eyes and the pupil where interference (light and shade) evidenced less effectiveness. In addition to ascertaining the effectiveness of the system via these exercises, the demo application has also helped to show that user constraints need not affect the enjoyment and use of a particular type of technology. In short, the results obtained are encouraging and these systems may continue to be developed if extended and updated in the future.
---
paper_title: Alternatives to single character entry and dwell time selection on eye typing

paper_content:

Eye typing could provide motor disabled people a reliable method of communication given that the text entry speed of current interfaces can be increased to allow for fluent communication. There are two reasons for the relatively slow text entry: dwell time selection requires waiting a certain time, and single character entry limits the maximum entry speed. We adopted a typing interface based on hierarchical pie menus, pEYEwrite [Urbina and Huckauf 2007] and included bigram text entry with one single pie iteration. Therefore, we introduced three different bigram building strategies. Moreover, we combined dwell time selection with selection by borders, providing an alternative selection method and extra functionality. In a longitudinal study we compared participants performance during character-by-character text entry with bigram entry and with text entry with bigrams derived by word prediction. Data showed large advantages of the new entry methods over single character text entry in speed and accuracy. Participants preferred selecting by borders, which allowed them faster selections than the dwell time method.
---
paper_title: Now Dasher! Dash away!: longitudinal study of fast text entry by Eye Gaze

paper_content:

Dasher is one of the best known inventions in the area of text entry in recent years. It can be used with many input devices, but studies on user performance with it are still scarce. We ran a longitudinal study where 12 participants transcribed Finnish text with Dasher in ten 15-minute sessions using a Tobii 1750 eye tracker as a pointing device. The mean text entry rate was 2.5 wpm during the first session and 17.3 wpm during the tenth session. Our results show that very high text entry rates can be achieved with eye-operated Dasher, but only after several hours of training.
---
paper_title: Improving eye cursor's stability for eye pointing tasks

paper_content:

In order to improve the stability of eye cursor, we introduce three methods, force field (FF), speed reduction (SR), and warping to target center (TC) to modulate eye cursor trajectories by counteracting eye jitter, which is the main cause of destabilizing the eye cursor. We evaluate these methods using two controlled experiments. One is an attention task experiment, which indicates that both FF and SR significantly alleviate the instability of eye cursor, but TC is not as we anticipated. The other is a 2D pointing task experiment, which shows that FF and SR as well as the improved implementation of SR (iSR) indeed improve human performance in dominant dwell-based eye pointing tasks of eye-based interactions. The method iSR is especially effective to accelerate eye pointing (10.5% and 8.5%) and reduce error rate (6.1% and 2.7%) when target diameter D = 45 and 60 pixels.
---
paper_title: BlinkWrite: efficient text entry using eye blinks

paper_content:

In this paper, a new text entry system is proposed, implemented, and evaluated. BlinkWrite provides a communication gateway for cognitively able motor-impaired individuals who cannot use a traditional eye-tracking system. In contrast to most hands-free systems, BlinkWrite allows text to be entered and corrected using a single input modality: blinks. The system was implemented using a scanning ambiguous keyboard, a new form of scanning keyboard that allows English text to be entered in less than two scanning intervals per character. In a user study, 12 participants entered text using the system with three settings for scanning interval: 1,000, 850, and 700 ms. An average text entry rate of 4.8 wpm was observed with accuracy >97%. The highest average text entry rate was achieved with the scanning interval of 850 ms.
---

Title: Object Acquisition and Selection in Human Computer Interaction Systems: A Review

Section 1: Introduction
Description 1: This section provides an overview of human-computer interaction (HCI) systems, their components, and the importance of designing user-friendly interfaces for both technical and non-technical users.

Section 2: Object Acquisition, Object Selection and Feedback Modes
Description 2: This section explains the techniques used for object acquisition and selection in HCI systems and discusses the types of feedback used to enhance interaction efficiency and user-friendliness.

Section 3: Object Acquisition
Description 3: This section describes various methods for object acquisition including gaze tracking, facial feature tracking, tongue movement control, eye typing with menus, scanning method, and hybrid approaches.

Section 4: Object Selection
Description 4: This section details the different selection triggers used to select objects after acquisition, such as key triggers, dwell time, eye blinking, on-off screen buttons, and other innovative methods.

Section 5: Object Acquisition & Selection Techniques as a Combination in HCI Systems
Description 5: This section discusses the importance of combining object acquisition and selection techniques based on user needs and capabilities, examining different combinations and their impacts on system performance.

Section 6: Object Selection in Virtual/Augmented Reality Environments
Description 6: This section explores object acquisition and selection techniques in virtual and augmented reality environments, highlighting their applications and specific triggering methods used.

Section 7: Mouse Analogous Operations
Description 7: This section reviews techniques that perform mouse operations such as right click, double click, drag-drop, and cursor movement, stressing the need for comprehensive solutions.

Section 8: Feedback Mechanism
Description 8: This section examines various feedback mechanisms used in HCI systems, including audio, visual, and haptic feedback, and their importance in interactive environments.

Section 9: Applications of Object Acquisition and Selection Techniques in Vision Based HCI Systems
Description 9: This section lists major applications of vision-based HCI systems for differently abled and aged persons as well as normal users for increased comfort.

Section 10: Issues Associated to Object Acquisition and Selection Techniques
Description 10: This section addresses the challenges and limitations related to object acquisition and selection techniques, such as accuracy, fatigue, cursor stability, and response time.

Section 11: Conclusions & Future Scope
Description 11: This section summarizes the review findings, highlighting the need for cost-effective and comprehensive solutions for object acquisition and selection in HCI systems, and discussing future research directions.

Section 12: Conflict of Interest
Description 12: This section declares any potential conflicts of interest related to the paper.

Section 13: Funding
Description 13: This section acknowledges the funding sources or declares the absence of specific grants for the conducted research.