Video based technology for ambient assisted living: A review of the literature
---
paper_title: Development of a mobile user interface for image-based dietary assessment

paper_content:

In this paper, we present a mobile user interface for image-based dietary assessment. The mobile user interface provides a front end to a client-server image recognition and portion estimation software. In the client-server configuration, the user interactively records a series of food images using a built-in camera on the mobile device. Images are sent from the mobile device to the server, and the calorie content of the meal is estimated. In this paper, we describe and discuss the design and development of our mobile user interface features. We discuss the design concepts, through initial ideas and implementations. For each concept, we discuss qualitative user feedback from participants using the mobile client application. We then discuss future designs, including work on design considerations for the mobile application to allow the user to interactively correct errors in the automatic processing while reducing the user burden associated with classical pen-and-paper dietary records.
---
paper_title: Activity summarisation and fall detection in a supportive home environment

paper_content:

Automatic semantic summarisation of human activity and detection of unusual inactivity are useful goals for a vision system operating in a supportive home environment. Learned models of spatial context are used in conjunction with a tracker to achieve these goals. The tracker uses a coarse ellipse model and a particle filter to cope with cluttered scenes with multiple sources of illumination. Summarisation in terms of semantic regions is demonstrated using acted scenes through automatic recovery of the instructions given to the actor. The use of 'unusual inactivity' detection as a cue for fall detection is also demonstrated.
---
paper_title: Self management of stroke supported by assistive technology

paper_content:

This presentation will describe our study which aims to examine the principles of technology-supported self management in the rehabilitation of stroke as a long term condition by integrating the mixed qualitative research and user-centered design methods.
---
paper_title: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation

paper_content:

Remote measurements of the cardiac pulse can provide comfortable physiological assessment without electrodes. However, attempts so far are non-automated, susceptible to motion artifacts and typically expensive. In this paper, we introduce a new methodology that overcomes these problems. This novel approach can be applied to color video recordings of the human face and is based on automatic face tracking along with blind source separation of the color channels into independent components. Using Bland-Altman and correlation analysis, we compared the cardiac pulse rate extracted from videos recorded by a basic webcam to an FDA-approved finger blood volume pulse (BVP) sensor and achieved high accuracy and correlation even in the presence of movement artifacts. Furthermore, we applied this technique to perform heart rate measurements from three participants simultaneously. This is the first demonstration of a low-cost accurate video-based method for contact-free heart rate measurements that is automated, motion-tolerant and capable of performing concomitant measurements on more than one person at a time.
---
paper_title: Monocular 3D Head Tracking to Detect Falls of Elderly People

paper_content:

Faced with the growing population of seniors, Western societies need to think about new technologies to ensure the safety of elderly people at home. Computer vision provides a good solution for healthcare systems because it allows a specific analysis of people behavior. Moreover, a system based on video surveillance is particularly well adapted to detect falls. We present a new method to detect falls using a single camera. Our approach is based on the 3D trajectory of the head, which allows us to distinguish falls from normal activities using 3D velocities. I. INTRODUCTION As other Western countries, Canada's population is grow- ing older. According to the Public Health Agency of Canada (7), one Canadian out of eight was older than 65 years old in 2001. In 2026, this proportion will be one out of five. Moreover, in 1996, 93% of all seniors resided in private households, and among them, 29% lived alone (7). Faced this reality, new technologies are developed to help them live in a more secure environment at home. Falls are one of the most dangerous situation at home. Almost 62% of injury-related hospitalizations for seniors are the result of falls (8). The first concerned are older people living alone because the situation can be aggravated if they cannot call for help, being unconscious or immobilized. Nowadays, the favored solution is to use wearable fall detectors like accelerometers or help buttons. However, older people often forget to wear them, and a help button is useless if the person is unconscious after the fall. Computer vision systems offer a new solution for fall detection which overcome these limitations. Some research has been done to detect falls using image sensors. The easiest method to detect a fall is based on the shape of the person's silhouette or bounding box, but this method can be inaccurate, depending on the relative position of the person, camera, and perhaps occluding objects. Indeed, if the camera is placed sideways, the point of view can be affected by object occlusions. To overcome this problem, some researchers put the camera in the ceiling. For instance, Lee and Mihailidis (11) detect a fall using the shape of the person's silhouette, and Nait-Charif and McKenna (12) detect inactivity outside the normal zones of inactivity like chairs or sofas. Sixsmith and Johnson (15) use an infrared sensor This work was supported by the Natural Sciences and Engineering
---
paper_title: Context aware inactivity recognition for visual fall detection

paper_content:

This paper introduces a method for automatic fall detection, targeted towards the monitoring of elderly people in a nursing home or in their natural home environment. The method uses information extracted from images obtained using novel 3D camera technology, combined with a context model. Visual information consists of body orientation calculated from posture extraction and of periods of inactivity. The context model allows for a different interpretation of the visual fall detection results, depending on the exact location, time and duration of the detected event. The context model is learnt during the ongoing monitoring task without human intervention and automatically adapts to the changing activity patterns of the monitored subject.
---
paper_title: Quickest change detection for health-care video surveillance

paper_content:

Detecting changes in video scenes is of fundamental importance for various video surveillance tasks. Of particular interest are abnormal changes of foreground human behaviors/activities that could pose damages or dangers to human properties and lives. In this paper, we propose a unified sequential approach to detecting, as soon as possible, human fall incidents for health-care purpose. Specifically, aspect ratio of human body is extracted as the representative feature, based on which an event-inference module parses observed feature sequences for possible falling behavioral signs. Experimental results are reported to show the efficacy of the proposed approach.
---
paper_title: Automatic Fall Incident Detection in Compressed Video for Intelligent Homecare

paper_content:

This paper presents a compressed-domain fall incident detection scheme for intelligent homecare applications. First, a compressed-domain object segmentation scheme is performed to extract moving objects based on global motion estimation and local motion clustering. After detecting the moving objects, three compressed-domain features of each object are then extracted for identifying and locating fall incidents. The proposed system can differentiate fall-down from squatting by taking into account the event duration. Our experiments show that the proposed method can correctly detect fall incidents in real time.
---
paper_title: An eigenspace-based approach for human fall detection using Integrated Time Motion Image and Neural Network

paper_content:

Falls are a major health hazard for the elderly and a serious obstacle for independent living. Since falling causes dramatic physical-psychological consequences, development of intelligent video surveillance systems is so important due to providing safe environments. To this end, this paper proposes a novel approach for human fall detection based on combination of integrated time motion images and eigenspace technique. Integrated time motion image (ITMI) is a type of spatio-temporal database that includes motion and time of motion occurrence. Applying eigenspace technique to ITMIs leads in extracting eigen-motion and finally MLP Neural Network is used for precise classification of motions and determination of a fall event. Unlike existent fall detection systems only deal with limited movement patterns, we considered wide range of motions consisting normal daily life activities, abnormal behaviors and also unusual events. Reliable recognition rate of experimental results underlines satisfactory performance of our system.
---
paper_title: An intelligent emergency response system: preliminary development and testing of automated fall detection

paper_content:

We have designed an intelligent emergency response system to detect falls in the home. It uses image-based sensors. A pilot study was conducted using 21 subjects to evaluate the efficacy and performance of the fall-detection component of the system. Trials were conducted in a mock-up bedroom setting, with a bed, a chair and other typical bedroom furnishings. A small digital videocamera was installed in the ceiling at a height of approximately 2.6 m. The digital camera covered an area of approximately 5.0 m × 3.8 m. The subjects were asked to assume a series of postures, namely walking/standing, sitting/lying down in an inactive zone, stooping, lying down in a 'stretched' position, and lying down in a 'tucked' position. These five scenarios were repeated three times by each subject in a random order. These test positions totalled 315 tasks with 126 fall-simulated tasks and 189 non-fall-simulated tasks. The system detected a fall on 77% of occasions and missed a fall on 23%. False alarms occurred on only 5%...
---
paper_title: Fall Incidents Detection for Intelligent Video Surveillance

paper_content:

We present in this paper an intelligent video surveillance system to detect human fall incidents for enhanced safety in indoor environments. The system consists of two main parts: a vision component which can reliably detect and track moving people in the view of a camera, and an event-inference module which parses observation sequences of people features for possible falling behavioral signs. In particular, we extract the aspect ratio of a person as observation feature, based on which fall incidents are detected as abrupt changes in the feature space. Our experiments show that the proposed approach can robustly detect human falls in real time
---
paper_title: An intelligent videomonitoring system for fall detection at home: perceptions of elderly people

paper_content:

We explored the perception and receptivity of elderly people regarding the introduction of an intelligent videomonitoring system (IVS) at home. Using a mixed methods design, 25 elderly people with a history of falls completed a structured interview (two questionnaires). In the year preceding the interview, 72% of the participants fell as many as seven times. Open-ended questions (qualitative data) were used to supplement the quantitative data. A content analysis (qualitative data) and a descriptive analysis (quantitative data) were carried out. The participants were 84% favourable or partially favourable to technologies which ensured home security and 96% favourable or partially favourable to the IVS. About half (48%) said that they would use it. The other participants did not wish to use it unless they had been left to live alone or if their health condition worsened. Thus the living conditions of the elderly appear to influence their perception and receptivity regarding the acceptance and use of an IVS.
---
paper_title: Fall Detection from Human Shape and Motion History Using Video Surveillance

paper_content:

Nowadays, Western countries have to face the growing population of seniors. New technologies can help people stay at home by providing a secure environment and improving their quality of life. The use of computer vision systems offers a new promising solution to analyze people behavior and detect some unusual events. In this paper, we propose a new method to detect falls, which are one of the greatest risk for seniors living alone. Our approach is based on a combination of motion history and human shape variation. Our algorithm provides promising results on video sequences of daily activities and simulated falls.
---
paper_title: A Smart and Passive Floor-Vibration Based Fall Detector for Elderly

paper_content:

Falls are very prevalent among the elderly. They are the second leading cause of unintentional-injury death for people of all ages and the leading cause of death for elders 79 years and older. Studies have shown that the medical outcome of a fall is largely dependent upon the response and rescue time. Hence, a highly accurate automatic fall detector is an important component of the living setting for older adult to expedite and improve the medical care provided to this population. Though there are several kinds of fall detectors currently available, they suffer from various drawbacks. Some of them are intrusive while others require the user to wear and activate the devices, and hence may fail in the event of user non-compliance. This paper describes the working principle and the design of a floor vibration-based fall detector that is completely passive and unobtrusive to the resident. The detector was designed to overcome some of the common drawbacks of the earlier fall detectors. The performance of the detector is evaluated by conducting controlled laboratory tests using anthropomorphic dummies. The results showed 100% fall detection rate with minimum potential for false alarms
---
paper_title: Detection of falls at home using floor vibrations and sound

paper_content:

Falls are very prevalent among the elderly especially in their home. Approximately one in every three adults 65 years old or older falls each year, 30% of those falls result in serious injuries and more than 70% of the event are at home. Rapid detection of fall events can reduce the rate of mortality and raise the chances to survive the event and return to independent living. In this paper we present an innovative automatic system for detection of elderly people falls at home. The system is based on floor vibration and acoustic sensing, and uses pattern recognition algorithm to discriminate between human fall events and other events. The proposed solution is unique, inexpensive, and does not require the person to wear anything. Using the proposed system we can detect human falls with a sensitivity of 97.5% and specificity of 98.5%.
---
paper_title: Intelligent Video Monitoring to Improve Safety of Older Persons

paper_content:

This paper presents the application of computer vision and machine learning technologies to a clinical task of paramount importance, improving safety of older persons. We propose an intelligent monitoring system equipped with a camera network and an automatic elopement detection algorithm to reduce the risks of un-witnessed elopements from a dementia unit in order to avoid their potential catastrophic consequences. The camera network employs 23 cameras to record daily activities in our test bed, which includes 15 residents, 4 registered and licensed practical nurses and a number of certified nursing assistants. An elopement detector is then built by using computer vision algorithms and a machine learning algorithm to automatically detect elopements and alert caregivers. The experiments demonstrate that the proposed system leverages the advantages of monitoring from multiple cameras and is able to detect elopements with almost 100% accuracy.
---
paper_title: A HHMM-Based Approach for Robust Fall Detection

paper_content:

Automatic detection of a falling person in video sequences is an important part of future pervasive home monitoring systems. We propose here a robust method to achieve this goal. Motion is modeled by a Hierarchical Hidden Markov Model (HHMM) whose first layer states are related to the orientation of the tracked person. Finding a consistent way for robustly linking the observation vector to the human poses is the heart of our contribution. In that sense, we carefully study the relationship between angles in the 3D world and their projection onto the image plane. After performing an initial image metric rectification, we derive theoretical properties making it possible to bound the error angle introduced by the image formation process for a standing posture. This allows us to confidently identify other poses as "non-standing" ones, and thus to robustly analyze pose sequences against a given motion model. Several results illustrate the efficiency of the algorithm by pointing out its ability to accurately recognize a person falling down from another walking or sitting, as well as its capacity to run in an unspecified configuration.
---
paper_title: Automatic intruder detection incorporating intelligent scene monitoring with video surveillance

paper_content:

An image processing based approach for the automatic identification of moving objects (primarily people) within monochrome video images is presented. Suspicious movement is distinguished from non-suspicious movement by analysis using object motion algorithms. A human operator can be alerted on recognition of a suspicious event occurrence. The methods used are described along with the results obtained.
---
paper_title: A Method for Automatic Fall Detection of Elderly People Using Floor Vibrations and Sound—Proof of Concept on Human Mimicking Doll Falls

paper_content:

Falls are a major risk for the elderly people living independently. Rapid detection of fall events can reduce the rate of mortality and raise the chances to survive the event and return to independent living. In the last two decades, several technological solutions for detection of falls were published, but most of them suffer from critical limitations. In this paper, we present a proof of concept to an automatic fall detection system for elderly people. The system is based on floor vibration and sound sensing, and uses signal processing and pattern recognition algorithm to discriminate between fall events and other events. The classification is based on special features like shock response spectrum and mel frequency ceptral coefficients. For the simulation of human falls, we have used a human mimicking doll: ldquoRescue Randy.rdquo The proposed solution is unique, reliable, and does not require the person to wear anything. It is designed to detect fall events in critical cases in which the person is unconscious or in a stress condition. From the preliminary research, the proposed system can detect human mimicking dolls falls with a sensitivity of 97.5% and specificity of 98.6%.
---
paper_title: Automatic Fall Detection in Real Time Video Based Applications

paper_content:

In this study, a fall detection system with real time video tracking is implemented to be used where people must be in continuous supervision. Automatic detection of a falling person from a video image is an important problem especially in security and safety applications such as supportive home environments and closed surveillance systems. In this work, first human motion is detected in videos and then tracked. The approach presented here is different from other approaches in the literature. In our work, images grabbed from a continuous video source are stored and if no motion is detected, previously stored still images are evaluated to decide whether the "no motion" condition is a result of a fall
---
paper_title: Socially assistive robotics for post-stroke rehabilitation

paper_content:

BackgroundAlthough there is a great deal of success in rehabilitative robotics applied to patient recovery post stroke, most of the research to date has dealt with providing physical assistance. However, new rehabilitation studies support the theory that not all therapy need be hands-on. We describe a new area, called socially assistive robotics, that focuses on non-contact patient/user assistance. We demonstrate the approach with an implemented and tested post-stroke recovery robot and discuss its potential for effectiveness.ResultsWe describe a pilot study involving an autonomous assistive mobile robot that aids stroke patient rehabilitation by providing monitoring, encouragement, and reminders. The robot navigates autonomously, monitors the patient's arm activity, and helps the patient remember to follow a rehabilitation program. We also show preliminary results from a follow-up study that focused on the role of robot physical embodiment in a rehabilitation context.ConclusionWe outline and discuss future experimental designs and factors toward the development of effective socially assistive post-stroke rehabilitation robots.
---
paper_title: AIBO: Toward the Era of Digital Creatures

paper_content:

The 21st century will become an era of autonomous robots that help and support people. Thus, they will be considered as partners of human beings. In this paper, the author introduces AIBO, the first product model of Robot Entertainment Systems. The main application of this robot is a pet-style robot, which must maintain a lifelike appearance. The author suggests how to maximize the complexity of responses and movements to solve the problem of substantially increasing the lifelike appearance of autonomous robots. The technologies used in AIBO are also described. Although AIBO is not intended for service or hazardous work, the development of AIBO is a major step toward a new era of autonomous robots in the new century.
---
paper_title: Assistive social robots in elderly care: a review

paper_content:

Assistive social robots, a particular type of assistive robotics designed for social interaction with humans, could play an important role with respect to the health and psychological well-being of the elderly. Objectives Assistive social robots are believed to be useful in eldercare for two reasons, a functional one and an affective one. Such robots are developed to function as an interface for the elderly with digital technology, and to help increase the quality of life of the elderly by providing companionship, respectively. There is a growing attention for these devices in the literature. However, no comprehensive review has yet been performed to investigate the effectiveness of such robots in the care of the elderly. Therefore, we systematically reviewed and analyzed existing literature on the effects of assistive social robots in health care for the elderly. We focused in particular on the companion function. Data Sources A systematic search of MEDLINE, CINAHL, PsycINFO, The Cochrane Library databases, IEEE, ACM libraries and finally Google Scholar was performed for records through December 2007 to identify articles of all studies with actual subjects aimed to assess the effects of assistive social robots on the elderly. This search was completed with information derived from personal expertise, contacts and reports. Study Selection and Data Extraction Since no randomized controlled trials (RCT)’s have been found within this field of research, all studies reporting effects of assistive robotics in elderly populations were included. Information on study design, interventions, controls, and findings were extracted for each article. In medical journals only a few articles were found, whereas about 50 publications were found in literature on ICT and robotics. Data Synthesis The identified studies were all published after 2000 indicating the novelty of this area of research. Most of these publications contain the results of studies that report positive effects of assistive social robots on health and psychological well-being of elders. Solid evidence indicating that these effects can indeed be attributed to the actual assistive social robot, its behavior and its functionality is scarce. Conclusions There is some qualitative evidence as well as limited quantitative evidence of the positive effects of assistive social robots with respect to the elderly. The research designs, however, are not robust enough to establish this. Confounding variables often cannot be excluded. This is partly due to the chosen research designs, but also because it is unclear what research methodology is adequate to investigate such effects. Therefore, more work on methods is needed as well as robust, large-scale studies to establish the effects of these devices.
---
paper_title: Sensor Networks for Ambient Intelligence

paper_content:

Due to rapid advances in networking and sensing technology we are witnessing a growing interest in sensor networks, in which a variety of sensors are connected to each other and to computational devices capable of multimodal signal processing and data analysis. Such networks are seen to play an increasingly important role as key enablers in emerging pervasive computing technologies. In the first part of this paper we give an overview of recent developments in the area of multimodal sensor networks, paying special attention to ambient intelligence applications. In the second part, we discuss how the time series generated by data streams emanating from the sensors can be mined for temporal patterns, indicating cross-sensor signal correlations.
---
paper_title: Monocular 3D Head Tracking to Detect Falls of Elderly People

paper_content:

Faced with the growing population of seniors, Western societies need to think about new technologies to ensure the safety of elderly people at home. Computer vision provides a good solution for healthcare systems because it allows a specific analysis of people behavior. Moreover, a system based on video surveillance is particularly well adapted to detect falls. We present a new method to detect falls using a single camera. Our approach is based on the 3D trajectory of the head, which allows us to distinguish falls from normal activities using 3D velocities. I. INTRODUCTION As other Western countries, Canada's population is grow- ing older. According to the Public Health Agency of Canada (7), one Canadian out of eight was older than 65 years old in 2001. In 2026, this proportion will be one out of five. Moreover, in 1996, 93% of all seniors resided in private households, and among them, 29% lived alone (7). Faced this reality, new technologies are developed to help them live in a more secure environment at home. Falls are one of the most dangerous situation at home. Almost 62% of injury-related hospitalizations for seniors are the result of falls (8). The first concerned are older people living alone because the situation can be aggravated if they cannot call for help, being unconscious or immobilized. Nowadays, the favored solution is to use wearable fall detectors like accelerometers or help buttons. However, older people often forget to wear them, and a help button is useless if the person is unconscious after the fall. Computer vision systems offer a new solution for fall detection which overcome these limitations. Some research has been done to detect falls using image sensors. The easiest method to detect a fall is based on the shape of the person's silhouette or bounding box, but this method can be inaccurate, depending on the relative position of the person, camera, and perhaps occluding objects. Indeed, if the camera is placed sideways, the point of view can be affected by object occlusions. To overcome this problem, some researchers put the camera in the ceiling. For instance, Lee and Mihailidis (11) detect a fall using the shape of the person's silhouette, and Nait-Charif and McKenna (12) detect inactivity outside the normal zones of inactivity like chairs or sofas. Sixsmith and Johnson (15) use an infrared sensor This work was supported by the Natural Sciences and Engineering
---
paper_title: Automatic Fall Incident Detection in Compressed Video for Intelligent Homecare

paper_content:

This paper presents a compressed-domain fall incident detection scheme for intelligent homecare applications. First, a compressed-domain object segmentation scheme is performed to extract moving objects based on global motion estimation and local motion clustering. After detecting the moving objects, three compressed-domain features of each object are then extracted for identifying and locating fall incidents. The proposed system can differentiate fall-down from squatting by taking into account the event duration. Our experiments show that the proposed method can correctly detect fall incidents in real time.
---
paper_title: An Intelligent Surveillance System Based on an Omnidirectional Vision Sensor

paper_content:

Video surveillance, object tracking and activity monitoring are some of the important issues for a home-care robotic system. In this work, an omnidirectional video camera is adopted to provide a 360 degree view angle of the indoor scene with a single image sequence. Some basic functions for smart living and elderly care, such as motion detection, object tracking and target behavior analysis, are implemented. For the motion detection, a background model is first created and the CamShift algorithm is used for object tracking by extracting color information of the target. To make the motion detection and object tracking fully automatic and robust under different illumination conditions, an optical flow approach is cooperated to detect small changes of the mobile object. In addition, the camera is calibrated to obtain the one-to-one correspondences between the image pixels and the locations on the ground. They are used for fall detection of the target by comparing the object length appeared in the image and the computed length according to the object's physical height.
---
paper_title: Machine Recognition of Human Activities: A Survey

paper_content:

The past decade has witnessed a rapid proliferation of video cameras in all walks of life and has resulted in a tremendous explosion of video content. Several applications such as content-based video annotation and retrieval, highlight extraction and video summarization require recognition of the activities occurring in the video. The analysis of human activities in videos is an area with increasingly important consequences from security and surveillance to entertainment and personal archiving. Several challenges at various levels of processing-robustness against errors in low-level processing, view and rate-invariant representations at midlevel processing and semantic representation of human activities at higher level processing-make this problem hard to solve. In this review paper, we present a comprehensive survey of efforts in the past couple of decades to address the problems of representation, recognition, and learning of human activities from video and related applications. We discuss the problem at two major levels of complexity: 1) "actions" and 2) "activities." "Actions" are characterized by simple motion patterns typically executed by a single human. "Activities" are more complex and involve coordinated actions among a small number of humans. We will discuss several approaches and classify them according to their ability to handle varying degrees of complexity as interpreted above. We begin with a discussion of approaches to model the simplest of action classes known as atomic or primitive actions that do not require sophisticated dynamical modeling. Then, methods to model actions with more complex dynamics are discussed. The discussion then leads naturally to methods for higher level representation of complex activities.
---
paper_title: Learning and Matching of Dynamic Shape Manifolds for Human Action Recognition

paper_content:

In this paper, we learn explicit representations for dynamic shape manifolds of moving humans for the task of action recognition. We exploit locality preserving projections (LPP) for dimensionality reduction, leading to a low-dimensional embedding of human movements. Given a sequence of moving silhouettes associated to an action video, by LPP, we project them into a low-dimensional space to characterize the spatiotemporal property of the action, as well as to preserve much of the geometric structure. To match the embedded action trajectories, the median Hausdorff distance or normalized spatiotemporal correlation is used for similarity measures. Action classification is then achieved in a nearest-neighbor framework. To evaluate the proposed method, extensive experiments have been carried out on a recent dataset including ten actions performed by nine different subjects. The experimental results show that the proposed method is able to not only recognize human actions effectively, but also considerably tolerate some challenging conditions, e.g., partial occlusion, low-quality videos, changes in viewpoints, scales, and clothes; within-class variations caused by different subjects with different physical build; styles of motion; etc.
---
paper_title: Human motion: modeling and recognition of actions and interactions

paper_content:

Processing of image sequences has progressed from simple structure from motion paradigm to the recognition of actions/interactions as events. Understanding human activities in video has many potential applications including automated surveillance, video archival/retrieval, medical diagnosis, sports analysis, and human-computer interaction. Understanding human activities involves various steps of low-level vision processing such as segmentation, tracking, pose recovery, and trajectory estimation as well as high-level processing tasks such as body modeling and representation of action. While low-level processing has been actively studied, high-level processing is just beginning to receive attention. This is partly because high-level processing depends on the results of low-level processing. However, high-level processing also requires some independent and additional approaches and methodologies. We focus on the following aspects of high-level processing: (1) human body modeling, (2) level of detail needed to understand human actions, (3) approaches to human action recognition, and (4) high-level recognition schemes with domain knowledge. The review is illustrated by examples of each of the areas discussed, including recent developments in our work on understanding human activities.
---
paper_title: Machine Recognition of Human Activities: A Survey

paper_content:

The past decade has witnessed a rapid proliferation of video cameras in all walks of life and has resulted in a tremendous explosion of video content. Several applications such as content-based video annotation and retrieval, highlight extraction and video summarization require recognition of the activities occurring in the video. The analysis of human activities in videos is an area with increasingly important consequences from security and surveillance to entertainment and personal archiving. Several challenges at various levels of processing-robustness against errors in low-level processing, view and rate-invariant representations at midlevel processing and semantic representation of human activities at higher level processing-make this problem hard to solve. In this review paper, we present a comprehensive survey of efforts in the past couple of decades to address the problems of representation, recognition, and learning of human activities from video and related applications. We discuss the problem at two major levels of complexity: 1) "actions" and 2) "activities." "Actions" are characterized by simple motion patterns typically executed by a single human. "Activities" are more complex and involve coordinated actions among a small number of humans. We will discuss several approaches and classify them according to their ability to handle varying degrees of complexity as interpreted above. We begin with a discussion of approaches to model the simplest of action classes known as atomic or primitive actions that do not require sophisticated dynamical modeling. Then, methods to model actions with more complex dynamics are discussed. The discussion then leads naturally to methods for higher level representation of complex activities.
---
paper_title: Non-parametric model for background subtraction

paper_content:

Background subtraction is a method typically used to segment moving regions in image sequences taken from a static camera by comparing each new frame to a model of the scene background. We present a novel non-parametric background model and a background subtraction approach. The model can handle situations where the background of the scene is cluttered and not completely static but contains small motions such as tree branches and bushes. The model estimates the probability of observing pixel intensity values based on a sample of intensity values for each pixel. The model adapts quickly to changes in the scene which enables very sensitive detection of moving targets. We also show how the model can use color information to suppress detection of shadows. The implementation of the model runs in real-time for both gray level and color imagery. Evaluation shows that this approach achieves very sensitive detection with very low false alarm rates.
---
paper_title: Fall Incidents Detection for Intelligent Video Surveillance

paper_content:

We present in this paper an intelligent video surveillance system to detect human fall incidents for enhanced safety in indoor environments. The system consists of two main parts: a vision component which can reliably detect and track moving people in the view of a camera, and an event-inference module which parses observation sequences of people features for possible falling behavioral signs. In particular, we extract the aspect ratio of a person as observation feature, based on which fall incidents are detected as abrupt changes in the feature space. Our experiments show that the proposed approach can robustly detect human falls in real time
---
paper_title: A Bayesian framework for robust human detection and occlusion handling human shape model

paper_content:

One challenging aspect of automated surveillance for real environments is the occurrences of various difficult scenarios brought about by practical unconstrained settings. We address foreground detection for automated surveillance under the following challenging situations: i) foregrounds being partially hidden due to close similarities to the background, and ii) foregrounds representing multiple objects being inseparable, forming a large contiguous blob due to occlusion. To build a robust system, we present a new foreground detection framework based on Bayesian formulation, comprising both bottom-up and top-down approaches. We first propose a region-based background subtraction and a localized spatial segmentation scheme as the bottom-up steps for foreground detection. We then incorporate a human shape model as the top-down step for foreground validation and occlusion handling. Segmentation is obtained when a maximum posteriori value is found, corresponding to the best description about foregrounds given by the approach. Such integration of bottom-up and top-down approaches leads directly to more robust performance in handling challenging situations within hostile real environments. Promising results are obtained when the algorithm is tested on real video sequences captured from a live surveillance system that operates at a public outdoor swimming pool.
---
paper_title: Detecting Moving Objects, Ghosts, and Shadows in Video Streams

paper_content:

Background subtraction methods are widely exploited for moving object detection in videos in many applications, such as traffic monitoring, human motion capture, and video surveillance. How to correctly and efficiently model and update the background model and how to deal with shadows are two of the most distinguishing and challenging aspects of such approaches. The article proposes a general-purpose method that combines statistical assumptions with the object-level knowledge of moving objects, apparent objects (ghosts), and shadows acquired in the processing of the previous frames. Pixels belonging to moving objects, ghosts, and shadows are processed differently in order to supply an object-based selective update. The proposed approach exploits color information for both background subtraction and shadow detection to improve object segmentation and background update. The approach proves fast, flexible, and precise in terms of both pixel accuracy and reactivity to background changes.
---
paper_title: Quickest change detection for health-care video surveillance

paper_content:

Detecting changes in video scenes is of fundamental importance for various video surveillance tasks. Of particular interest are abnormal changes of foreground human behaviors/activities that could pose damages or dangers to human properties and lives. In this paper, we propose a unified sequential approach to detecting, as soon as possible, human fall incidents for health-care purpose. Specifically, aspect ratio of human body is extracted as the representative feature, based on which an event-inference module parses observed feature sequences for possible falling behavioral signs. Experimental results are reported to show the efficacy of the proposed approach.
---
paper_title: Intelligent Video Surveillance for Monitoring Elderly in Home Environments

paper_content:

In this paper we propose a novel method to detect and record various posture-based events of interest in a typical elderly monitoring application in a home surveillance scenario. These events include standing, sitting, bending/squatting, side lying and lying toward the camera. The projection histograms of segmented human body silhouette are used as the main feature vector for posture classification. k-nearest neighbor (k-NN) algorithm and evidence accumulation technique is proposed to infer human postures. With this technique we have achieved a robust recognition rate of above 90% and a stable classifier's output. The modified classifier structure also improves greatly the recognition rate of lying toward the camera events as compared to the result of classifier's structure in GHOST (Haritaoglu et al., 1998). Furthermore, we use the speed of fall to differentiate real fall incident and an event where the person is simply lying without falling.
---
paper_title: Automatic Fall Incident Detection in Compressed Video for Intelligent Homecare

paper_content:

This paper presents a compressed-domain fall incident detection scheme for intelligent homecare applications. First, a compressed-domain object segmentation scheme is performed to extract moving objects based on global motion estimation and local motion clustering. After detecting the moving objects, three compressed-domain features of each object are then extracted for identifying and locating fall incidents. The proposed system can differentiate fall-down from squatting by taking into account the event duration. Our experiments show that the proposed method can correctly detect fall incidents in real time.
---
paper_title: Quickest change detection for health-care video surveillance

paper_content:

Detecting changes in video scenes is of fundamental importance for various video surveillance tasks. Of particular interest are abnormal changes of foreground human behaviors/activities that could pose damages or dangers to human properties and lives. In this paper, we propose a unified sequential approach to detecting, as soon as possible, human fall incidents for health-care purpose. Specifically, aspect ratio of human body is extracted as the representative feature, based on which an event-inference module parses observed feature sequences for possible falling behavioral signs. Experimental results are reported to show the efficacy of the proposed approach.
---
paper_title: Intelligent Video Surveillance for Monitoring Elderly in Home Environments

paper_content:

In this paper we propose a novel method to detect and record various posture-based events of interest in a typical elderly monitoring application in a home surveillance scenario. These events include standing, sitting, bending/squatting, side lying and lying toward the camera. The projection histograms of segmented human body silhouette are used as the main feature vector for posture classification. k-nearest neighbor (k-NN) algorithm and evidence accumulation technique is proposed to infer human postures. With this technique we have achieved a robust recognition rate of above 90% and a stable classifier's output. The modified classifier structure also improves greatly the recognition rate of lying toward the camera events as compared to the result of classifier's structure in GHOST (Haritaoglu et al., 1998). Furthermore, we use the speed of fall to differentiate real fall incident and an event where the person is simply lying without falling.
---
paper_title: An eigenspace-based approach for human fall detection using Integrated Time Motion Image and Neural Network

paper_content:

Falls are a major health hazard for the elderly and a serious obstacle for independent living. Since falling causes dramatic physical-psychological consequences, development of intelligent video surveillance systems is so important due to providing safe environments. To this end, this paper proposes a novel approach for human fall detection based on combination of integrated time motion images and eigenspace technique. Integrated time motion image (ITMI) is a type of spatio-temporal database that includes motion and time of motion occurrence. Applying eigenspace technique to ITMIs leads in extracting eigen-motion and finally MLP Neural Network is used for precise classification of motions and determination of a fall event. Unlike existent fall detection systems only deal with limited movement patterns, we considered wide range of motions consisting normal daily life activities, abnormal behaviors and also unusual events. Reliable recognition rate of experimental results underlines satisfactory performance of our system.
---
paper_title: Fall Detection from Human Shape and Motion History Using Video Surveillance

paper_content:

Nowadays, Western countries have to face the growing population of seniors. New technologies can help people stay at home by providing a secure environment and improving their quality of life. The use of computer vision systems offers a new promising solution to analyze people behavior and detect some unusual events. In this paper, we propose a new method to detect falls, which are one of the greatest risk for seniors living alone. Our approach is based on a combination of motion history and human shape variation. Our algorithm provides promising results on video sequences of daily activities and simulated falls.
---
paper_title: Intelligent Video Monitoring to Improve Safety of Older Persons

paper_content:

This paper presents the application of computer vision and machine learning technologies to a clinical task of paramount importance, improving safety of older persons. We propose an intelligent monitoring system equipped with a camera network and an automatic elopement detection algorithm to reduce the risks of un-witnessed elopements from a dementia unit in order to avoid their potential catastrophic consequences. The camera network employs 23 cameras to record daily activities in our test bed, which includes 15 residents, 4 registered and licensed practical nurses and a number of certified nursing assistants. An elopement detector is then built by using computer vision algorithms and a machine learning algorithm to automatically detect elopements and alert caregivers. The experiments demonstrate that the proposed system leverages the advantages of monitoring from multiple cameras and is able to detect elopements with almost 100% accuracy.
---
paper_title: Monocular 3D Head Tracking to Detect Falls of Elderly People

paper_content:

Faced with the growing population of seniors, Western societies need to think about new technologies to ensure the safety of elderly people at home. Computer vision provides a good solution for healthcare systems because it allows a specific analysis of people behavior. Moreover, a system based on video surveillance is particularly well adapted to detect falls. We present a new method to detect falls using a single camera. Our approach is based on the 3D trajectory of the head, which allows us to distinguish falls from normal activities using 3D velocities. I. INTRODUCTION As other Western countries, Canada's population is grow- ing older. According to the Public Health Agency of Canada (7), one Canadian out of eight was older than 65 years old in 2001. In 2026, this proportion will be one out of five. Moreover, in 1996, 93% of all seniors resided in private households, and among them, 29% lived alone (7). Faced this reality, new technologies are developed to help them live in a more secure environment at home. Falls are one of the most dangerous situation at home. Almost 62% of injury-related hospitalizations for seniors are the result of falls (8). The first concerned are older people living alone because the situation can be aggravated if they cannot call for help, being unconscious or immobilized. Nowadays, the favored solution is to use wearable fall detectors like accelerometers or help buttons. However, older people often forget to wear them, and a help button is useless if the person is unconscious after the fall. Computer vision systems offer a new solution for fall detection which overcome these limitations. Some research has been done to detect falls using image sensors. The easiest method to detect a fall is based on the shape of the person's silhouette or bounding box, but this method can be inaccurate, depending on the relative position of the person, camera, and perhaps occluding objects. Indeed, if the camera is placed sideways, the point of view can be affected by object occlusions. To overcome this problem, some researchers put the camera in the ceiling. For instance, Lee and Mihailidis (11) detect a fall using the shape of the person's silhouette, and Nait-Charif and McKenna (12) detect inactivity outside the normal zones of inactivity like chairs or sofas. Sixsmith and Johnson (15) use an infrared sensor This work was supported by the Natural Sciences and Engineering
---
paper_title: Tracking Poorly Modelled Motion Using Particle Filters with Iterated Likelihood Weighting

paper_content:

Human motion in cluttered scenes is often tracked using particle filtering. However, poorly modelled inter-frame motion is not uncommon, resulting in poor priors for the filtering step. Alternatives to the Condensation algorithm in the form of an Auxiliary Particle Filter (APF) and Iterated Likelihood Weighting (ILW) are described. Experimental results comparing these filters’ accuracy and consistency are presented for a scenario in which a person is tracked in an overhead view using an ellipse model with a likelihood based on colour and gradient cues. ILW is not intended to give unbiased estimates of a posterior but rather to reduce approximation error. It is shown to outperform both Condensation and the APF on sequences from this scenario.
---
paper_title: CONDENSATION—Conditional Density Propagation for Visual Tracking

paper_content:

The problem of tracking curves in dense visual clutter is challenging. Kalman filtering is inadequate because it is based on Gaussian densities which, being unimo dal, cannot represent simultaneous alternative hypotheses. The Condensation algorithm uses “factored sampling”, previously applied to the interpretation of static images, in which the probability distribution of possible interpretations is represented by a randomly generated set. Condensation uses learned dynamical models, together with visual observations, to propagate the random set over time. The result is highly robust tracking of agile motion. Notwithstanding the use of stochastic methods, the algorithm runs in near real-time.
---
paper_title: Model-Based Object Pose in 25 Lines of Code

paper_content:

We find the pose of an object from a single image when the relative geometry of four or more noncoplanar visible feature points is known. We first describe an algorithm, POS (Pose from Orthography and Scaling), that solves for the rotation matrix and the translation vector of the object by a linear algebra technique under the scaled orthographic projection approximation. We then describe an iterative algorithm, POSIT (POS with ITerations), that uses the pose found by POS to remove the “perspective distortions” from the image, then applies POS to the corrected image instead of the original image. POSIT generally converges to accurate pose measurements in a few iterations. Mathematica code is provided in an Appendix.
---
paper_title: Activity summarisation and fall detection in a supportive home environment

paper_content:

Automatic semantic summarisation of human activity and detection of unusual inactivity are useful goals for a vision system operating in a supportive home environment. Learned models of spatial context are used in conjunction with a tracker to achieve these goals. The tracker uses a coarse ellipse model and a particle filter to cope with cluttered scenes with multiple sources of illumination. Summarisation in terms of semantic regions is demonstrated using acted scenes through automatic recovery of the instructions given to the actor. The use of 'unusual inactivity' detection as a cue for fall detection is also demonstrated.
---
paper_title: Intelligent Video Surveillance for Monitoring Elderly in Home Environments

paper_content:

In this paper we propose a novel method to detect and record various posture-based events of interest in a typical elderly monitoring application in a home surveillance scenario. These events include standing, sitting, bending/squatting, side lying and lying toward the camera. The projection histograms of segmented human body silhouette are used as the main feature vector for posture classification. k-nearest neighbor (k-NN) algorithm and evidence accumulation technique is proposed to infer human postures. With this technique we have achieved a robust recognition rate of above 90% and a stable classifier's output. The modified classifier structure also improves greatly the recognition rate of lying toward the camera events as compared to the result of classifier's structure in GHOST (Haritaoglu et al., 1998). Furthermore, we use the speed of fall to differentiate real fall incident and an event where the person is simply lying without falling.
---
paper_title: On combining classifiers

paper_content:

We develop a common theoretical framework for combining classifiers which use distinct pattern representations and show that many existing schemes can be considered as special cases of compound classification where all the pattern representations are used jointly to make a decision. An experimental comparison of various classifier combination schemes demonstrates that the combination rule developed under the most restrictive assumptions-the sum rule-outperforms other classifier combinations schemes. A sensitivity analysis of the various schemes to estimation errors is carried out to show that this finding can be justified theoretically.
---
paper_title: Automatic Fall Incident Detection in Compressed Video for Intelligent Homecare

paper_content:

This paper presents a compressed-domain fall incident detection scheme for intelligent homecare applications. First, a compressed-domain object segmentation scheme is performed to extract moving objects based on global motion estimation and local motion clustering. After detecting the moving objects, three compressed-domain features of each object are then extracted for identifying and locating fall incidents. The proposed system can differentiate fall-down from squatting by taking into account the event duration. Our experiments show that the proposed method can correctly detect fall incidents in real time.
---
paper_title: Fall Detection from Human Shape and Motion History Using Video Surveillance

paper_content:

Nowadays, Western countries have to face the growing population of seniors. New technologies can help people stay at home by providing a secure environment and improving their quality of life. The use of computer vision systems offers a new promising solution to analyze people behavior and detect some unusual events. In this paper, we propose a new method to detect falls, which are one of the greatest risk for seniors living alone. Our approach is based on a combination of motion history and human shape variation. Our algorithm provides promising results on video sequences of daily activities and simulated falls.
---
paper_title: Recognition of visual activities and interactions by stochastic parsing

paper_content:

This paper describes a probabilistic syntactic approach to the detection and recognition of temporally extended activities and interactions between multiple agents. The fundamental idea is to divide the recognition problem into two levels. The lower level detections are performed using standard independent probabilistic event detectors to propose candidate detections of low-level features. The outputs of these detectors provide the input stream for a stochastic context-free grammar parsing mechanism. The grammar and parser provide longer range temporal constraints, disambiguate uncertain low-level detections, and allow the inclusion of a priori knowledge about the structure of temporal events in a given domain. We develop a real-time system and demonstrate the approach in several experiments on gesture recognition and in video surveillance. In the surveillance application, we show how the system correctly interprets activities of multiple interacting objects.
---
paper_title: Monocular 3D Head Tracking to Detect Falls of Elderly People

paper_content:

Faced with the growing population of seniors, Western societies need to think about new technologies to ensure the safety of elderly people at home. Computer vision provides a good solution for healthcare systems because it allows a specific analysis of people behavior. Moreover, a system based on video surveillance is particularly well adapted to detect falls. We present a new method to detect falls using a single camera. Our approach is based on the 3D trajectory of the head, which allows us to distinguish falls from normal activities using 3D velocities. I. INTRODUCTION As other Western countries, Canada's population is grow- ing older. According to the Public Health Agency of Canada (7), one Canadian out of eight was older than 65 years old in 2001. In 2026, this proportion will be one out of five. Moreover, in 1996, 93% of all seniors resided in private households, and among them, 29% lived alone (7). Faced this reality, new technologies are developed to help them live in a more secure environment at home. Falls are one of the most dangerous situation at home. Almost 62% of injury-related hospitalizations for seniors are the result of falls (8). The first concerned are older people living alone because the situation can be aggravated if they cannot call for help, being unconscious or immobilized. Nowadays, the favored solution is to use wearable fall detectors like accelerometers or help buttons. However, older people often forget to wear them, and a help button is useless if the person is unconscious after the fall. Computer vision systems offer a new solution for fall detection which overcome these limitations. Some research has been done to detect falls using image sensors. The easiest method to detect a fall is based on the shape of the person's silhouette or bounding box, but this method can be inaccurate, depending on the relative position of the person, camera, and perhaps occluding objects. Indeed, if the camera is placed sideways, the point of view can be affected by object occlusions. To overcome this problem, some researchers put the camera in the ceiling. For instance, Lee and Mihailidis (11) detect a fall using the shape of the person's silhouette, and Nait-Charif and McKenna (12) detect inactivity outside the normal zones of inactivity like chairs or sofas. Sixsmith and Johnson (15) use an infrared sensor This work was supported by the Natural Sciences and Engineering
---
paper_title: Leveraging temporal, contextual and ordering constraints for recognizing complex activities in video

paper_content:

We present a scalable approach to recognizing and describing complex activities in video sequences. We are interested in long-term, sequential activities that may have several parallel streams of action. Our approach integrates temporal, contextual and ordering constraints with output from low-level visual detectors to recognize complex, long-term activities. We argue that a hierarchical, object-oriented design lends our solution to be scalable in that higher-level reasoning components are independent from the particular low-level detector implementation and that recognition of additional activities and actions can easily be added. Three major components to realize this design are: a dynamic Bayesian network structure for representing activities comprised of partially ordered sub-actions, an object-oriented action hierarchy for building arbitrarily complex action detectors and an approximate Viterbi-like algorithm for inferring the most likely observed sequence of actions. Additionally, this study proposes the Erlang distribution as a comprehensive model of idle time between actions and frequency of observing new actions. We show results for our approach on real video sequences containing complex activities.
---
paper_title: Automatic Fall Incident Detection in Compressed Video for Intelligent Homecare

paper_content:

This paper presents a compressed-domain fall incident detection scheme for intelligent homecare applications. First, a compressed-domain object segmentation scheme is performed to extract moving objects based on global motion estimation and local motion clustering. After detecting the moving objects, three compressed-domain features of each object are then extracted for identifying and locating fall incidents. The proposed system can differentiate fall-down from squatting by taking into account the event duration. Our experiments show that the proposed method can correctly detect fall incidents in real time.
---
paper_title: An eigenspace-based approach for human fall detection using Integrated Time Motion Image and Neural Network

paper_content:

Falls are a major health hazard for the elderly and a serious obstacle for independent living. Since falling causes dramatic physical-psychological consequences, development of intelligent video surveillance systems is so important due to providing safe environments. To this end, this paper proposes a novel approach for human fall detection based on combination of integrated time motion images and eigenspace technique. Integrated time motion image (ITMI) is a type of spatio-temporal database that includes motion and time of motion occurrence. Applying eigenspace technique to ITMIs leads in extracting eigen-motion and finally MLP Neural Network is used for precise classification of motions and determination of a fall event. Unlike existent fall detection systems only deal with limited movement patterns, we considered wide range of motions consisting normal daily life activities, abnormal behaviors and also unusual events. Reliable recognition rate of experimental results underlines satisfactory performance of our system.
---
paper_title: An intelligent emergency response system: preliminary development and testing of automated fall detection

paper_content:

We have designed an intelligent emergency response system to detect falls in the home. It uses image-based sensors. A pilot study was conducted using 21 subjects to evaluate the efficacy and performance of the fall-detection component of the system. Trials were conducted in a mock-up bedroom setting, with a bed, a chair and other typical bedroom furnishings. A small digital videocamera was installed in the ceiling at a height of approximately 2.6 m. The digital camera covered an area of approximately 5.0 m × 3.8 m. The subjects were asked to assume a series of postures, namely walking/standing, sitting/lying down in an inactive zone, stooping, lying down in a 'stretched' position, and lying down in a 'tucked' position. These five scenarios were repeated three times by each subject in a random order. These test positions totalled 315 tasks with 126 fall-simulated tasks and 189 non-fall-simulated tasks. The system detected a fall on 77% of occasions and missed a fall on 23%. False alarms occurred on only 5%...
---
paper_title: Fall Incidents Detection for Intelligent Video Surveillance

paper_content:

We present in this paper an intelligent video surveillance system to detect human fall incidents for enhanced safety in indoor environments. The system consists of two main parts: a vision component which can reliably detect and track moving people in the view of a camera, and an event-inference module which parses observation sequences of people features for possible falling behavioral signs. In particular, we extract the aspect ratio of a person as observation feature, based on which fall incidents are detected as abrupt changes in the feature space. Our experiments show that the proposed approach can robustly detect human falls in real time
---
paper_title: An intelligent videomonitoring system for fall detection at home: perceptions of elderly people

paper_content:

We explored the perception and receptivity of elderly people regarding the introduction of an intelligent videomonitoring system (IVS) at home. Using a mixed methods design, 25 elderly people with a history of falls completed a structured interview (two questionnaires). In the year preceding the interview, 72% of the participants fell as many as seven times. Open-ended questions (qualitative data) were used to supplement the quantitative data. A content analysis (qualitative data) and a descriptive analysis (quantitative data) were carried out. The participants were 84% favourable or partially favourable to technologies which ensured home security and 96% favourable or partially favourable to the IVS. About half (48%) said that they would use it. The other participants did not wish to use it unless they had been left to live alone or if their health condition worsened. Thus the living conditions of the elderly appear to influence their perception and receptivity regarding the acceptance and use of an IVS.
---
paper_title: Fall Detection from Human Shape and Motion History Using Video Surveillance

paper_content:

Nowadays, Western countries have to face the growing population of seniors. New technologies can help people stay at home by providing a secure environment and improving their quality of life. The use of computer vision systems offers a new promising solution to analyze people behavior and detect some unusual events. In this paper, we propose a new method to detect falls, which are one of the greatest risk for seniors living alone. Our approach is based on a combination of motion history and human shape variation. Our algorithm provides promising results on video sequences of daily activities and simulated falls.
---
paper_title: Intelligent Video Monitoring to Improve Safety of Older Persons

paper_content:

This paper presents the application of computer vision and machine learning technologies to a clinical task of paramount importance, improving safety of older persons. We propose an intelligent monitoring system equipped with a camera network and an automatic elopement detection algorithm to reduce the risks of un-witnessed elopements from a dementia unit in order to avoid their potential catastrophic consequences. The camera network employs 23 cameras to record daily activities in our test bed, which includes 15 residents, 4 registered and licensed practical nurses and a number of certified nursing assistants. An elopement detector is then built by using computer vision algorithms and a machine learning algorithm to automatically detect elopements and alert caregivers. The experiments demonstrate that the proposed system leverages the advantages of monitoring from multiple cameras and is able to detect elopements with almost 100% accuracy.
---
paper_title: Contact-Free Measurement of Cardiac Pulse Based on the Analysis of Thermal Imagery

paper_content:

We have developed a novel method to measure human cardiac pulse at a distance. It is based on the information contained in the thermal signal emitted from major superficial vessels. This signal is acquired through a highly sensitive thermal imaging system. Temperature on the vessel is modulated by pulsative blood flow. To compute the frequency of modulation (pulse), we extract a line-based region along the vessel. Then, we apply fast Fourier transform (FFT) to individual points along this line of interest to capitalize on the pulse's thermal propagation effect. Finally, we use an adaptive estimation function on the average FFT outcome to quantify the pulse. We have carried out experiments on a data set of 34 subjects and compared the pulse computed from our thermal signal analysis method to concomitant ground-truth measurements obtained through a standard contact sensor (piezo-electric transducer). The performance of the new method ranges from 88.52% to 90.33% depending on the clarity of the vessel's thermal imprint. To the best of our knowledge, it is the first time that cardiac pulse has been measured several feet away from a subject with passive means.
---
paper_title: Heart rate measurement based on a time-lapse image.

paper_content:

Using a time-lapse image acquired from a CCD camera, we developed a non-contact and non-invasive device, which could measure both the respiratory and pulse rate simultaneously. The time-lapse image of a part of the subject's skin was consecutively captured, and the changes in the average image brightness of the region of interest (ROI) were measured for 30s. The brightness data were processed by a series of operations of interpolation as follows a first-order derivative, a low pass filter of 2 Hz, and a sixth-order auto-regressive (AR) spectral analysis. Fourteen sound and healthy female subjects (22-27 years of age) participated in the experiments. Each subject was told to keep a relaxed seating posture with no physical restriction. At the same time, heart rate was measured by a pulse oximeter and respiratory rate was measured by a thermistor placed at the external naris. Using AR spectral analysis, two clear peaks could be detected at approximately 0.3 and 1.2 Hz. The peaks were thought to correspond to the respiratory rate and the heart rate. Correlation coefficients of 0.90 and 0.93 were obtained for the measurement of heart rate and respiratory rate, respectively.
---
paper_title: Thermistor at a Distance: Unobtrusive Measurement of Breathing

paper_content:

This paper unveils a thermal imaging methodology to recover the breathing waveform from the subject's nostrils. The resulting functionality is equivalent to that of a thermistor, but it is materialized in a contact-free manner. First, the nostril region is segmented and tracked over time through a network of cooperating probabilistic trackers. The mean thermal signal of the nostril region carries the breathing information. This information is extracted through wavelet analysis. The method has been tested on 20 healthy individuals. The breathing waveforms determined via the imaging computation were compared with the corresponding ones extracted from thermistors. The high degree of agreement between the two measurement methods confirms the validity of the proposed approach and opens the way for clinical applications. Furthermore, thermal imaging can be potentially used as an investigative tool to understand breathing physiology in ways not possible with contact sensors.
---
paper_title: Activity summarisation and fall detection in a supportive home environment

paper_content:

Automatic semantic summarisation of human activity and detection of unusual inactivity are useful goals for a vision system operating in a supportive home environment. Learned models of spatial context are used in conjunction with a tracker to achieve these goals. The tracker uses a coarse ellipse model and a particle filter to cope with cluttered scenes with multiple sources of illumination. Summarisation in terms of semantic regions is demonstrated using acted scenes through automatic recovery of the instructions given to the actor. The use of 'unusual inactivity' detection as a cue for fall detection is also demonstrated.
---
paper_title: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation

paper_content:

Remote measurements of the cardiac pulse can provide comfortable physiological assessment without electrodes. However, attempts so far are non-automated, susceptible to motion artifacts and typically expensive. In this paper, we introduce a new methodology that overcomes these problems. This novel approach can be applied to color video recordings of the human face and is based on automatic face tracking along with blind source separation of the color channels into independent components. Using Bland-Altman and correlation analysis, we compared the cardiac pulse rate extracted from videos recorded by a basic webcam to an FDA-approved finger blood volume pulse (BVP) sensor and achieved high accuracy and correlation even in the presence of movement artifacts. Furthermore, we applied this technique to perform heart rate measurements from three participants simultaneously. This is the first demonstration of a low-cost accurate video-based method for contact-free heart rate measurements that is automated, motion-tolerant and capable of performing concomitant measurements on more than one person at a time.
---
paper_title: Intelligent Video Monitoring to Improve Safety of Older Persons

paper_content:

This paper presents the application of computer vision and machine learning technologies to a clinical task of paramount importance, improving safety of older persons. We propose an intelligent monitoring system equipped with a camera network and an automatic elopement detection algorithm to reduce the risks of un-witnessed elopements from a dementia unit in order to avoid their potential catastrophic consequences. The camera network employs 23 cameras to record daily activities in our test bed, which includes 15 residents, 4 registered and licensed practical nurses and a number of certified nursing assistants. An elopement detector is then built by using computer vision algorithms and a machine learning algorithm to automatically detect elopements and alert caregivers. The experiments demonstrate that the proposed system leverages the advantages of monitoring from multiple cameras and is able to detect elopements with almost 100% accuracy.
---
paper_title: Model-based face de-identification

paper_content:

With the emergence of new applications centered around the sharing of image data, questions concerning the protection of the privacy of people visible in the scene arise. In most of these applications, knowledge of the identity of people in the image is not required. This makes the case for image de-identification, the removal of identifying information from images, prior to sharing of the data. Privacy protection methods are well established for field-structured data; however, work on images is still limited. In this chapter, we review previously proposed naive and formal face de-identification methods. We then describe a novel framework for the de-identification of face images using multi-factor models which unify linear, bilinear, and quadratic data models. We show in experiments on a large expression-variant face database that the new algorithm is able to protect privacy while preserving data utility. The new model extends directly to image sequences, which we demonstrate on examples from a medical face database.
---
paper_title: Scrambling for Video Surveillance with Privacy

paper_content:

In this paper, we address the problem of scrambling regions of interest in a video sequence for the purpose of preserving privacy in video surveillance. We propose an efficient solution based on transform-domain scrambling. More specifically, the sign of selected transform coefficients is pseudo-randomly flipped during encoding. We address more specifically the two cases of MPEG-4 and Motion JPEG 2000. Simulation results show that the technique can be successfully applied to conceal information in regions of interest in the scene while providing with a good level of security. Furthermore, the scrambling is flexible and allows adjusting the amount of distortion introduced. Finally, this is achieved with a small impact on coding performance and negligible computational complexity increase.
---
paper_title: TrustCAM: Security and Privacy-Protection for an Embedded Smart Camera Based on Trusted Computing

paper_content:

Security and privacy protection are critical issues forpublic acceptance of camera networks. Smart cameras,with onboard image processing, can be used to identifyand remove privacy sensitive image regions. Existing approaches,however, only address isolated aspects withoutconsidering the integration with established security technologiesand the underlying platform. This work tries to fillthis gap and presents TrustCAM, a security-enhanced smartcamera. Based on Trusted Computing, we realize integrityprotection, authenticity and confidentiality of image data.Multiple levels of privacy protection, together with accesscontrol, are supported. Impact on overall system performanceis evaluated on a real prototype implementation.
---
paper_title: From sensor networks to behaviour profiling: a homecare perspective of intelligent buildings

paper_content:

It is a well-known fact that due to improved medical treatments and other factors people are generally living longer. Whilst this may be considered beneficial, it also brings with it many problems; for example diseases such as osteo-arthritis and dementia are more likely to affect a person as they age. In many cases these diseases eventually cause the sufferer to become completely reliant upon social services, and can force them to give up their home and move into residential care or a nursing home. Technological developments of sensing and monitoring devices is now reshaping the general practice in clinical medicine. The development of ambient sensing technology can allow detailed level of activity information for the identification of both temporally and spatially daily living activities as such as eating, drinking, reading, resting or sleeping out of bed. Furthermore activity states related to emotion can be identified such as agitation, restlessness or pacing up and down. (7 pages)
---
paper_title: A Secure Authentication Scheme for Telecare Medicine Information Systems

paper_content:

The telecare medicine information system enables or supports health-care delivery services. In recent years, the increased availability of lower-cost telecommunications systems and custom made physiological monitoring devices for patients have made it possible to bring the advantages of telemedicine directly into the patient's home. These systems are moving towards an environment where automated patient medical records and electronically interconnected telecare facilities are prevalent. A secure authentication scheme will thus be needed to safeguard data integrity, confidentiality, and availability. Many schemes based on cryptography have been proposed for the goals. However, much of the schemes are vulnerable to various attacks, and are neither efficient, nor user friendly. Specially, in terms of efficiency, some schemes need the exponential computation resulting in high time cost. Therefore, we propose a novel authentication scheme that is added the pre-computing idea within the communication process to avoid the time-consuming exponential computations. Finally, it is shown to be more secure and practical for telecare medicine environments.
---
paper_title: An intelligent videomonitoring system for fall detection at home: perceptions of elderly people

paper_content:

We explored the perception and receptivity of elderly people regarding the introduction of an intelligent videomonitoring system (IVS) at home. Using a mixed methods design, 25 elderly people with a history of falls completed a structured interview (two questionnaires). In the year preceding the interview, 72% of the participants fell as many as seven times. Open-ended questions (qualitative data) were used to supplement the quantitative data. A content analysis (qualitative data) and a descriptive analysis (quantitative data) were carried out. The participants were 84% favourable or partially favourable to technologies which ensured home security and 96% favourable or partially favourable to the IVS. About half (48%) said that they would use it. The other participants did not wish to use it unless they had been left to live alone or if their health condition worsened. Thus the living conditions of the elderly appear to influence their perception and receptivity regarding the acceptance and use of an IVS.
---
paper_title: Gathering the requirements for a fall monitor using drama and video with older people

paper_content:

A system providing fall detection and movement monitoring to support older people living at home using computer vision technology is being developed. Sensitive design with user involvement is important if such a system is to be experienced as supportive rather than invasive. Four scenarios, based on material from focus groups and anecdotal evidence, have been developed and performed by a theatre group. These feature older people falling at home with different outcomes and carers discussing an older person's needs. They were filmed and shown to three different groups of older people and a group of sheltered housing wardens to provoke discussion. This method of user requirements gathering provided a shared user context which enabled groups to focus very effectively on the details of a system at the pre-prototyping stages. The results of the discussions are described and the use of this methodology is discussed.
---
paper_title: An intelligent videomonitoring system for fall detection at home: perceptions of elderly people

paper_content:

We explored the perception and receptivity of elderly people regarding the introduction of an intelligent videomonitoring system (IVS) at home. Using a mixed methods design, 25 elderly people with a history of falls completed a structured interview (two questionnaires). In the year preceding the interview, 72% of the participants fell as many as seven times. Open-ended questions (qualitative data) were used to supplement the quantitative data. A content analysis (qualitative data) and a descriptive analysis (quantitative data) were carried out. The participants were 84% favourable or partially favourable to technologies which ensured home security and 96% favourable or partially favourable to the IVS. About half (48%) said that they would use it. The other participants did not wish to use it unless they had been left to live alone or if their health condition worsened. Thus the living conditions of the elderly appear to influence their perception and receptivity regarding the acceptance and use of an IVS.
---
paper_title: From sensor networks to behaviour profiling: a homecare perspective of intelligent buildings

paper_content:

It is a well-known fact that due to improved medical treatments and other factors people are generally living longer. Whilst this may be considered beneficial, it also brings with it many problems; for example diseases such as osteo-arthritis and dementia are more likely to affect a person as they age. In many cases these diseases eventually cause the sufferer to become completely reliant upon social services, and can force them to give up their home and move into residential care or a nursing home. Technological developments of sensing and monitoring devices is now reshaping the general practice in clinical medicine. The development of ambient sensing technology can allow detailed level of activity information for the identification of both temporally and spatially daily living activities as such as eating, drinking, reading, resting or sleeping out of bed. Furthermore activity states related to emotion can be identified such as agitation, restlessness or pacing up and down. (7 pages)
---
paper_title: Gathering the requirements for a fall monitor using drama and video with older people

paper_content:

A system providing fall detection and movement monitoring to support older people living at home using computer vision technology is being developed. Sensitive design with user involvement is important if such a system is to be experienced as supportive rather than invasive. Four scenarios, based on material from focus groups and anecdotal evidence, have been developed and performed by a theatre group. These feature older people falling at home with different outcomes and carers discussing an older person's needs. They were filmed and shown to three different groups of older people and a group of sheltered housing wardens to provoke discussion. This method of user requirements gathering provided a shared user context which enabled groups to focus very effectively on the details of a system at the pre-prototyping stages. The results of the discussions are described and the use of this methodology is discussed.
---
paper_title: Acceptance of Healthcare Robots for the Older Population: Review and Future Directions

paper_content:

The rapidly ageing population is placing increasing strain on healthcare services. Robots have been proposed as a way to assist people to stay healthy and safe in their own homes. However, despite the need for such assistive devices and the success of some healthcare robots, other robots have had a poor response. This article reviews the literature about human responses to healthcare robots and summarises the variables that have been found to influence responses. It may be possible to increase acceptance of healthcare robots by properly assessing the needs of the human user and then matching the robot’s role, appearance and behaviour to these needs. Because robots have limitations in their abilities, another way to increase acceptance may be to modify the expectations of users to better match robots’ abilities. More research needs to investigate potential users’ needs and expectations in specific situations and whether interventions to increase the match between robot and human can increase acceptance.
---
paper_title: An evaluation of second and third generation telecare services in older people's housing

paper_content:

In a controlled study of older people living in sheltered housing (retirement housing), 24 people provided with telecare were compared with a control group of 28 people. The intervention consisted of second generation telecare equipment, such as automatic flood or falls detectors, a third generation lifestyle reassurance system and an Internet cafe. After a 12-month monitoring period, there was no noticeable change in the fear of falling. There was no significant difference for eight of the nine SF-36 domains. However, the Social Functioning domain showed a significant difference (P = 0.049), with scores 8% higher in the intervention group,suggesting a beneficial effect of telecare. Positive trends were also evident in areas such as an increase in the length of time spent out of the home, improved feelings of safety during the day and night, and a reduction in the fear of crime. The Internet cafe was used by 25% of people for at least 20 min per week. The results suggest that second generation telecare systems and Internet facilities could be more widely used in service delivery, but that lifestyle reassurance requires further development. Language: en
---
paper_title: Privacy Impact Assessments: The UK Experience

paper_content:

This paper builds on original work undertaken as part of a team of researchers into Privacy Impact Assessments (PIAs), defined as a systematic risk assessment tool that can be usefully integrated into decision-making processes. The team were commissioned by the UK Information Commissioner’s Office (ICO) in June 2007 to develop a study of PIAs in overseas jurisdictions and a handbook to guide UK organisations through the PIA process. This research has subsequently attracted interest in the UK and overseas. PIAs are now mandatory for all UK central government departments. In this paper, the development of the project team’s PIA methodology and subsequent user experiences led to a key project output, the PIA handbook. The handbook has become a significant part of the privacy ‘toolkit’ and has impacted on public policy. Some important lessons from PIAs conducted in the UK and overseas are identified. Finally, areas are outlined for further development.
---
paper_title: A questionnaire for the assessment of patients' impressions of the risks and benefits of home telecare

paper_content:

Home telecare is a promising method of improving access to care for rural and urban populations. It requires, however, that patients accept the installation and use of equipment in their home. How patients perceive home telecare may influence its acceptability and diffusion. We developed a questionnaire to assess patients' impressions of the risks and benefits of home telecare. A preliminary 20-item questionnaire was developed and tested on 32 subjects. It proved to be of acceptable reliability (Cronbach's alpha of 0.8) and validity, but three items appeared to be redundant. The final 17-item version was tested twice on 10 more subjects and was found to have high test-retest reliability. Most subjects showed an overall positive attitude towards home telecare and agreed it could improve their health. Many were concerned about being able to trust the equipment. The questionnaire can be used to investigate the acceptance of telemedicine and to improve the design of applications.
---
paper_title: Performance metrics for activity recognition

paper_content:

In this article, we introduce and evaluate a comprehensive set of performance metrics and visualisations for continuous activity recognition (AR). We demonstrate how standard evaluation methods, often borrowed from related pattern recognition problems, fail to capture common artefacts found in continuous AR—specifically event fragmentation, event merging and timing offsets. We support our assertion with an analysis on a set of recently published AR papers. Building on an earlier initial work on the topic, we develop a frame-based visualisation and corresponding set of class-skew invariant metrics for the one class versus all evaluation. These are complemented by a new complete set of event-based metrics that allow a quick graphical representation of system performance—showing events that are correct, inserted, deleted, fragmented, merged and those which are both fragmented and merged. We evaluate the utility of our approach through comparison with standard metrics on data from three different published experiments. This shows that where event- and frame-based precision and recall lead to an ambiguous interpretation of results in some cases, the proposed metrics provide a consistently unambiguous explanation.
---
paper_title: Activity Recognition in the Home Using Simple and Ubiquitous Sensors

paper_content:

In this work, a system for recognizing activities in the home setting using a set of small and simple state-change sensors is introduced. The sensors are designed to be “tape on and forget” devices that can be quickly and ubiquitously installed in home environments. The proposed sensing system presents an alternative to sensors that are sometimes perceived as invasive, such as cameras and microphones. Unlike prior work, the system has been deployed in multiple residential environments with non-researcher occupants. Preliminary results on a small dataset show that it is possible to recognize activities of interest to medical professionals such as toileting, bathing, and grooming with detection accuracies ranging from 25% to 89% depending on the evaluation criteria used.
---

Title: Video based technology for ambient assisted living: A review of the literature
Section 1: Introduction
Description 1: Introduce the increasing need for ambient assisted living (AAL) due to the aging population and define AAL with its specific context. Briefly describe the primary focus of the paper.

Section 2: Application Domains and Scope
Description 2: Discuss the various application domains of AAL and the specific scope of video technology applications within AAL. Mention the exclusion of certain types of video camera applications not covered by the survey.

Section 3: Health, rehabilitation and care
Description 3: Explain how video technology can be used to monitor physiological data and users' activities to support health, rehabilitation, and care.

Section 4: Personal and home safety and security
Description 4: Address the use of video technology for enhancing safety and security at home, with emphasis on fall detection and monitoring vulnerable individuals such as those with dementia.

Section 5: Biorobotic systems
Description 5: Discuss the incorporation of video technology in biorobotic systems and the role of companion robots in providing support to older people.

Section 6: Video Based vs Sensor Based Solutions
Description 6: Compare the advantages and disadvantages of video-based solutions versus traditional sensor-based solutions in AAL applications.

Section 7: Multi-modal Approaches
Description 7: Describe the use of multi-modal approaches that combine video with other sensors for AAL solutions, enhancing robustness and providing complementary information.

Section 8: General Structure of Video Based AAL
Description 8: Present the general architecture and structure of video-based AAL systems, including centralised and distributed camera networks.

Section 9: Visual Processing
Description 9: Detail the computer vision algorithms and methodologies used for recognising actions and activities, and discuss video processing challenges.

Section 10: Action Recognition Algorithms
Description 10: Describe specific action recognition techniques used in video-based monitoring, including static and dynamic approaches, and motion tracking.

Section 11: Monitoring Behaviour
Description 11: Explain methodologies for monitoring and recognising human behaviour over extended periods, differentiating between actions and activities.

Section 12: Physiological monitoring
Description 12: Cover the techniques used for contact-free measurement of physiological data through video, such as heart rate and respiratory rate monitoring.

Section 13: Discussion and Challenges
Description 13: Discuss the key challenges in the field, such as occlusions, multi-occupancy, and improving the sensitivity and specificity of detection algorithms.

Section 14: Acceptability and Privacy
Description 14: Explore privacy concerns and the acceptability of video-based AAL systems among potential users, carers, and society.

Section 15: Data and Privacy Protection
Description 15: Detail the strategies for ensuring data security and privacy protection in video-based AAL systems.

Section 16: User Requirements and Acceptability
Description 16: Summarize user studies and surveys on the acceptability of video-based AAL systems, highlighting concerns and conditions for acceptance.

Section 17: System Evaluation
Description 17: Discuss the methodologies for evaluating the performance and effectiveness of VAAL systems, covering technical, outcome, privacy protection, and user acceptability evaluations.

Section 18: Technical Performance Evaluation
Description 18: Explain the metrics and methodologies used for assessing the technical performance of VAAL systems, including the use of ROC curves and challenges in standardising evaluations.

Section 19: Conclusion
Description 19: Summarize the findings of the review, the challenges to be addressed, and the potential future directions for the development of video-based AAL technology.