Background subtraction methods in video streams: A review
---
paper_title: Background subtraction techniques: a review

paper_content:

Background subtraction is a widely used approach for detecting moving objects from static cameras. Many different methods have been proposed over the recent years and both the novice and the expert can be confused about their benefits and limitations. In order to overcome this problem, this paper provides a review of the main methods and an original categorisation based on speed, memory requirements and accuracy. Such a review can effectively guide the designer to select the most suitable method for a given application in a principled way. Methods reviewed include parametric and non-parametric background density estimates and spatial correlation approaches.
---
paper_title: Performance Evaluation Metrics for Video Tracking

paper_content:

AbstractVarious tracking methods have been developed to track objects with different degrees or levels of tracking ability. The ability or performance of each tracking method is dependent on the feature or data that is being used for tracking purpose. The ability of a tracking method can be measured by utilizing tracking metrics to give an indication of the tracking ability of an algorithm. This paper offers some insights into the issues and similarities of performance measurement reporting of video tracking algorithms and proposes a method in assessing the robustness of a video tracking algorithm. The proposed metric introduces another measure to measure the consistency of a tracking algorithm. The work presented in this paper shows that using only one metric to measure the tracking performance is inadequate. The proposed metric presented in this paper shows that the utilization of multiple metrics such as tracking success rate and tracking consistency or robustness would give a better indication of the ...
---
paper_title: Motion of disturbances: detection and tracking of multi-body non-rigid motion

paper_content:

We present a new approach to the tracking of very non rigid patterns of motion, such as water flowing down a stream. The algorithm is based on a "disturbance map", which is obtained by linearly subtracting the temporal average of the previous frames from the new frame. Every local motion creates a disturbance having the form of a wave, with a "head" at the present position of the motion and a historical "tail" that indicates the previous locations of that motion. These disturbances serve as loci of attraction for "tracking particles" that are scattered throughout the image. The algorithm is very fast and can be performed in real time. We provide excellent tracking results on various complex sequences, using both stabilized and moving cameras, showing: a busy ant column, waterfalls, rapids and, flowing streams, shoppers in a mall, and cars in a traffic intersection.
---
paper_title: Detecting Moving Objects, Ghosts, and Shadows in Video Streams

paper_content:

Background subtraction methods are widely exploited for moving object detection in videos in many applications, such as traffic monitoring, human motion capture, and video surveillance. How to correctly and efficiently model and update the background model and how to deal with shadows are two of the most distinguishing and challenging aspects of such approaches. The article proposes a general-purpose method that combines statistical assumptions with the object-level knowledge of moving objects, apparent objects (ghosts), and shadows acquired in the processing of the previous frames. Pixels belonging to moving objects, ghosts, and shadows are processed differently in order to supply an object-based selective update. The proposed approach exploits color information for both background subtraction and shadow detection to improve object segmentation and background update. The approach proves fast, flexible, and precise in terms of both pixel accuracy and reactivity to background changes.
---
paper_title: Background subtraction techniques: a review

paper_content:

Background subtraction is a widely used approach for detecting moving objects from static cameras. Many different methods have been proposed over the recent years and both the novice and the expert can be confused about their benefits and limitations. In order to overcome this problem, this paper provides a review of the main methods and an original categorisation based on speed, memory requirements and accuracy. Such a review can effectively guide the designer to select the most suitable method for a given application in a principled way. Methods reviewed include parametric and non-parametric background density estimates and spatial correlation approaches.
---
paper_title: A real-time system for monitoring of cyclists and pedestrians

paper_content:

Camera based fixed systems are routinely used for monitoring highway traffic. For this purpose inductive loops and microwave sensors are mainly used. Both techniques achieve very good counting accuracy and are capable of discriminating trucks and cars. However pedestrians and cyclists are mostly counted manually. In this paper, we describe a new camera based automatic system that utilizes Kalman filtering in tracking and Learning Vector Quantization (LVQ) for classifying the observations to pedestrians and cyclists. Both the requirements for such systems and the algorithms used are described. The tests performed show that the system achieves around 80%-90% accuracy in counting and classification.
---
paper_title: An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection

paper_content:

Real-time segmentation of moving regions in image sequences is a fundamental step in many vision systems including automated visual surveillance, human-machine interface, and very low-bandwidth telecommunications. A typical method is background subtraction. Many background models have been introduced to deal with different problems. One of the successful solutions to these problems is to use a multi-colour background model per pixel proposed by Grimson et al [1, 2,3]. However, the method suffers from slow learning at the beginning, especially in busy environments. In addition, it can not distinguish between moving shadows and moving objects. This paper presents a method which improves this adaptive background mixture model. By reinvestigating the update equations, we utilise different equations at different phases. This allows our system learn faster and more accurately as well as adapts effectively to changing environment. A shadow detection scheme is also introduced in this paper. It is based on a computational colour space that makes use of our background model. A comparison has been made between the two algorithms. The results show the speed of learning and the accuracy of the model using our update algorithm over the Grimson et al’s tracker. When incorporate with the shadow detection, our method results in far better segmentation than The Thirteenth Conference on Uncertainty in Artificial Intelligence that of Grimson et al.
---
paper_title: An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection

paper_content:

Real-time segmentation of moving regions in image sequences is a fundamental step in many vision systems including automated visual surveillance, human-machine interface, and very low-bandwidth telecommunications. A typical method is background subtraction. Many background models have been introduced to deal with different problems. One of the successful solutions to these problems is to use a multi-colour background model per pixel proposed by Grimson et al [1, 2,3]. However, the method suffers from slow learning at the beginning, especially in busy environments. In addition, it can not distinguish between moving shadows and moving objects. This paper presents a method which improves this adaptive background mixture model. By reinvestigating the update equations, we utilise different equations at different phases. This allows our system learn faster and more accurately as well as adapts effectively to changing environment. A shadow detection scheme is also introduced in this paper. It is based on a computational colour space that makes use of our background model. A comparison has been made between the two algorithms. The results show the speed of learning and the accuracy of the model using our update algorithm over the Grimson et al’s tracker. When incorporate with the shadow detection, our method results in far better segmentation than The Thirteenth Conference on Uncertainty in Artificial Intelligence that of Grimson et al.
---
paper_title: Robust techniques for background subtraction in urban traffic video

paper_content:

Identifying moving objects from a video sequence is a fundamental and ::: critical task in many computer-vision applications. A common approach ::: is to perform background subtraction, which identifies moving objects ::: from the portion of a video frame that differs significantly from a ::: background model. There are many challenges in developing a good ::: background subtraction algorithm. First, it must be robust against ::: changes in illumination. Second, it should avoid detecting ::: non-stationary background objects such as swinging leaves, rain, snow, ::: and shadow cast by moving objects. Finally, its internal background ::: model should react quickly to changes in background such as starting ::: and stopping of vehicles. In this paper, we compare various background subtraction algorithms for detecting moving vehicles and pedestrians in urban traffic video sequences. We consider approaches varying from simple techniques such as frame differencing and adaptive median filtering, to more sophisticated probabilistic modeling techniques. While complicated techniques often produce superior performance, our experiments show that simple techniques such as adaptive median filtering can produce good results with much lower computational complexity.
---
paper_title: Evaluation of background subtraction techniques for video surveillance

paper_content:

Background subtraction is one of the key techniques for automatic video analysis, especially in the domain of video surveillance. Although its importance, evaluations of recent background subtraction methods with respect to the challenges of video surveillance suffer from various shortcomings. To address this issue, we first identify the main challenges of background subtraction in the field of video surveillance. We then compare the performance of nine background subtraction methods with post-processing according to their ability to meet those challenges. Therefore, we introduce a new evaluation data set with accurate ground truth annotations and shadow masks. This enables us to provide precise in-depth evaluation of the strengths and drawbacks of background subtraction methods.
---
paper_title: Motion of disturbances: detection and tracking of multi-body non-rigid motion

paper_content:

We present a new approach to the tracking of very non rigid patterns of motion, such as water flowing down a stream. The algorithm is based on a "disturbance map", which is obtained by linearly subtracting the temporal average of the previous frames from the new frame. Every local motion creates a disturbance having the form of a wave, with a "head" at the present position of the motion and a historical "tail" that indicates the previous locations of that motion. These disturbances serve as loci of attraction for "tracking particles" that are scattered throughout the image. The algorithm is very fast and can be performed in real time. We provide excellent tracking results on various complex sequences, using both stabilized and moving cameras, showing: a busy ant column, waterfalls, rapids and, flowing streams, shoppers in a mall, and cars in a traffic intersection.
---
paper_title: Performance Evaluation Metrics for Video Tracking

paper_content:

AbstractVarious tracking methods have been developed to track objects with different degrees or levels of tracking ability. The ability or performance of each tracking method is dependent on the feature or data that is being used for tracking purpose. The ability of a tracking method can be measured by utilizing tracking metrics to give an indication of the tracking ability of an algorithm. This paper offers some insights into the issues and similarities of performance measurement reporting of video tracking algorithms and proposes a method in assessing the robustness of a video tracking algorithm. The proposed metric introduces another measure to measure the consistency of a tracking algorithm. The work presented in this paper shows that using only one metric to measure the tracking performance is inadequate. The proposed metric presented in this paper shows that the utilization of multiple metrics such as tracking success rate and tracking consistency or robustness would give a better indication of the ...
---
paper_title: An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection

paper_content:

Real-time segmentation of moving regions in image sequences is a fundamental step in many vision systems including automated visual surveillance, human-machine interface, and very low-bandwidth telecommunications. A typical method is background subtraction. Many background models have been introduced to deal with different problems. One of the successful solutions to these problems is to use a multi-colour background model per pixel proposed by Grimson et al [1, 2,3]. However, the method suffers from slow learning at the beginning, especially in busy environments. In addition, it can not distinguish between moving shadows and moving objects. This paper presents a method which improves this adaptive background mixture model. By reinvestigating the update equations, we utilise different equations at different phases. This allows our system learn faster and more accurately as well as adapts effectively to changing environment. A shadow detection scheme is also introduced in this paper. It is based on a computational colour space that makes use of our background model. A comparison has been made between the two algorithms. The results show the speed of learning and the accuracy of the model using our update algorithm over the Grimson et al’s tracker. When incorporate with the shadow detection, our method results in far better segmentation than The Thirteenth Conference on Uncertainty in Artificial Intelligence that of Grimson et al.
---
paper_title: Review and evaluation of commonly-implemented background subtraction algorithms

paper_content:

Locating moving objects in a video sequence is the first step of many computer vision applications. Among the various motion-detection techniques, background subtraction methods are commonly implemented, especially for applications relying on a fixed camera. Since the basic inter-frame difference with global threshold is often a too simplistic method, more elaborate (and often probabilistic) methods have been proposed. These methods often aim at making the detection process more robust to noise, background motion and camera jitter. In this paper, we present commonly-implemented background subtraction algorithms and we evaluate them quantitatively. In order to gauge performances of each method, tests are performed on a wide range of real, synthetic and semi-synthetic video sequences representing different challenges.
---
paper_title: Background segmentation with feedback: The Pixel-Based Adaptive Segmenter

paper_content:

In this paper we present a novel method for foreground segmentation. Our proposed approach follows a non-parametric background modeling paradigm, thus the background is modeled by a history of recently observed pixel values. The foreground decision depends on a decision threshold. The background update is based on a learning parameter. We extend both of these parameters to dynamic per-pixel state variables and introduce dynamic controllers for each of them. Furthermore, both controllers are steered by an estimate of the background dynamics. In our experiments, the proposed Pixel-Based Adaptive Segmenter (PBAS) outperforms most state-of-the-art methods.
---
paper_title: Adaptive background mixture models for real-time tracking

paper_content:

A common method for real-time segmentation of moving regions in image sequences involves "background subtraction", or thresholding the error between an estimate of the image without moving objects and the current image. The numerous approaches to this problem differ in the type of background model used and the procedure used to update the model. This paper discusses modeling each pixel as a mixture of Gaussians and using an on-line approximation to update the model. The Gaussian, distributions of the adaptive mixture model are then evaluated to determine which are most likely to result from a background process. Each pixel is classified based on whether the Gaussian distribution which represents it most effectively is considered part of the background model. This results in a stable, real-time outdoor tracker which reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes. This system has been run almost continuously for 16 months, 24 hours a day, through rain and snow.
---
paper_title: Background Modeling using Mixture of Gaussians for Foreground Detection-A Survey

paper_content:

Mixture of Gaussians is a widely used approach for background modeling to detect moving objects from static cameras. Numerous improvements of the original method developed by Stauffer and Grimson [1] have been proposed over the recent years and the purpose of this paper is to provide a survey and an original classification of these improvements. We also discuss relevant issues to reduce the computation time. Firstly, the original MOG are reminded and discussed following the challenges met in video sequences. Then, we categorize the different improvements found in the literature. We have classified them in term of strategies used to improve the original MOG and we have discussed them in term of the critical situations they claim to handle. After analyzing the strategies and identifying their limitations, we conclude with several promising directions for future research.
---
paper_title: A Fuzzy Background Modeling Approach for Motion Detection in Dynamic Backgrounds

paper_content:

Based on Type-2 Fuzzy Gaussian Mixture Model (T2-FGMM) and Markov Random Field (MRF), we propose a novel background modeling method for motion detection in dynamic scenes. The key idea of the proposed approach is the successful introduction of the spatial-temporal constraints into the T2-FGMM by a Bayesian framework. The evaluation results in pixel level demonstrate that the proposed method performs better than the sound Gaussian Mixture Model (GMM) and T2-FGMM in such typical dynamic backgrounds as waving trees and water rippling.
---
paper_title: Background segmentation with feedback: The Pixel-Based Adaptive Segmenter

paper_content:

In this paper we present a novel method for foreground segmentation. Our proposed approach follows a non-parametric background modeling paradigm, thus the background is modeled by a history of recently observed pixel values. The foreground decision depends on a decision threshold. The background update is based on a learning parameter. We extend both of these parameters to dynamic per-pixel state variables and introduce dynamic controllers for each of them. Furthermore, both controllers are steered by an estimate of the background dynamics. In our experiments, the proposed Pixel-Based Adaptive Segmenter (PBAS) outperforms most state-of-the-art methods.
---
paper_title: Removal of Fog from Images: A Review

paper_content:

AbstractIn this paper, reported algorithms for the removal of fog are reviewed. Fog reduces the visibility of scene and thus performance of various computer vision algorithms which use feature information. Formation of fog is the function of the depth. Estimation of depth information is under constraint problem if single image is available. Hence, removal of fog requires assumptions or prior information. Fog removal algorithms estimate the depth information with various assumptions, which are discussed in detail here. Fog removal algorithm has a wide application in tracking and navigation, consumer electronics, and entertainment industries.
---
paper_title: Motion-based background subtraction using adaptive kernel density estimation

paper_content:

Background modeling is an important component of many vision systems. Existing work in the area has mostly addressed scenes that consist of static or quasi-static structures. When the scene exhibits a persistent dynamic behavior in time, such an assumption is violated and detection performance deteriorates. In this paper, we propose a new method for the modeling and subtraction of such scenes. Towards the modeling of the dynamic characteristics, optical flow is computed and utilized as a feature in a higher dimensional space. Inherent ambiguities in the computation of features are addressed by using a data-dependent bandwidth for density estimation using kernels. Extensive experiments demonstrate the utility and performance of the proposed approach.
---
paper_title: Vehicle trajectories evaluation by static video sensors

paper_content:

Metrology of vehicle trajectories has several applications in the field of road safety, particularly in dangerous curves. Actually, it is of great interest to observe trajectories of vehicles with the aim of designing a real time driver warning device in dangerous areas. This paper addresses the first step of a work with a video system placed along the road with the objective of vehicle's position and speed estimation. This system has been totally developed for this project and can record simultaneously three cameras with 640 times 480 pixels up to 30 frames per second (fps) and rangefinder informations. The best contribution of this paper is an original probabilistic background subtraction algorithm, first step of a global method (calibration, tracking, ...) implemented to be able to measure vehicle trajectories. Kinematic GPS (in post-processing) has been extensively used to get ground truth
---
paper_title: Visual tracking of human visitors under variable-lighting conditions for a responsive audio art installation

paper_content:

For a responsive audio art installation in a skylit atrium, we introduce a single-camera statistical segmentation and tracking algorithm. The algorithm combines statistical background image estimation, per-pixel Bayesian segmentation, and an approximate solution to the multi-target tracking problem using a bank of Kalman filters and Gale-Shapley matching. A heuristic confidence model enables selective filtering of tracks based on dynamic data. We demonstrate that our algorithm has improved recall and F 2 -score over existing methods in OpenCV 2.1 in a variety of situations. We further demonstrate that feedback between the tracking and the segmentation systems improves recall and F 2 -score. The system described operated effectively for 5–8 hours per day for 4 months; algorithms are evaluated on video from the camera installed in the atrium. Source code and sample data is open source and available in OpenCV.
---
paper_title: Background subtraction techniques: a review

paper_content:

Background subtraction is a widely used approach for detecting moving objects from static cameras. Many different methods have been proposed over the recent years and both the novice and the expert can be confused about their benefits and limitations. In order to overcome this problem, this paper provides a review of the main methods and an original categorisation based on speed, memory requirements and accuracy. Such a review can effectively guide the designer to select the most suitable method for a given application in a principled way. Methods reviewed include parametric and non-parametric background density estimates and spatial correlation approaches.
---
paper_title: A Bayesian computer vision system for modeling human interactions

paper_content:

We describe a real-time computer vision and machine learning system for modeling and recognizing human behaviors in a visual surveillance task. The system deals in particularly with detecting when interactions between people occur and classifying the type of interaction. Examples of interesting interaction behaviors include following another person, altering one's path to meet another, and so forth. Our system combines top-down with bottom-up information in a closed feedback loop, with both components employing a statistical Bayesian approach. We propose and compare two different state-based learning architectures, namely, HMMs and CHMMs for modeling behaviors and interactions. Finally, a synthetic "Alife-style" training system is used to develop flexible prior models for recognizing human interactions. We demonstrate the ability to use these a priori models to accurately classify real human behaviors and interactions with no additional tuning or training.
---
paper_title: An Efficient Neural Network Based Background Subtraction Method

paper_content:

The paper presents a neural network based segmentation method which can extract moving objects in video. This proposed neural network architecture is multilayer so as to match the complexity of the frames in a video stream and deal with the problems of segmentation. The neural network combines inputs that exploit spatio-temporal correlation among pixels. Each of these unit themselves produce imperfect results, but the neural network learns to combine their results for better overall segmentation, even though it is trained with noisy results from a simpler method. The proposed algorithm converges from an initial stage where all the pixels are considered to be part of the background to a stage where only the appropriate pixels are classified as background. Results are shown to demonstrate the efficacy of the method compared to a more memory intensive MoG method.
---
paper_title: Fast Robust Eigen-Background Updating for Foreground Detection

paper_content:

A fast robust eigen-background update algorithm is proposed for foreground object detection. The update procedure involves no eigen decomposition, thus faster than former eigen-background based algorithms. Meanwhile, the algorithm can robustly maintain the desired background model, resistant to outlying objects.
---

Title: Background Subtraction Methods in Video Streams: A Review

Section 1: INTRODUCTION
Description 1: Introduce the concept of background subtraction, its applications, and its importance in video processing and computer vision.

Section 2: METHODS
Description 2: Describe various background subtraction methods, classified into their respective categories, including conventional, statistical, fuzzy-based, non-parametric, mean-shift based estimation, and combined estimation and propagation methods.

Section 3: Conventional Methods
Description 3: Discuss traditional methods like Median Filtering, Frame Difference, Average Filtering, MIN-MAX Filtering, Approximated Median Filtering, Single Gaussian Filtering, Kalman Filtering, and Hidden Markov Models.

Section 4: Statistical Methods
Description 4: Discuss statistical approaches including single Gaussian, Gaussian Mixture Model (GMM), and techniques that address animated textures using multimodal densities.

Section 5: Fuzzy Based Methods
Description 5: Explain fuzzy logic based methods for background subtraction, including Fuzzy Sugeno Integral, Fuzzy Choquet Integral, and Fuzzy Gaussian techniques.

Section 6: Non-Parametric Methods
Description 6: Explore non-parametric methods such as Kernel Density Estimation (KDE) and other advanced pixel-based segmentation approaches.

Section 7: Mean-Shift Based Estimation
Description 7: Detail methods involving mean-shift based estimation for detecting modes in multimodal distributions.

Section 8: Combined Estimation and Propagation
Description 8: Discuss hybrid methods that combine estimation and propagation techniques for better performance and lower memory requirements.

Section 9: Methods Based on Eigen Features
Description 9: Explain techniques that utilize eigen features and principal component analysis for background subtraction in video streams.

Section 10: CHALLENGES OF BACKGROUND SUBTRACTION FOR VIDEO SURVEILLANCE
Description 10: Address the various challenges faced in background subtraction applications, such as illumination changes, dynamic backgrounds, bootstrapping, video noise, and camouflage.

Section 11: DISCUSSION
Description 11: Provide a critical discussion of recent advancements, comparing different background subtraction methods and highlighting their strengths and weaknesses.

Section 12: CONCLUSIONS
Description 12: Summarize the findings of the review, discussing the comparative effectiveness of different methods, their computational requirements, and suitability for real-time applications.