A Comprehensive Survey of Video Datasets for Background Subtraction
---
paper_title: Adaptive weighted non-parametric background model for efficient video coding

paper_content:

Dynamic background frame based video coding using mixture of Gaussian (MoG) based background modelling has achieved better rate distortion performance compared to the H.264 standard. However, they suffer from high computation time, low coding efficiency for dynamic videos, and prior knowledge requirement of video content. In this paper, we introduce the application of the non-parametric (NP) background modelling approach for video coding domain. We present a novel background modelling technique, called weighted non-parametric (WNP) which balances the historical trend and the recent value of the pixel intensities adaptively based on the content and characteristics of any particular video. WNP is successfully embedded into the latest HEVC video coding standard for better rate-distortion performance. Moreover, a novel scene adaptive non-parametric (SANP) technique is also developed to handle video sequences with high dynamic background. Being non-parametric, the proposed techniques naturally exhibit superior performance in dynamic background modelling without a priori knowledge of video data distribution.
---
paper_title: Exemplar-based background model initialization

paper_content:

Most of the automated video-surveillance applications are based on background (BG) subtraction techniques, that aim at distinguishing moving objects in a static scene. These strategies strongly depend on the BG model, that has to be initialized and updated. A good initialization is crucial for the successive processing. In this paper, we propose a novel method for BG initialization and recovery, that merges interesting ideas coming from the video inpainting and the generative modelling subfields. The method takes as input a video sequence, in which several objects move in front of a stationary BG. Then, a statistical representation of the BG is iteratively built, discarding automatically the moving objects. The method is based on the following hypotheses: (i) a portion of the BG, called sure BG, can be identified with high certainty by using only per-pixel reasoning and (ii) the remaining scene BG can be generated utilizing exemplars of the sure BG. The proposed algorithm is able to exploit these hypotheses in a principled and effective way.
---
paper_title: Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset

paper_content:

Recent research on problem formulations based on decomposition into low-rank plus sparse matrices shows a suitable framework to separate moving objects from the background. The most representative problem formulation is the Robust Principal Component Analysis (RPCA) solved via Principal Component Pursuit (PCP) which decomposes a data matrix in a low-rank matrix and a sparse matrix. However, similar robust implicit or explicit decompositions can be made in the following problem formulations: Robust Non-negative Matrix Factorization (RNMF), Robust Matrix Completion (RMC), Robust Subspace Recovery (RSR), Robust Subspace Tracking (RST) and Robust Low-Rank Minimization (RLRM). The main goal of these similar problem formulations is to obtain explicitly or implicitly a decomposition into low-rank matrix plus additive matrices. In this context, this work aims to initiate a rigorous and comprehensive review of the similar problem formulations in robust subspace learning and tracking based on decomposition into low-rank plus additive matrices for testing and ranking existing algorithms for background/foreground separation. For this, we first provide a preliminary review of the recent developments in the different problem formulations which allows us to define a unified view that we called Decomposition into Low-rank plus Additive Matrices (DLAM). Then, we examine carefully each method in each robust subspace learning/tracking frameworks with their decomposition, their loss functions, their optimization problem and their solvers. Furthermore, we investigate if incremental algorithms and real-time implementations can be achieved for background/foreground separation. Finally, experimental results on a large-scale dataset called Background Models Challenge (BMC 2012) show the comparative performance of 32 different robust subspace learning/tracking methods.
---
paper_title: Moving Object Detection Using Scanning Camera on a High-Precision Intelligent Holder

paper_content:

During the process of moving object detection in an intelligent visual surveillance system, a scenario with complex background is sure to appear. The traditional methods, such as “frame difference” and “optical flow”, may not able to deal with the problem very well. In such scenarios, we use a modified algorithm to do the background modeling work. In this paper, we use edge detection to get an edge difference image just to enhance the ability of resistance illumination variation. Then we use a “multi-block temporal-analyzing LBP (Local Binary Pattern)” algorithm to do the segmentation. In the end, a connected component is used to locate the object. We also produce a hardware platform, the core of which consists of the DSP (Digital Signal Processor) and FPGA (Field Programmable Gate Array) platforms and the high-precision intelligent holder.
---
paper_title: Extensive Benchmark and Survey of Modeling Methods for Scene Background Initialization

paper_content:

Scene background initialization is the process by which a method tries to recover the background image of a video without foreground objects in it. Having a clear understanding about which approach is more robust and/or more suited to a given scenario is of great interest to many end users or practitioners. The aim of this paper is to provide an extensive survey of scene background initialization methods as well as a novel benchmarking framework. The proposed framework involves several evaluation metrics and state-of-the-art methods, as well as the largest video data set ever made for this purpose. The data set consists of several camera-captured videos that: 1) span categories focused on various background initialization challenges; 2) are obtained with different cameras of different lengths, frame rates, spatial resolutions, lighting conditions, and levels of compression; and 3) contain indoor and outdoor scenes. The wide variety of our data set prevents our analysis from favoring a certain family of background initialization methods over others. Our evaluation framework allows us to quantitatively identify solved and unsolved issues related to scene background initialization. We also identify scenarios for which state-of-the-art methods systematically fail.
---
paper_title: Interactive digital photomontage

paper_content:

We describe an interactive, computer-assisted framework for combining parts of a set of photographs into a single composite picture, a process we call "digital photomontage." Our framework makes use of two techniques primarily: graph-cut optimization, to choose good seams within the constituent images so that they can be combined as seamlessly as possible; and gradient-domain fusion, a process based on Poisson equations, to further reduce any remaining visible artifacts in the composite. Also central to the framework is a suite of interactive tools that allow the user to specify a variety of high-level image objectives, either globally across the image, or locally through a painting-style interface. Image objectives are applied independently at each pixel location and generally involve a function of the pixel values (such as "maximum contrast") drawn from that same location in the set of source images. Typically, a user applies a series of image objectives iteratively in order to create a finished composite. The power of this framework lies in its generality; we show how it can be used for a wide variety of applications, including "selective composites" (for instance, group photos in which everyone looks their best), relighting, extended depth of field, panoramic stitching, clean-plate production, stroboscopic visualization of movement, and time-lapse mosaics.
---
paper_title: A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos

paper_content:

Abstract Background subtraction (BS) is a crucial step in many computer vision systems, as it is first applied to detect moving objects within a video stream. Many algorithms have been designed to segment the foreground objects from the background of a sequence. In this article, we propose to use the BMC (Background Models Challenge) dataset, and to compare the 29 methods implemented in the BGSLibrary. From this large set of various BG methods, we have conducted a relevant experimental analysis to evaluate both their robustness and their practical performance in terms of processor/memory requirements.
---
paper_title: Stand-alone quality estimation of background subtraction algorithms

paper_content:

Abstract Foreground segmentation is a key stage in multiple computer vision applications, where existing algorithms are commonly evaluated making use of ground-truth data. Reference-free or stand-alone evaluations that estimate segmented foreground quality are an alternative methodology to overcome the limitations inherent to ground-truth based evaluations. In this work, we survey and explore existing stand-alone measures proposed in related research areas to determine good object properties for estimating the segmentation quality in background subtraction algorithms. We propose a new taxonomy for stand-alone evaluation measures and analyze 21 proposals. We demonstrate the utility of the selected measures to evaluate the segmentation masks of eight background subtraction algorithms. The experiments are performed over a large heterogeneous dataset with varied challenges (CDNET2014) and identify which properties of the measures are the most effective to estimate quality. The experiments also demonstrate that qualitative performance levels can be distinguished and background subtraction algorithms can be ranked without the need of ground-truth.
---
paper_title: An Evaluation of Background Subtraction for Object Detection Vis-a-Vis Mitigating Challenging Scenarios

paper_content:

Background subtraction is a popular technique for detecting objects moving across a fixed camera view. The performance of this paradigm is influenced by various challenges, such as object relocation, illumination change, cast shadows, waving background, camera shake, bootstrapping, camouflage, and so on. In this paper, we present a synopsis on the evolution of the background subtraction techniques over the last two decades. The different ways of mathematical modeling are taken into consideration to categorize the methods. We also evaluate the performance of some of the state-of-the-art techniques vis-a-vis the challenges associated. Eleven different algorithms of background subtraction have been simulated on thirty-four image sequences collected from five benchmark datasets. For each image sequence, seven performance metrics are evaluated and an exhaustive comparative analysis has been made to derive inferences. The potential findings in the result analysis are presented for future exploration. The obtained image and video results are uploaded at https://sites.google.com/site/soaBSevaluation .
---
paper_title: Background Subtraction for Automated Multisensor Surveillance: A Comprehensive Review

paper_content:

Background subtraction is a widely used operation in the video surveillance, aimed at separating the expected scene (the background) from the unexpected entities (the foreground). There are several problems related to this task, mainly due to the blurred boundaries between background and foreground definitions. Therefore, background subtraction is an open issue worth to be addressed under different points of view. In this paper, we propose a comprehensive review of the background subtraction methods, that considers also channels other than the sole visible optical one (such as the audio and the infrared channels). In addition to the definition of novel kinds of background, the perspectives that these approaches open up are very appealing: in particular, the multisensor direction seems to be well-suited to solve or simplify several hoary background subtraction problems. All the reviewed methods are organized in a novel taxonomy that encapsulates all the brand-new approaches in a seamless way.
---
paper_title: Background Subtraction in Real Applications: Challenges, Current Models and Future Directions

paper_content:

Abstract Computer vision applications based on videos often require the detection of moving objects in their first step. Background subtraction is then applied in order to separate the background and the foreground. In literature, background subtraction is surely among the most investigated field in computer vision providing a big amount of publications. Most of them concern the application of mathematical and machine learning models to be more robust to the challenges met in videos. However, the ultimate goal is that the background subtraction methods developed in research could be employed in real applications like traffic surveillance. But looking at the literature, we can remark that there is often a gap between the current methods used in real applications and the current methods in fundamental research. In addition, the videos evaluated in large-scale datasets are not exhaustive in the way that they only covered a part of the complete spectrum of the challenges met in real applications. In this context, we attempt to provide the most exhaustive survey as possible on real applications that used background subtraction in order to identify the real challenges met in practice, the current used background models and to provide future directions. Thus, challenges are investigated in terms of camera (i.e CCD cameras, omnidirectional cameras, …), foreground objects and environments. In addition, we identify the background models that are effectively used in these applications in order to find potential usable recent background models in terms of robustness, time and memory requirements.
---
paper_title: Design and implementation of video synopsis using online video inpainting

paper_content:

Nowadays with a rapid growth of multimedia in-formation and advances in digital video technologies number of video captured is increasing day by day. The most of the recorded video which was generated is far more than the operators can watch, so identifying any interesting object or event has become very time consuming task, this kind of task might be feasible for some personal videos but this tedious task impossible when there will be endless video. So video synopsis acts as productive tool for indexing and browsing of such lengthy video. Video synopsis is a way of creating a compact representation of original video by presenting essential activities. In this paper video inpainting algorithm is proposed to generate video synopsis. Video inpainting is the method which aims to remove portion of video or object and restoring that missing portion by visually steady manner. In this paper video inpainting algorithm is implemented to inpaint the missing portion of a video sequences. This algorithm includes two stages one is simple processing and another one is video inpainting. In first stage foreground and background frame segmentation will be done. In second stage to inpaint the background objects image inpainting method and background replacement method will be used. To inpaint moving foreground objects background subtraction and object segmentation is used.
---
paper_title: High-Speed Target Tracking System Based on a Hierarchical Parallel Vision Processor and Gray-Level LBP Algorithm

paper_content:

Visual target tracking has made significant advances in past decades. However, fast and robust vision target tracking systems are still greatly demanded. This paper proposes a novel high-speed target tracking system based on hierarchical parallel vision processor architecture. This system contains three main parts: 1) a CMOS image sensor; 2) a vision processor; and 3) an actuator with two degrees of freedom. The vision processor integrates a pixel-parallel processing element (PE) array, a row-parallel row processor (RP) array, dual-core microprocessor unit (MPU) and motor controller. The PE array and RP array can speed up low-level and middle-level image processing operations by ${O(M^{2})}$ and ${O}$ ( ${M}$ ), respectively. The MPU is responsible for the high-level image processing and the overall chip management. A novel tracking algorithm based on a gray-level local binary pattern descriptor is proposed. The descriptor describes not only local texture feature but also distribution of luminance. The algorithm increases the robustness of the tracking system under low resolution scenery and complex background. It can be carried out by the vision processor with very high efficiency. Experiment results demonstrate that the system can track a fast moving target under complex conditions and the vision processor can achieve over 2000 frames/s processing speed of the target tracking algorithm with $ {750\times 480}$ image resolution.
---
paper_title: Traditional and Recent Approaches in Background Modeling for Foreground Detection: An Overview

paper_content:

Background modeling for foreground detection is often used in different applications ::: to model the background and then detect the moving objects in the scene like in video ::: surveillance. The last decade witnessed very significant publications in this field. Furthermore, ::: several surveys can be found in literature but none of them addresses an overall ::: review in this field. So, the purpose of this paper is to provide a complete survey ::: of the traditional and recent approaches. First, we categorize the different approaches ::: found in literature. We have classified them in terms of the mathematical models used ::: and we have discussed them in terms of the critical situations that they claim to handle. ::: Furthermore, we present the available resources, datasets and libraries. Then, we ::: conclude with several promising directions for future research.
---
paper_title: Scene background initialization: A taxonomy

paper_content:

The availability of an initial background model that describes the scene without foreground objects is the prerequisite for a wide range of applications, ranging from video surveillance to computational photography. Limited attention to the problem is given in the literature on background modeling, that mainly regards model representation and updating. Therefore, we propose a taxonomy study for background initialization, providing the basis for a fair and easy comparison of existing and future methods, on a common dataset of groundtruthed sequences, with a common set of metrics, and based on reproducible results. Experimental results highlight the most promising approaches as well as main open issues for background initialization.
---
paper_title: Comparison of Background Subtraction Methods for a Multimedia Learning Space.

paper_content:

A press-formed article has a quenched area and a required strength distribution. The quenched area has a hardness distribution exhibiting a hardness transition which corresponds to the required strength distribution. The press-formed article may include one end portion, the other end portion, and a central area disposed between the one end portion and the other end portion. The quenched area is extended from the one end portion and to the other end portion. The hardness distribution of the quenched area exhibits a condition that hardness is the highest in the central area in such a manner that hardness is gradually decreased from the central area to the one end portion and to the other end portion.
---
paper_title: Human Body Model Acquisition and Tracking Using Voxel Data

paper_content:

We present an integrated system for automatic acquisition of the human body model and motion tracking using input from multiple synchronized video streams. The video frames are segmented and the 3D voxel reconstructions of the human body shape in each frame are computed from the foreground silhouettes. These reconstructions are then used as input to the model acquisition and tracking algorithms. ::: ::: The human body model consists of ellipsoids and cylinders and is described using the twists framework resulting in a non-redundant set of model parameters. Model acquisition starts with a simple body part localization procedure based on template fitting and growing, which uses prior knowledge of average body part shapes and dimensions. The initial model is then refined using a Bayesian network that imposes human body proportions onto the body part size estimates. The tracker is an extended Kalman filter that estimates model parameters based on the measurements made on the labeled voxel data. A voxel labeling procedure that handles large frame-to-frame displacements was designed resulting in very robust tracking performance. ::: ::: Extensive evaluation shows that the system performs very reliably on sequences that include different types of motion such as walking, sitting, dancing, running and jumping and people of very different body sizes, from a nine year old girl to a tall adult male.
---
paper_title: Image melding: combining inconsistent images using patch-based synthesis

paper_content:

Current methods for combining two different images produce visible artifacts when the sources have very different textures and structures. We present a new method for synthesizing a transition region between two source images, such that inconsistent color, texture, and structural properties all change gradually from one source to the other. We call this process image melding. Our method builds upon a patch-based optimization foundation with three key generalizations: First, we enrich the patch search space with additional geometric and photometric transformations. Second, we integrate image gradients into the patch representation and replace the usual color averaging with a screened Poisson equation solver. And third, we propose a new energy based on mixed L2/L0 norms for colors and gradients that produces a gradual transition between sources without sacrificing texture sharpness. Together, all three generalizations enable patch-based solutions to a broad class of image melding problems involving inconsistent sources: object cloning, stitching challenging panoramas, hole filling from multiple photos, and image harmonization. In several cases, our unified method outperforms previous state-of-the-art methods specifically designed for those applications.
---
paper_title: Recent Advanced Statistical Background Modeling for Foreground Detection - A Systematic Survey

paper_content:

Background modeling is currently used to detect moving objects in video acquired from static cameras. Numerous statistical methods have been developed over the recent years. The aim of this paper is firstly to provide an extended and updated survey of the recent researches and patents which concern statistical background modeling and secondly to achieve a comparative evaluation. For this, we firstly classified the statistical methods in terms of category. Then, the original methods are reminded and discussed following the challenges met in video sequences. We classified their respective improvements in terms of strategies used. Furthermore, we discussed them in terms of the critical situations they claim to handle. Finally, we conclude with several promising directions for future research. The survey also discussed relevant patents.
---
paper_title: Adaptive Multicue Background Subtraction for Robust Vehicle Counting and Classification

paper_content:

In this paper, we present a robust vision-based system for vehicle tracking and classification devised for traffic flow surveillance. The system performs in real time, achieving good results, even in challenging situations, such as with moving casted shadows on sunny days, headlight reflections on the road, rainy days, and traffic jams, using only a single standard camera. We propose a robust adaptive multicue segmentation strategy that detects foreground pixels corresponding to moving and stopped vehicles, even with noisy images due to compression. First, the approach adaptively thresholds a combination of luminance and chromaticity disparity maps between the learned background and the current frame. It then adds extra features derived from gradient differences to improve the segmentation of dark vehicles with casted shadows and removes headlight reflections on the road. The segmentation is further used by a two-step tracking approach, which combines the simplicity of a linear 2-D Kalman filter and the complexity of a 3-D volume estimation using Markov chain Monte Carlo (MCMC) methods. Experimental results show that our method can count and classify vehicles in real time with a high level of performance under different environmental situations comparable with those of inductive loop detectors.
---
paper_title: Intelligent Video Systems and Analytics: A Survey

paper_content:

Recent technology and market trends have demanded the significant need for feasible solutions to video/camera systems and analytics. This paper provides a comprehensive account on theory and application of intelligent video systems and analytics. It highlights the video system architectures, tasks, and related analytic methods. It clearly demonstrates that the importance of the role that intelligent video systems and analytics play can be found in a variety of domains such as transportation and surveillance. Research directions are outlined with a focus on what is essential to achieve the goals of intelligent video systems and analytics.
---
paper_title: Visual Tracking in Background Subtracted Image Sequences via Multi-Bernoulli Filtering

paper_content:

This correspondence presents a novel method for simultaneous tracking of multiple non-stationary targets in video. Our method operates directly on the video data and does not require any detection. We propose a multi-target likelihood function for the background-subtracted grey-scale image data, which admits multi-target conjugate priors. This allows the multi-target posterior to be efficiently propagated forward using the multi-Bernoulli filter. Our method does not need any training pattern or target templates and makes no prior assumptions about object types or object appearance. Case studies from the CAVIAR dataset show that our method can automatically track multiple targets and quickly finds targets entering or leaving the scene.
---
paper_title: Moving Object Detection and Shadow Removing under Changing Illumination Condition

paper_content:

Moving object detection is a fundamental step in video surveillance system. To eliminate the influence of illumination change and shadow associated with the moving objects, we proposed a local intensity ratio model (LIRM) which is robust to illumination change. Based on the analysis of the illumination and shadow model, we discussed the distribution of local intensity ratio. And the moving objects are segmented without shadow using normalized local intensity ratio via Gaussian ::: mixture model (GMM). Then erosion is used to get the moving objects contours and erase the scatter shadow patches and noises. After that, we get the enhanced moving objects contours by a new contour enhancement method, in which foreground ratio ::: and spatial relation are considered. At last, a new method is used to fill foreground with holes. Experimental results ::: demonstrate that the proposed approach can get moving objects without cast shadow and shows excellent performance under ::: various illumination change conditions.
---
paper_title: Illumination-Invariant Background Subtraction: Comparative Review, Models, and Prospects

paper_content:

Background subtraction is a key prerequisite for a wide range of image processing applications due to its pervasiveness in various contexts. In particular, video surveillance highly requires the reliable background subtraction for further operations, such as object tracking and recognition, and thus, enormous efforts for this task have been devoted in recent decades. However, the path of technological evolution for background subtraction has now faced with an important issue that has started to be resolved: sensitivity to dynamic changes of scene contexts (e.g., illumination variations and moving backgrounds). Such dynamic changes are hardly tolerated by most of traditional background models, since they yield the drastically different statistics of pixel values even onto the relevant position between consecutive frames. To resolve this problem, many researchers in this field have developed robust and efficient methods. The goal of this paper is to provide a comprehensive review with a special attention to schemes related to handling varying illuminations frequently occurring in the outdoor surveillance scenario. This paper covers a systematic taxonomy, methodologies, and performance evaluations on benchmark databases, and also provides constructive discussions for the smart video surveillance under unconstrained outdoor environments.
---
paper_title: A Hierarchical Model Incorporating Segmented Regions and Pixel Descriptors for Video Background Subtraction

paper_content:

Background subtraction is important for detecting moving objects in videos. Currently, there are many approaches to performing background subtraction. However, they usually neglect the fact that the background images consist of different objects whose conditions may change frequently. In this paper, a novel hierarchical background model is proposed based on segmented background images. It first segments the background images into several regions by the mean-shift algorithm. Then, a hierarchical model, which consists of the region models and pixel models, is created. The region model is a kind of approximate Gaussian mixture model extracted from the histogram of a specific region. The pixel model is based on the cooccurrence of image variations described by histograms of oriented gradients of pixels in each region. Benefiting from the background segmentation, the region models and pixel models corresponding to different regions can be set to different parameters. The pixel descriptors are calculated only from neighboring pixels belonging to the same object. The experimental results are carried out with a video database to demonstrate the effectiveness, which is applied to both static and dynamic scenes by comparing it with some well-known background subtraction methods.
---
paper_title: Moving Object Detection and Shadow Removing under Changing Illumination Condition

paper_content:

Moving object detection is a fundamental step in video surveillance system. To eliminate the influence of illumination change and shadow associated with the moving objects, we proposed a local intensity ratio model (LIRM) which is robust to illumination change. Based on the analysis of the illumination and shadow model, we discussed the distribution of local intensity ratio. And the moving objects are segmented without shadow using normalized local intensity ratio via Gaussian ::: mixture model (GMM). Then erosion is used to get the moving objects contours and erase the scatter shadow patches and noises. After that, we get the enhanced moving objects contours by a new contour enhancement method, in which foreground ratio ::: and spatial relation are considered. At last, a new method is used to fill foreground with holes. Experimental results ::: demonstrate that the proposed approach can get moving objects without cast shadow and shows excellent performance under ::: various illumination change conditions.
---
paper_title: Fusion-based foreground enhancement for background subtraction using multivariate multi-model Gaussian distribution

paper_content:

Abstract These days, detection of visual attention regions (VAR) such as moving objects have become an essential pre-processing stage in many computer vision applications. In this paper, we focus on the vital issue of separating moving objects a.k.a. Foreground (FG) in a scene, which has a near-static background (BG). We address the difficulty in setting an adaptive threshold in the multi-model Gaussian-based BG-FG separation through a novel FG enhancement strategy by assimilating color and illumination measures. We formulate the problem mathematically by using a histogram of a fused feature of color and illumination measures. The proposed method improves the BG-FG separation by introducing the following items: (i) A new distance measure to check if a pixel matches a Gaussian distribution. (ii) A new strategy to enhance the results of traditional background subtraction (BGS) with a fusion of color and illumination measures. (iii) A methodology to find appropriate threshold adaptively that separates BG and FG. (iv) A foreground validation process through probability estimation of multivariate Gaussian model distribution (MVGMD). We test the proposed algorithm on five different benchmark video sequences. The experimental results reveal that the proposed approach works well in challenging conditions, at the same time, it performs competitively against state-of-the-art Gaussian-based algorithms and few other non-Gaussian-based methods as well.
---
paper_title: Detecting Moving Shadows: Algorithms and Evaluation

paper_content:

Moving shadows need careful consideration in the development of robust dynamic scene analysis systems. Moving shadow detection is critical for accurate object detection in video streams since shadow points are often misclassified as object points, causing errors in segmentation and tracking. Many algorithms have been proposed in the literature that deal with shadows. However, a comparative evaluation of the existing approaches is still lacking. In this paper, we present a comprehensive survey of moving shadow detection approaches. We organize contributions reported in the literature in four classes two of them are statistical and two are deterministic. We also present a comparative empirical evaluation of representative algorithms selected from these four classes. Novel quantitative (detection and discrimination rate) and qualitative metrics (scene and object independence, flexibility to shadow situations, and robustness to noise) are proposed to evaluate these classes of algorithms on a benchmark suite of indoor and outdoor video sequences. These video sequences and associated "ground-truth" data are made available at http://cvrr.ucsd.edu/aton/shadow to allow for others in the community to experiment with new algorithms and metrics.
---
paper_title: ViBe: A Universal Background Subtraction Algorithm for Video Sequences

paper_content:

This paper presents a technique for motion detection that incorporates several innovative mechanisms. For example, our proposed technique stores, for each pixel, a set of values taken in the past at the same location or in the neighborhood. It then compares this set to the current pixel value in order to determine whether that pixel belongs to the background, and adapts the model by choosing randomly which values to substitute from the background model. This approach differs from those based upon the classical belief that the oldest values should be replaced first. Finally, when the pixel is found to be part of the background, its value is propagated into the background model of a neighboring pixel. We describe our method in full details (including pseudo-code and the parameter values used) and compare it to other background subtraction techniques. Efficiency figures show that our method outperforms recent and proven state-of-the-art methods in terms of both computation speed and detection rate. We also analyze the performance of a downscaled version of our algorithm to the absolute minimum of one comparison and one byte of memory per pixel. It appears that even such a simplified version of our algorithm performs better than mainstream techniques.
---
paper_title: A Bayesian Approach to Camouflaged Moving Object Detection

paper_content:

Moving object detection is about foreground and background separation based on motion detection. Detecting moving objects from similarly colored background (known as camouflage problem) has been a long-standing open question in this field. Discriminative modeling (DM), which focuses on enhancing the performance to distinguish foreground from background with discriminative features and well-designed classifiers, has been widely used for moving object detection. However, DM may tend to fail when encountering the camouflage problem, as the class separability in camouflaged areas is generally poor. In this paper, we propose a new strategy, camouflage modeling (CM), to identify camouflaged foreground pixels. In view of the fact that camouflage involves both foreground and background, we need to model both the background and the foreground, and compare them in a well-designed way in camouflage detection. Specifically, we develop a global model for the background, and an integration of global and local models for the foreground, respectively. Based on both background and foreground models, we introduce a factor to measure the degree of camouflage, and further identify truly camouflaged areas. In view of the fact that a moving object is usually composed of both camouflaged and noncamouflaged areas, CM and DM are fused in a Bayesian framework to perform complete object detection. Experiments are conducted on testing sequences to demonstrate the effectiveness of the proposed algorithm.
---
paper_title: SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity

paper_content:

Foreground/background segmentation via change detection in video sequences is often used as a stepping stone in high-level analytics and applications. Despite the wide variety of methods that have been proposed for this problem, none has been able to fully address the complex nature of dynamic scenes in real surveillance tasks. In this paper, we present a universal pixel-level segmentation method that relies on spatiotemporal binary features as well as color information to detect changes. This allows camouflaged foreground objects to be detected more easily while most illumination variations are ignored. Besides, instead of using manually set, frame-wide constants to dictate model sensitivity and adaptation speed, we use pixel-level feedback loops to dynamically adjust our method’s internal parameters without user intervention. These adjustments are based on the continuous monitoring of model fidelity and local segmentation noise levels. This new approach enables us to outperform all 32 previously tested state-of-the-art methods on the 2012 and 2014 versions of the ChangeDetection.net dataset in terms of overall F-Measure. The use of local binary image descriptors for pixel-level modeling also facilitates high-speed parallel implementations: our own version, which used no low-level or architecture-specific instruction, reached real-time processing speed on a midlevel desktop CPU. A complete C++ implementation based on OpenCV is available online.
---
paper_title: New trends on moving object detection in video images captured by a moving camera: A survey

paper_content:

Abstract This paper presents a survey on the latest methods of moving object detection in video sequences captured by a moving camera. Although many researches and excellent works have reviewed the methods of object detection and background subtraction for a fixed camera, there is no survey which presents a complete review of the existing different methods in the case of moving camera. Most methods in this field can be classified into four categories; modeling based background subtraction, trajectory classification, low rank and sparse matrix decomposition, and object tracking. We discuss in details each category and present the main methods which proposed improvements in the general concept of the techniques. We also present challenges and main concerns in this field as well as performance metrics and some benchmark databases available to evaluate the performance of different moving object detection algorithms.
---
paper_title: Moving Object Detection With a Freely Moving Camera via Background Motion Subtraction

paper_content:

Detection of moving objects in a video captured by a freely moving camera is a challenging problem in computer vision. Most existing methods often assume that the background (BG) can be approximated by dominant single plane/multiple planes or impose significant geometric constraints on BG, or utilize a complex BG/foreground probabilistic model. Instead, we propose a computationally efficient algorithm that is able to detect moving objects accurately and robustly in a general 3D scene. This problem is formulated as a coarse-to-fine thresholding scheme on the particle trajectories in the video sequence. First, a coarse foreground (CFG) region is extracted by performing reduced singular value decomposition on multiple matrices that are built from bundles of particle trajectories. Next, the BG motion of pixels in the CFG region is reconstructed by a fast inpainting method. After subtracting the BG motion, the fine foreground is segmented out by an adaptive thresholding method that is capable of solving multiple-moving-objects scenarios. Finally, the detected foreground is further refined by the mean-shift segmentation method. Extensive simulations and a comparison with the state-of-the-art methods verify the effectiveness of the proposed method.
---
paper_title: DeepAnomaly: Combining Background Subtraction and Deep Learning for Detecting Obstacles and Anomalies in an Agricultural Field

paper_content:

Convolutional neural network (CNN)-based systems are increasingly used in autonomous vehicles for detecting obstacles. CNN-based object detection and per-pixel classification (semantic segmentation) algorithms are trained for detecting and classifying a predefined set of object types. These algorithms have difficulties in detecting distant and heavily occluded objects and are, by definition, not capable of detecting unknown object types or unusual scenarios. The visual characteristics of an agriculture field is homogeneous, and obstacles, like people, animals and other obstacles, occur rarely and are of distinct appearance compared to the field. This paper introduces DeepAnomaly, an algorithm combining deep learning and anomaly detection to exploit the homogenous characteristics of a field to perform anomaly detection. We demonstrate DeepAnomaly as a fast state-of-the-art detector for obstacles that are distant, heavily occluded and unknown. DeepAnomaly is compared to state-of-the-art obstacle detectors including “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks” (RCNN). In a human detector test case, we demonstrate that DeepAnomaly detects humans at longer ranges (45–90 m) than RCNN. RCNN has a similar performance at a short range (0–30 m). However, DeepAnomaly has much fewer model parameters and (182 ms/25 ms =) a 7.28-times faster processing time per image. Unlike most CNN-based methods, the high accuracy, the low computation time and the low memory footprint make it suitable for a real-time system running on a embedded GPU (Graphics Processing Unit).
---
paper_title: Region-based Mixture of Gaussians modelling for foreground detection in dynamic scenes

paper_content:

One of the most widely used techniques in computer vision for foreground detection is to model each background pixel as a Mixture of Gaussians (MoG). While this is effective for a static camera with a fixed or a slowly varying background, it fails to handle any fast, dynamic movement in the background. In this paper, we propose a generalised framework, called region-based MoG (RMoG), that takes into consideration neighbouring pixels while generating the model of the observed scene. The model equations are derived from expectation maximisation theory for batch mode, and stochastic approximation is used for online mode updates. We evaluate our region-based approach against ten sequences containing dynamic backgrounds, and show that the region-based approach provides a performance improvement over the traditional single pixel MoG. For feature and region sizes that are equal, the effect of increasing the learning rate is to reduce both true and false positives. Comparison with four state-of-the art approaches shows that RMoG outperforms the others in reducing false positives whilst still maintaining reasonable foreground definition. Lastly, using the ChangeDetection (CDNet 2014) benchmark, we evaluated RMoG against numerous surveillance scenes and found it to be amongst the leading performers for dynamic background scenes, whilst providing comparable performance for other commonly occurring surveillance scenes. HighlightsA generalised framework for region based mixture modelling is proposed.Batch and online update equations are derived using expectation maximisation theory.Region-based Mixture of Gaussians (RMoG) algorithm for dynamic background subtractionExperiments show state-of-the-art results in subtracting dynamic backgrounds.
---
paper_title: Background Subtraction in Real Applications: Challenges, Current Models and Future Directions

paper_content:

Abstract Computer vision applications based on videos often require the detection of moving objects in their first step. Background subtraction is then applied in order to separate the background and the foreground. In literature, background subtraction is surely among the most investigated field in computer vision providing a big amount of publications. Most of them concern the application of mathematical and machine learning models to be more robust to the challenges met in videos. However, the ultimate goal is that the background subtraction methods developed in research could be employed in real applications like traffic surveillance. But looking at the literature, we can remark that there is often a gap between the current methods used in real applications and the current methods in fundamental research. In addition, the videos evaluated in large-scale datasets are not exhaustive in the way that they only covered a part of the complete spectrum of the challenges met in real applications. In this context, we attempt to provide the most exhaustive survey as possible on real applications that used background subtraction in order to identify the real challenges met in practice, the current used background models and to provide future directions. Thus, challenges are investigated in terms of camera (i.e CCD cameras, omnidirectional cameras, …), foreground objects and environments. In addition, we identify the background models that are effectively used in these applications in order to find potential usable recent background models in terms of robustness, time and memory requirements.
---
paper_title: Adaptive bootstrapping management by keypoint clustering for background initialization

paper_content:

A novel adaptive bootstrapping management method for background initialization and updating is proposed.It works with both static and mobile cameras.It works with a small number of frames to initialize the background model. The availability of a background model that describes the scene is a prerequisite for many computer vision applications. In several situations, the model cannot be easily generated when the background contains some foreground objects (i.e., bootstrapping problem). In this letter, an Adaptive Bootstrapping Management (ABM) method, based on keypoint clustering, is proposed to model the background on video sequences acquired by mobile and static cameras. First, keypoints are detected on each frame by the A-KAZE feature extractor, then Density-Based Spatial Clustering of Application with Noise (DBSCAN) is used to find keypoint clusters. These clusters represent the candidate regions of foreground elements inside the scene. The ABM method manages the scene changes generated by foreground elements, both in the background model initialization, managing the bootstrapping problem, and in the background model updating. Moreover, it achieves good results with both mobile and static cameras and it requires a small number of frames to initialize the background model.
---
paper_title: Automated Video Surveillance for Retail Store Statistics Generation

paper_content:

Video Surveillance Systems have gained immense popularity in the recent past because of the fact that it can be used in numerous real-world scenario applications. Monitoring the people flow pattern as well as counting them serves as valuable information in many surveillance related applications. In this paper we propose a system that is used for counting the number of people passing through the camera field of view. A single overhead camera is used to get a clear top-view which avoids occlusions. For background subtraction, running Gaussian approach has been used as a preprocessing step, to facilitate the further segmentation and tracking procedures. Connected component analysis is used to group the similar blobs together followed by intensity based correlation for blob matching followed by Kalman tracking. The percentage of blobs that crosses a reference line is recorded. Two counters are incremented depending on the direction of movement of the blobs and the algorithm is able to count the number of people moving up/down the scene.
---
paper_title: Low complexity techniques for robust real-time traffic incident detection

paper_content:

Traffic congestion is one of the leading reasons for the development of intelligent transportation systems(ITS). Traffic incidents are the second biggest cause of traffic congestions after infrastructural bottlenecks. Real-time traffic incident detection for timely clearing of roads is required to ensure smooth traffic flow. Apart from the real-time performance, scalable solutions which can monitor wide areas in a cost-effective manner are required. In this paper, robust, lean and real-time stationary foreground object detection technique to detect traffic incidents has been presented. We use block-based analysis in contrast to the conventional pixel-based analysis to lower the computational complexity of the proposed technique and achieve real-time performance. Experimental evaluations on widely used datasets demonstrate that the proposed method can achieve comparable accuracy to the existing state-of-the-art techniques. The real-time performance of the proposed system has also been demonstrated by implementing it on a low-cost embedded platform, Odroid XU-4, that still achieves a frame rate of 40 frames/second, thereby enabling real-time detection of traffic incidents.
---
paper_title: Motion characteristics estimation of animals in video surveillance

paper_content:

This paper proposes a system for the motion characteristics estimation of multiple objects with uncertain quantity. The system employs background subtraction method based on Gaussian Mixture Model to detect objects and tracks them through an improved algorithm combining Camshift with Kalman filtering. By analyzing the directions and trails of targets, the system can figure out their pixel distance and velocity. Experiment conducted realizes the proposed system, verifying its performance on motion characteristics estimation.
---
paper_title: Comparison of Background Subtraction Methods for a Multimedia Application

paper_content:

This article presents the Aqu@theque application and a comparison of background subtraction methods to improve the performance of this application. The Aqu@theque application consists in elaborating an information system dedicated to aquariums in an interactive learning area. In particular, this article presents our comparison of different background subtraction methods to detect fish in video sequences and the improvement for the Aqu@theque application.
---
paper_title: Real time hand detection in a complex background

paper_content:

Hand gesture recognition has gained the interest of many researchers in recent years, as it has become one of the most popular Human Computer Interfaces. The first step in most vision-based gesture recognition systems is the hand region detection and segmentation. This segmentation can be a particularly challenging task when it comes to complex backgrounds and varying illumination. In such environments, most hand detection techniques fail to obtain the exact region of the hand shape, especially in cases of dynamic gestures. Meeting these requirements becomes even more difficult, due to real-time operation demand. To overcome these problems, in this paper, we propose a new method for real-time hand detection in a complex background. We employ a combination of existing techniques, based on motion detection and introduce a novel skin color classifier to improve segmentation accuracy. Motion detection is based on image differencing and background subtraction. Skin color detection is accomplished via a color classification technique that employs online color training, so that the system can dynamically adapt to the variety of lighting conditions and the [email protected]?s skin color as well as possible. Morphological features of the detected hand in previous frames are employed to estimate the probability of a pixel belonging to the hand section in the current frame. Finally, the derived motion, color and morphological information are combined to detect the hand region. Experimental results show significant improvement in hand region detection, compared to existing methods with an average accuracy of 98.75%.
---
paper_title: Practical automatic background substitution for live video

paper_content:

In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new background. In this paper, we use a color line model to improve the Gaussian mixture model in the background cut method to obtain a binary foreground segmentation result that is less sensitive to brightness differences. Based on the high quality binary segmentation results, we can automatically create a reliable trimap for alpha matting to refine the segmentation boundary. To make the composition result more realistic, an automatic foreground color adjustment step is added to make the foreground look consistent with the new background. Compared to previous approaches, our method can produce higher quality binary segmentation results, and to the best of our knowledge, this is the first time such an automatic and integrated background substitution system has been proposed which can run in real time, which makes it practical for everyday applications.
---
paper_title: Background Subtraction in Real Applications: Challenges, Current Models and Future Directions

paper_content:

Abstract Computer vision applications based on videos often require the detection of moving objects in their first step. Background subtraction is then applied in order to separate the background and the foreground. In literature, background subtraction is surely among the most investigated field in computer vision providing a big amount of publications. Most of them concern the application of mathematical and machine learning models to be more robust to the challenges met in videos. However, the ultimate goal is that the background subtraction methods developed in research could be employed in real applications like traffic surveillance. But looking at the literature, we can remark that there is often a gap between the current methods used in real applications and the current methods in fundamental research. In addition, the videos evaluated in large-scale datasets are not exhaustive in the way that they only covered a part of the complete spectrum of the challenges met in real applications. In this context, we attempt to provide the most exhaustive survey as possible on real applications that used background subtraction in order to identify the real challenges met in practice, the current used background models and to provide future directions. Thus, challenges are investigated in terms of camera (i.e CCD cameras, omnidirectional cameras, …), foreground objects and environments. In addition, we identify the background models that are effectively used in these applications in order to find potential usable recent background models in terms of robustness, time and memory requirements.
---
paper_title: Wallflower: principles and practice of background maintenance

paper_content:

Background maintenance is a frequent element of video surveillance systems. We develop Wallflower, a three-component system for background maintenance: the pixel-level component performs Wiener filtering to make probabilistic predictions of the expected background; the region-level component fills in homogeneous regions of foreground objects; and the frame-level component detects sudden, global changes in the image and swaps in better approximations of the background. We compare our system with 8 other background subtraction algorithms. Wallflower is shown to outperform previous algorithms by handling a greater set of the difficult situations that can occur. Finally, we analyze the experimental results and propose normative principles for background maintenance.
---
paper_title: A deep convolutional neural network for video sequence background subtraction

paper_content:

We propose a novel approach based on deep learning for background subtraction from video sequences.A new algorithm to generate background model has been proposed.Input image patches and their corresponding background images are fed into CNN to do background subtraction.We utilized median filter to enhance the segmentation results.Experiments of Change detection results confirm the performance of the proposed approach. In this work, we present a novel background subtraction from video sequences algorithm that uses a deep Convolutional Neural Network (CNN) to perform the segmentation. With this approach, feature engineering and parameter tuning become unnecessary since the network parameters can be learned from data by training a single CNN that can handle various video scenes. Additionally, we propose a new approach to estimate background model from video sequences. For the training of the CNN, we employed randomly 5% video frames and their ground truth segmentations taken from the Change Detection challenge 2014 (CDnet 2014). We also utilized spatial-median filtering as the post-processing of the network outputs. Our method is evaluated with different data-sets, and it (so-called DeepBS) outperforms the existing algorithms with respect to the average ranking over different evaluation metrics announced in CDnet 2014. Furthermore, due to the network architecture, our CNN is capable of real time processing.
---
paper_title: Three-Pronged Compensation and Hysteresis Thresholding for Moving Object Detection in Real-Time Video Surveillance

paper_content:

Moving object extraction is the core of event detection in video surveillance. Although many related methods have been proposed to extract moving objects, even advanced applications still encounter cavity problems, which are false detection and deficiencies resulting from cavities inside the body or fragmented foreground objects. In this paper, an entirely new structure for extracting moving objects is proposed. This scheme is based on the concepts of hysteresis thresholding and motion compensation, which constitute spatial and temporal compensations, respectively. Experimental results show that the proposed scheme can outperform modern moving object detection methods in terms of precision, recall, F -measure, and other measurements.
---
paper_title: Deep learning driven blockwise moving object detection with binary scene modeling

paper_content:

As an important and challenging topic in computer vision, moving object detection is typically posed as a problem of scene analysis, which usually requires both robust feature description and effective statistical modeling. In general, the conventional detection methods make use of pre-defined hand-crafted features and sophisticated background models for scene analysis. Therefore, they usually have a low generalization capability of adapting to different scenes with diverse spatio-temporal motion information. In the face of high-definition video data, sophisticated statistical modeling often suffers from an expensive computation or memory cost because of its low efficiency in evaluation and parallelization. In order to address this issue, we propose a deep learning based block-wise scene analysis method equipped with a binary spatio-temporal scene model. Based on the stacked denoising autoencoder, the deep learning module of the proposed method aims to learn an effective deep image representation encoding the intrinsic scene information, which leads to the robustness of feature description. Furthermore, the proposed binary scene model captures the spatio-temporal scene distribution information in the Hamming space, which ensures the high efficiency of moving object detection. Experimental results on several datasets demonstrate the effectiveness and efficiency of the proposed method.
---
paper_title: PETS Metrics: On-Line Performance Evaluation Service

paper_content:

This paper presents the PETS Metrics On-line Evaluation Service for computational visual surveillance algorithms. The service allows researchers to submit their algorithm results for evaluation against a set of applicable metrics. The results of the evaluation processes are publicly displayed allowing researchers to instantly view how their algorithm performs against previously submitted algorithms. The approach has been validated using seven motion segmentation algorithms.
---
paper_title: Abnormal Event Detection on BMTT-PETS 2017 Surveillance Challenge

paper_content:

In this paper, we have proposed a method to detect abnormal events for human group activities. Our main contribution is to develop a strategy that learns with very few videos by isolating the action and by using supervised learning. First, we subtract the background of each frame by modeling each pixel as a mixture of Gaussians(MoG) to concatenate the higher order learning only on the foreground. Next, features are extracted from each frame using a convolutional neural network (CNN) that is trained to classify between normal and abnormal frames. These feature vectors are fed into long short term memory (LSTM) network to learn the long-term dependencies between frames. The LSTM is also trained to classify abnormal frames, while extracting the temporal features of the frames. Finally, we classify the frames as abnormal or normal depending on the output of a linear SVM, whose input are the features computed by the LSTM.
---
paper_title: Multi-human tracking using part-based appearance modelling and grouping-based tracklet association for visual surveillance applications

paper_content:

Although much progress has been made in multi-object tracking in recent decades due to its variety of applications including visual surveillance, traffic monitoring and medical image analysis, some difficult challenges such as the variation of object appearance and partial occlusion are still going on. In this work, we propose an effective multi-human tracking system called part-based appearance modelling and grouping-based tracklet association-based multi-human tracking (PAM-GTA-MHT). The proposed appearance model based on the upper body-centered multi-view human body part model can effectively resolve the drawback caused by inter-object occlusions and low camera positions. The grouping method embedded in global tracklet association can improve discriminability among targets with similar appearances when they are located sufficiently far away from each other. Thus, there is no need to compare all possible pairs of the detected targets in the tracklet association stage and thus it has the potential to enhance the tracking speed. We quantitatively evaluated the performance of our proposed approach on four challenging publicly available datasets and achieved a significant improvement compared to the state-of-the-art methods.
---
paper_title: Database architecture for autonomous transportation agents for on-scene networked incident management (ATON)

paper_content:

A collection of distributed databases forms an important architectural component of the ATON project for networked incidence management of highway traffic. The database sub-architecture supports the architectural integration of many thematic areas of the ATON, and provides many high level abstractions that semantically correspond to traffic incidents. These databases are queried for the detection of local or distributed traffic incidents by many distributed control and analysis algorithms. The abstract database sub-architecture leads to many databases of different types having different functions. The semantic event/activity database is one of them. The distributed multi-sensory sub-architecture detects atomic semantic events that occur in the environment. The event/activity database stores them and their temporal structures. An activity is a temporal composition of atomic events. A specialized query language is used to flexibly define and detect activities of interest. The query language embodies high level semantic pattern matching abstractions. Preliminary results of using the event/activity database are also presented.
---
paper_title: Detecting Moving Shadows: Algorithms and Evaluation

paper_content:

Moving shadows need careful consideration in the development of robust dynamic scene analysis systems. Moving shadow detection is critical for accurate object detection in video streams since shadow points are often misclassified as object points, causing errors in segmentation and tracking. Many algorithms have been proposed in the literature that deal with shadows. However, a comparative evaluation of the existing approaches is still lacking. In this paper, we present a comprehensive survey of moving shadow detection approaches. We organize contributions reported in the literature in four classes two of them are statistical and two are deterministic. We also present a comparative empirical evaluation of representative algorithms selected from these four classes. Novel quantitative (detection and discrimination rate) and qualitative metrics (scene and object independence, flexibility to shadow situations, and robustness to noise) are proposed to evaluate these classes of algorithms on a benchmark suite of indoor and outdoor video sequences. These video sequences and associated "ground-truth" data are made available at http://cvrr.ucsd.edu/aton/shadow to allow for others in the community to experiment with new algorithms and metrics.
---
paper_title: Moving shadow and object detection in traffic scenes

paper_content:

We present an algorithm for segmentation of traffic scenes that distinguishes moving objects from their moving cast shadows. A fading memory estimator calculates mean and variance of all three color components for each background pixel. Given the statistics for a background pixel, simple rules for calculating its statistics when covered by a shadow are used. Then, MAP classification decisions are made for each pixel. In addition to the color features, we examine the use of neighborhood information to produce smoother classification. We also propose the use of temporal information by modifying class a priori probabilities based on predictions from the previous frame.
---
paper_title: Detection and tracking in the IBM PeopleVision system

paper_content:

The detection and tracking of people lie at the heart of many current and near-future applications of computer vision. We describe a background subtraction system designed to detect moving objects in a wide variety of conditions, and a second system to detect objects moving in front of moving backgrounds. Detected foreground regions are tracked with a tracking system which can initiate real-time alarms and generate a smart surveillance index which can be searched to find interesting events in stored video
---
paper_title: Comparison of target detection algorithms using adaptive background models

paper_content:

This article compares the performance of target detectors based on adaptive background differencing on public benchmark data. Five state of the art methods are described. The performance is evaluated using state of the art measures with respect to ground truth. The original points are the comparison to hand labelled ground truth and the evaluation on a large database. The simpler methods LOTS and SGM are more appropriate to the particular task as MGM using a more complex background model.
---
paper_title: Automatic parameter regulation for a tracking system with an auto-critical function

paper_content:

In this article we propose an architecture of a tracking system that can judge its own performance by an auto-critical function. Performance drops can be detected which trigger an automatic parameter regulation module. This regulation module is an expert system that searches a parameter setting with better performance and returns it to the tracking system. With such an architecture, a robust tracking system can be implemented which automatically adapts its parameters in case of changes in the environmental conditions. This article opens a way to self-adaptive systems in detection and recognition.
---
paper_title: Extracting Motion Features for Visual Human Activity Representation

paper_content:

This paper presents a technique to characterize human actions in visual surveillance scenarios in order to describe, in a qualitative way, basic human movements in general imaging conditions. The representation proposed is based on focus of attention concepts, as part of an active tracking process to describe target movements. The introduced representation, named “focus of attention” representation, FOA, is based on motion information. A segmentation method is also presented to group the FOA in uniform temporal segments. The segmentation will allow providing a higher level description of human actions, by means of further classifying each segment in different types of basic movements.
---
paper_title: MOTION SEGMENTATION FOR ACTIVITY SURVEILLANCE ⋆

paper_content:

This paper describes an algorithm for segmenting and classifying human activities from video sequences of a shopping center. These activities comprise entering or exiting a shop, passing, or browsing in front of shop windows. The proposed approach recognizes these activities by using a priori knowledge of the layout of the shopping view. Human actions are represented by a bank of switch dynamical models, each tailored to describe a specific motion regime. Experimental tests illustrate the effectiveness of the proposed approach with synthetic and real data.
---
paper_title: Adaptive Retrodiction Particle PHD Filter for Multiple Human Tracking

paper_content:

The probability hypothesis density (PHD) filter is well known for addressing the problem of multiple human tracking for a variable number of targets, and the sequential Monte Carlo implementation of the PHD filter, known as the particle PHD filter, can give state estimates with nonlinear and non-Gaussian models. Recently, Mahler et al. have introduced a PHD smoother to gain more accurate estimates for both target states and number. However, as highlighted by Psiaki in the context of a backward-smoothing extended Kalman filter, with a nonlinear state evolution model the approximation error in the backward filtering requires careful consideration. Psiaki suggests that to minimize the aggregated least-squares error over a batch of data. We instead use the term retrodiction PHD filter to describe the backward filtering algorithm in recognition of the approximation error proposed in the original PHD smoother, and we propose an adaptive recursion step to improve the approximation accuracy. This step combines forward and backward processing through the measurement set and thereby mitigates the problems with the original PHD smoother when the target number changes significantly and the targets appear and disappear randomly. Simulation results show the improved performance of the proposed algorithm and its capability in handling a variable number of targets.
---
paper_title: A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications

paper_content:

Detection of moving objects in video streams is the first relevant step of information extraction in many computer vision applications. Aside from the intrinsic usefulness of being able to segment video streams into moving and background components, detecting moving objects provides a focus of attention for recognition, classification, and activity analysis, making these later steps more efficient. We propose an approach based on self organization through artificial neural networks, widely applied in human image processing systems and more generally in cognitive science. The proposed approach can handle scenes containing moving backgrounds, gradual illumination variations and camouflage, has no bootstrapping limitations, can include into the background model shadows cast by moving objects, and achieves robust detection for different types of videos taken with stationary cameras. We compare our method with other modeling techniques and report experimental results, both in terms of detection accuracy and in terms of processing speed, for color video sequences that represent typical situations critical for video surveillance systems.
---
paper_title: Moving Object Detection Using Tensor-Based Low-Rank and Saliently Fused-Sparse Decomposition

paper_content:

In this paper, we propose a new low-rank and sparse representation model for moving object detection. The model preserves the natural space-time structure of video sequences by representing them as three-way tensors. Then, it operates the low-rank background and sparse foreground decomposition in the tensor framework. On the one hand, we use the tensor nuclear norm to exploit the spatio-temporal redundancy of background based on the circulant algebra. On the other, we use the new designed saliently fused-sparse regularizer (SFS) to adaptively constrain the foreground with spatio-temporal smoothness. To refine the existing foreground smooth regularizers, the SFS incorporates the local spatio-temporal geometric structure information into the tensor total variation by using the 3D locally adaptive regression kernel (3D-LARK). What is more, the SFS further uses the 3D-LARK to compute the space-time motion saliency of foreground, which is combined with the $l_{1}$ norm and improves the robustness of foreground extraction. Finally, we solve the proposed model with globally optimal guarantee. Extensive experiments on challenging well-known data sets demonstrate that our method significantly outperforms the state-of-the-art approaches and works effectively on a wide range of complex scenarios.
---
paper_title: Foreground detection based on low-rank and block-sparse matrix decomposition

paper_content:

Foreground detection is the first step in video surveillance system to detect moving objects. Principal Components Analysis (PCA) shows a nice framework to separate moving objects from the background but without a mechanism of robust analysis, the moving objects may be absorbed into the background model. This drawback can be solved by recent researches on Robust Principal Component Analysis (RPCA). The background sequence is then modeled by a low rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. In this paper, we propose to use a RPCA method based on low-rank and block-sparse matrix decomposition to achieve foreground detection. This decomposition enforces the low-rankness of the background and the block-sparsity aspect of the foreground. Experimental results on different datasets show the pertinence of the proposed approach.
---
paper_title: Real-time nonparametric background subtraction with tracking-based foreground update

paper_content:

Abstract A nonparametric real-time and high-quality moving object detection strategy in a GPU is proposed. To improve the quality of the results in sequences where the moving objects and the background have similar appearance, not only the background but also the foreground is modelled. Both models are constructed from spatio-temporal reference data to reduce false detections due to small displacements of the background, and to take into consideration the natural displacements of the foreground. To avoid using kernels with too large spatial widths, the spatial positions of the foreground reference data are updated at each new frame using a particle filter that is able to deal with an unknown and variable amount of regions. Additionally, an automatic selection of regions of interest is carried out, which allows reducing drastically the computational cost of both foreground and background models. The proposed strategy has been validated using three databases containing many challenges for motion detection and the results have been compared to those of other state-of-the-art approaches.
---
paper_title: Real time moving vehicle detection and reconstruction for improving classification

paper_content:

Vehicle images captured by traffic and surveillance video cameras in various conditions usually exhibit several unexpected variations that worsen vehicle classification. These factors include occlusions, motion blur, and changes in perspective views. Complete and normalized views of vehicle images, if being able to reconstructed from the unsatisfactory data, will facilitate more accurate data labeling, feature extraction and multi-class vehicle classification. We propose a multimodal temporal panorama (MTP) approach to accurately extracting and reconstructing moving vehicles in real-time using a remote multimodal (audio/video) monitoring system. The MTP representation consists of: 1) a panoramic view image (PVI) for detecting vehicles using the concept of 1D vertical detection line; 2) an epipolar plane image (EPI), generated from 1D epipolar lines along the vehicles' moving paths, to characterize their speeds and directions; and 3) corresponding audio signals collected at the vehicle detection point to reduce false target detection in the PVI. Using the MTP approach, reconstructed vehicles all have the same side views, with less or no occlusions and motion blur. Using SVM classifiers for multiclass problems indicates that the classification accuracy using reconstruction results improves about 10% over that using corresponding vehicle images from original video for a dataset of about 140 vehicles. Our ultimate goal is to use the audio-visual vehicle data for multimodal vehicle classification and anomaly detection.
---
paper_title: A Two-Stage Template Approach to Person Detection in Thermal Imagery

paper_content:

We present a two-stage template-based method to detect people in widely varying thermal imagery. The approach initially performs a fast screening procedure using a generalized template to locate potential person locations. Next an AdaBoosted ensemble classifier using automatically tuned filters is employed to test the hypothesized person locations. We demonstrate and evaluate the approach using a challenging dataset of thermal imagery
---
paper_title: A Thermal Infrared Video Benchmark for Visual Analysis

paper_content:

We hereby publish a new thermal infrared video benchmark, called TIV, for various visual analysis tasks, which include single object tracking in clutter, multi-object tracking in single or multiple views, analyzing motion patterns of large groups, and censusing wild animals in flight. Our data describe real world scenarios, such as bats emerging from their caves in large numbers, a crowded street view during a marathon competition, and students walking through an atrium during class break. We also introduce baseline methods and evaluation protocols for these tasks. Our TIV benchmark enriches and diversifies video data sets available to the research community with thermal infrared footage, which poses new and challenging video analysis problems. We hope the TIV benchmark will help the community to better understand these interesting problems, generate new ideas, and value it as a testbed to compare solutions.
---
paper_title: Background-subtraction using contour-based fusion of thermal and visible imagery

paper_content:

We present a new background-subtraction technique fusing contours from thermal and visible imagery for persistent object detection in urban settings. Statistical background-subtraction in the thermal domain is used to identify the initial regions-of-interest. Color and intensity information are used within these areas to obtain the corresponding regions-of-interest in the visible domain. Within each region, input and background gradient information are combined to form a Contour Saliency Map. The binary contour fragments, obtained from corresponding Contour Saliency Maps, are then fused into a single image. An A^* path-constrained search along watershed boundaries of the regions-of-interest is used to complete and close any broken segments in the fused contour image. Lastly, the contour image is flood-filled to produce silhouettes. Results of our approach are evaluated quantitatively and compared with other low- and high-level fusion techniques using manually segmented data.
---
paper_title: Thermal-Visible Registration of Human Silhouettes: a Similarity Measure Performance Evaluation

paper_content:

Abstract When dealing with the registration of information from different image sources, the de facto similarity measure used is Mutual Information (MI). Although MI gives good performance in many image registration applications, recent works in thermal–visible registration have shown that other similarity measures can give results that are as accurate, if not more than MI. Furthermore, some of these measures also have the advantage of being calculated independently from each image to register, which allows them to be integrated more easily in energy minimization frameworks. In this article, we investigate the accuracy of similarity measures for thermal–visible image registration of human silhouettes, including MI, Sum of Squared Differences (SSD), Normalized Cross-Correlation (NCC), Histograms of Oriented Gradients (HOG), Local Self-Similarity (LSS), Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Census, Fast Retina Keypoint (FREAK), and Binary Robust Independent Elementary Feature (BRIEF). We tested the various similarity measures in dense stereo matching tasks over 25,000 windows to have statistically significant results. To do so, we created a new dataset in which one to five humans are walking in a scene in various depth planes. Results show that even if MI is a very strong performer, particularly for large regions of interest (ROI), LSS gives better accuracies when ROI are small or segmented into small fragments because of its ability to capture shape. The other tested similarity measures did not give consistently accurate results.
---
paper_title: Adaptive contour-based statistical background subtraction method for moving target detection in infrared video sequences

paper_content:

A robust contour-based statistical background subtraction method for detection of non-uniform thermal targets in infrared imagery is presented. The foremost step of the method comprises of generation of background frame using statistical information of an initial set of frames not containing any targets. The generated background frame is made adaptive by continuously updating the background using the motion information of the scene. The background subtraction method followed by a clutter rejection stage ensure the detection of foreground objects. The next step comprises of detection of contours and distinguishing the target boundaries from the noisy background. This is achieved by using the Canny edge detector that extracts the contours followed by a k-means clustering approach to differentiate the object contour from the background contours. The post processing step comprises of morphological edge linking approach to close any broken contours and finally flood fill is performed to generate the silhouettes of moving targets. This method is validated on infrared video data consisting of a variety of moving targets. Experimental results demonstrate a high detection rate with minimal false alarms establishing the robustness of the proposed method.
---
paper_title: Moving target detection in thermal infrared imagery using spatiotemporal information

paper_content:

An efficient target detection algorithm for detecting moving targets in infrared imagery using spatiotemporal information is presented. The output of the spatial processing serves as input to the temporal stage in a layered manner. The spatial information is obtained using joint space–spatial-frequency distribution and Renyi entropy. Temporal information is incorporated using background subtraction. By utilizing both spatial and temporal information, it is observed that the proposed method can achieve both high detection and a low false-alarm rate. The method is validated with experimentally generated data consisting of a variety of moving targets. Experimental results demonstrate a high value of F-measure for the proposed algorithm.
---
paper_title: Bayesian modeling of dynamic scenes for object detection

paper_content:

Accurate detection of moving objects is an important precursor to stable tracking or recognition. In this paper, we present an object detection scheme that has three innovations over existing approaches. First, the model of the intensities of image pixels as independent random variables is challenged and it is asserted that useful correlation exists in intensities of spatially proximal pixels. This correlation is exploited to sustain high levels of detection accuracy in the presence of dynamic backgrounds. By using a nonparametric density estimation method over a joint domain-range representation of image pixels, multimodal spatial uncertainties and complex dependencies between the domain (location) and range (color) are directly modeled. We propose a model of the background as a single probability density. Second, temporal persistence is proposed as a detection criterion. Unlike previous approaches to object detection which detect objects by building adaptive models of the background, the foregrounds modeled to augment the detection of objects (without explicit tracking) since objects detected in the preceding frame contain substantial evidence for detection in the current frame. Finally, the background and foreground models are used competitively in a MAP-MRF decision framework, stressing spatial context as a condition of detecting interesting objects and the posterior function is maximized efficiently by finding the minimum cut of a capacitated graph. Experimental validation of the proposed method is performed and presented on a diverse set of dynamic scenes.
---
paper_title: Fusion-based foreground enhancement for background subtraction using multivariate multi-model Gaussian distribution

paper_content:

Abstract These days, detection of visual attention regions (VAR) such as moving objects have become an essential pre-processing stage in many computer vision applications. In this paper, we focus on the vital issue of separating moving objects a.k.a. Foreground (FG) in a scene, which has a near-static background (BG). We address the difficulty in setting an adaptive threshold in the multi-model Gaussian-based BG-FG separation through a novel FG enhancement strategy by assimilating color and illumination measures. We formulate the problem mathematically by using a histogram of a fused feature of color and illumination measures. The proposed method improves the BG-FG separation by introducing the following items: (i) A new distance measure to check if a pixel matches a Gaussian distribution. (ii) A new strategy to enhance the results of traditional background subtraction (BGS) with a fusion of color and illumination measures. (iii) A methodology to find appropriate threshold adaptively that separates BG and FG. (iv) A foreground validation process through probability estimation of multivariate Gaussian model distribution (MVGMD). We test the proposed algorithm on five different benchmark video sequences. The experimental results reveal that the proposed approach works well in challenging conditions, at the same time, it performs competitively against state-of-the-art Gaussian-based algorithms and few other non-Gaussian-based methods as well.
---
paper_title: ETISEO, performance evaluation for video surveillance systems

paper_content:

This paper presents the results of ETISEO, a performance evaluation project for video surveillance systems. Many other projects have already evaluated the performance of video surveillance systems, but more on an end-user point of view. ETISEO aims at studying the dependency between algorithms and the video characteristics. Firstly we describe ETISEO methodology which consists in addressing each video processing problem separately. Secondly, we present the main evaluation metrics of ETISEO as well as their benefits, limitations and conditions of use. Finally, we discuss about the contributions of ETISEO to the evaluation community.
---
paper_title: Tracking people under heavy occlusions by layered data association

paper_content:

One of the main difficulties in video tracking of people arises in scenarios where targets are repeatedly and extensively occluded by other moving objects. These types of occlusions significantly affect the measurements of the person's position, motion, shape and appearance, posing major challenges to correct tracking and data association. In this paper, we present a method for tracking people in videos based on a simplified part-based model only loosely associated with body parts. Data association is provided by a layered data association approach which performs association at feature, part and global levels in a hierarchical fashion. Occlusions are detected and managed at the part level, with corresponding model update strategies. In addition, the tracker does not make any assumption on the target's motion direction, thus allowing tracking to withstand abrupt sideways movements and changes of directions that frequently occur in busy scenes. Experimental results against popular trackers such as mean shift, particle filters and the recent k-shortest paths (KSP) tracker based on a variety of performance indicators and datasets including ETISEO, AVSS 2007 and PETS 2009 show the effectiveness of the proposed tracker.
---
paper_title: Real-time background subtraction based on GPGPU for high-resolution video surveillance

paper_content:

Demand for intelligent surveillance has been increasing, to automatically detect and prevent dangerous situations with surveillance cameras. Image analysis, the most essential element in intelligent surveillance system, has continuously developed and contributed to the improvement. To analyze surveillance videos, foreground segmentation is vital which require background modeling. This paper proposes background modeling method which is robust to illumination variation and shadow area. Also, the proposed method is applicable to high-resolution videos in real time with modification for GPU implementation. We validate our method on different types of dataset including our new benchmark dataset to analyze the result quantitatively and qualitatively. The execution time of proposed method is 228.2 FPS for High Definition videos with NVIDIA GTX660.
---
paper_title: Extended Gaussian Mixture Model Enhanced by Hole Filling Algorithm (GMMHF) Utilize GPU Acceleration

paper_content:

Video surveillance system as one of the most important areas in computer vision needs background subtraction in their initial process. Recently there is no method that robust enough to handle all of the prevalent issues in this field, such as water rippling, waving trees, illuminations change, and slow motion object. In this study, we propose an enhanced Gaussian Mixture Model (GMM) with Hole Filling (HF) algorithm through the post processing operation. Graphic Processing Unit (GPU) acceleration is also applied to reduce the computational time. The experimental result shows that the proposed method can achieve 8x of speedup and give a better performance compared to the original extended Gaussian Mixture Model. It can remove most of the noises from various complex scenes of some benchmark publicly available data.
---
paper_title: Background Subtraction Based on Time-Series Clustering and Statistical Modeling

paper_content:

This paper proposes a robust method to detect and extract silhouettes of foreground objects from a video sequence of a static camera based on the improved background subtraction technique. The proposed method analyses statistically the pixel history as time series observations. The proposed method presents a robust technique to detect motions based on kernel density estimation. Two consecutive stages of the k-means clustering algorithm are utilized to identify the most reliable background regions and decrease the detection of false positives. Pixel and object based updating mechanism for the background model is presented to cope with challenges like gradual and sudden illumination changes, ghost appearance, non-stationary background objects, and moving objects that remain stable for more than the half of the training period. Experimental results show the efficiency and the robustness of the proposed method to detect and extract the silhouettes of moving objects in outdoor and indoor environments compared with conventional methods.
---
paper_title: Reliable background suppression for complex scenes

paper_content:

This paper describes a system for motion detection based on background suppression,specifically conceived for working in complex scenes with vacillating background,camouflage, illumination changing, etc.. The system contains proper techniques for background bootstrapping, shadow removal, ghost suppression and selective updating of the background model. The results on the challenging videos provided in VSSN '06 Open Source Algorithm Competition dataset demonstrate that the proposed system outperforms the widely-used mixture-of-Gaussians approach.
---
paper_title: A new method for violence detection in surveillance scenes

paper_content:

Violence detection is a hot topic for surveillance systems. However, it has not been studied as much as for action recognition. Existing vision-based methods mainly concentrate on violence detection and make little effort to determine the location of violence. In this paper, we propose a fast and robust framework for detecting and localizing violence in surveillance scenes. For this purpose, a Gaussian Model of Optical Flow (GMOF) is proposed to extract candidate violence regions, which are adaptively modeled as a deviation from the normal behavior of crowd observed in the scene. Violence detection is then performed on each video volume constructed by densely sampling the candidate violence regions. To distinguish violent events from nonviolent events, we also propose a novel descriptor, named as Orientation Histogram of Optical Flow (OHOF), which are fed into a linear SVM for classification. Experimental results on several benchmark datasets have demonstrated the superiority of our proposed method over the state-of-the-arts in terms of both detection accuracy and processing speed, even in crowded scenes.
---
paper_title: MoWLD: a robust motion image descriptor for violence detection

paper_content:

Automatic violence detection from video is a hot topic for many video surveillance applications. However, there has been little success in designing an algorithm that can detect violence in surveillance videos with high performance. Existing methods typically apply the Bag-of-Words (BoW) model on local spatiotemporal descriptors. However, traditional spatiotemporal features are not discriminative enough, and also the BoW model roughly assigns each feature vector to only one visual word and therefore ignores the spatial relationships among the features. To tackle these problems, in this paper we propose a novel Motion Weber Local Descriptor (MoWLD) in the spirit of the well-known WLD and make it a powerful and robust descriptor for motion images. We extend the WLD spatial descriptions by adding a temporal component to the appearance descriptor, which implicitly captures local motion information as well as low-level image appear information. To eliminate redundant and irrelevant features, the non-parametric Kernel Density Estimation (KDE) is employed on the MoWLD descriptor. In order to obtain more discriminative features, we adopt the sparse coding and max pooling scheme to further process the selected MoWLDs. Experimental results on three benchmark datasets have demonstrated the superiority of the proposed approach over the state-of-the-arts.
---
paper_title: A ground truth for motion-based video-object segmentation

paper_content:

This paper describes the design procedure followed to generate a ground truth for the evaluation of motion-based algorithms for video-object segmentation. A thorough review and classification of the critical factors that affect the behavior of segmentation algorithms results in a set of video scripts which have then been filmed. Foreground objects have been recorded in a chroma studio, in order to automatically obtain pixel-level high quality segmentation masks for each generated sequence. The resulting corpus (segmentation ground-.truth plus filmed sequences mounted over different backgrounds) is available for research purposes under a license agreement.
---
paper_title: Spatiotemporal Saliency in Dynamic Scenes

paper_content:

A spatiotemporal saliency algorithm based on a center-surround framework is proposed. The algorithm is inspired by biological mechanisms of motion-based perceptual grouping and extends a discriminant formulation of center-surround saliency previously proposed for static imagery. Under this formulation, the saliency of a location is equated to the power of a predefined set of features to discriminate between the visual stimuli in a center and a surround window, centered at that location. The features are spatiotemporal video patches and are modeled as dynamic textures, to achieve a principled joint characterization of the spatial and temporal components of saliency. The combination of discriminant center-surround saliency with the modeling power of dynamic textures yields a robust, versatile, and fully unsupervised spatiotemporal saliency algorithm, applicable to scenes with highly dynamic backgrounds and moving cameras. The related problem of background subtraction is treated as the complement of saliency detection, by classifying nonsalient (with respect to appearance and motion dynamics) points in the visual field as background. The algorithm is tested for background subtraction on challenging sequences, and shown to substantially outperform various state-of-the-art techniques. Quantitatively, its average error rate is almost half that of the closest competitor.
---
paper_title: Patch-based local histograms and contour estimation for static foreground classification

paper_content:

This paper presents an approach to classify static foreground blobs in surveillance scenarios. Possible application is the detection of abandoned and removed objects. In order to classify the blobs, we developed two novel features based on the assumption that the neighborhood of a removed object is fairly continuous. In other words, there is a continuity, in the input frame, ranging from inside the corresponding blob contour to its surrounding region. Conversely, it is usual to find a discontinuity, i.e., edges, surrounding an abandoned object. We combined the two features to provide a reliable classification. In the first feature, we use several local histograms as a measure of similarity instead of previous attempts that used a single one. In the second, we developed an innovative method to quantify the ratio of the blob contour that corresponds to actual edges in the input image. A representative set of experiments shows that the proposed approach can outperform other equivalent techniques published recently.
---
paper_title: Background subtraction in highly dynamic scenes

paper_content:

A new algorithm is proposed for background subtraction in highly dynamic scenes. Background subtraction is equated to the dual problem of saliency detection: background points are those considered not salient by suitable comparison of object and background appearance and dynamics. Drawing inspiration from biological vision, saliency is defined locally, using center-surround computations that measure local feature contrast. A discriminant formulation is adopted, where the saliency of a location is the discriminant power of a set of features with respect to the binary classification problem which opposes center to surround. To account for both motion and appearance, and achieve robustness to highly dynamic backgrounds, these features are spatiotemporal patches, which are modeled as dynamic textures. The resulting background subtraction algorithm is fully unsupervised, requires no training stage to learn background parameters, and depends only on the relative disparity of motion between the center and surround regions. This makes it insensitive to camera motion. The algorithm is tested on challenging video sequences, and shown to outperform various state-of-the-art techniques for background subtraction.
---
paper_title: Low complexity techniques for robust real-time traffic incident detection

paper_content:

Traffic congestion is one of the leading reasons for the development of intelligent transportation systems(ITS). Traffic incidents are the second biggest cause of traffic congestions after infrastructural bottlenecks. Real-time traffic incident detection for timely clearing of roads is required to ensure smooth traffic flow. Apart from the real-time performance, scalable solutions which can monitor wide areas in a cost-effective manner are required. In this paper, robust, lean and real-time stationary foreground object detection technique to detect traffic incidents has been presented. We use block-based analysis in contrast to the conventional pixel-based analysis to lower the computational complexity of the proposed technique and achieve real-time performance. Experimental evaluations on widely used datasets demonstrate that the proposed method can achieve comparable accuracy to the existing state-of-the-art techniques. The real-time performance of the proposed system has also been demonstrated by implementing it on a low-cost embedded platform, Odroid XU-4, that still achieves a frame rate of 40 frames/second, thereby enabling real-time detection of traffic incidents.
---
paper_title: Deep Feature Learning with Relative Distance Comparison for Person Re-identification

paper_content:

Identifying the same individual across different scenes is an important yet difficult task in intelligent video surveillance. Its main difficulty lies in how to preserve similarity of the same person against large appearance and structure variation while discriminating different individuals. In this paper, we present a scalable distance driven feature learning framework based on the deep neural network for person re-identification, and demonstrate its effectiveness to handle the existing challenges. Specifically, given the training images with the class labels (person IDs), we first produce a large number of triplet units, each of which contains three images, i.e. one person with a matched reference and a mismatched reference. Treating the units as the input, we build the convolutional neural network to generate the layered representations, and follow with the L 2 distance metric. By means of parameter optimization, our framework tends to maximize the relative distance between the matched pair and the mismatched pair for each triplet unit. Moreover, a nontrivial issue arising with the framework is that the triplet organization cubically enlarges the number of training triplets, as one image can be involved into several triplet units. To overcome this problem, we develop an effective triplet generation scheme and an optimized gradient descent algorithm, making the computational load mainly depend on the number of original images instead of the number of triplets. On several challenging databases, our approach achieves very promising results and outperforms other state-of-the-art approaches. HighlightsWe present a novel feature learning framework for person re-identification.Our framework is based on the maximum relative distance comparison.The learning algorithm is scalable to process large amount of data.We demonstrate superior performances over other state-of-the-arts.
---
paper_title: Person Re-Identification by Iterative Re-Weighted Sparse Ranking

paper_content:

In this paper we introduce a method for person re-identification based on discriminative, sparse basis expansions of targets in terms of a labeled gallery of known individuals. We propose an iterative extension to sparse discriminative classifiers capable of ranking many candidate targets. The approach makes use of soft- and hard- re-weighting to redistribute energy among the most relevant contributing elements and to ensure that the best candidates are ranked at each iteration. Our approach also leverages a novel visual descriptor which we show to be discriminative while remaining robust to pose and illumination variations. An extensive comparative evaluation is given demonstrating that our approach achieves state-of-the-art performance on single- and multi-shot person re-identification scenarios on the VIPeR, i-LIDS, ETHZ, and CAVIAR4REID datasets. The combination of our descriptor and iterative sparse basis expansion improves state-of-the-art rank-1 performance by six percentage points on VIPeR and by 20 on CAVIAR4REID compared to other methods with a single gallery image per person. With multiple gallery and probe images per person our approach improves by 17 percentage points the state-of-the-art on i-LIDS and by 72 on CAVIAR4REID at rank-1. The approach is also quite efficient, capable of single-shot person re-identification over galleries containing hundreds of individuals at about 30 re-identifications per second.
---
paper_title: Cumulative Dual Foreground Differences for Illegally Parked Vehicles Detection

paper_content:

Illegally parked vehicles on the urban road may create a traffic flow problem as well as a potential traffic accident, such as crashing between parked and other vehicles. Thus, the intelligent traffic monitoring system should be able to prevent this situation by integrating an illegally parked vehicle detection module. However, implementing such a module becomes more challenging due to road environments, such as weather conditions, occlusion, and illumination changing. Hence, this work addresses a method to implement an illegally parked vehicle detection based on the cumulative dual foreground differences from the short- and long-term background models, temporal analysis, vehicle detector, and tracking. The extensive experiments were conducted using both iLIDS and our proposed datasets to evaluate the effectiveness of the proposed method by comparing with other methods. The results showed that the method is effective in detecting illegally parked vehicles and can be considered as part of the intelligent traffic monitoring system.
---
paper_title: Feature-Based Image Patch Classification for Moving Shadow Detection

paper_content:

The presence of shadows in images significantly affects the performance of many computer vision tasks and visual processing applications, such as object tracking, object classification, and behavior recognition. Most methods have been designed to detect shadows in specific situations, but they often fail to distinguish shadow points from the foreground object in many problematic situations, such as chromatic shadows, non-textured and dark surfaces, and foreground–background camouflage. In this paper, we propose a new feature-based image patch approximation and multi-independent sparse representation technique to tackle these environmental problems. In this method, two illumination-invariant features—binary patterns of local color constancy and light-based gradient matching—are introduced, along with the intensity-reduction histogram. These features are extracted from image patches and are used to construct two over-complete dictionaries for objects and shadows, respectively. Given a new image patch, its best approximation for a number of iterations is found from each dictionary. For each iteration, an independent class assignment is performed by finding its distances from the reference dictionaries. The patch is then assigned to a class based on its probability of occurrence. The proposed framework is evaluated on common shadow detection data sets, and it shows improved performance in terms of the shadow detection rate and discrimination rate compared with the state-of-the-art methods.
---
paper_title: Gaussian Mixture Model With Advanced Distance Measure Based on Support Weights and Histogram of Gradients for Background Suppression

paper_content:

The proposed work is targeted toward improving the Gaussian mixture model (GMM) for the background suppression-based moving object detection. The GMM has been widely used for moving object detection due to its high applicability. However, the GMM cannot properly model noisy or nonstationary backgrounds and fails to discriminate between the foreground and background modes. The extensions to GMM provide increased accuracy in expense of complex implementation and reduced applicability. In response, this work proposes two simple improvements: 1) a novel distance measure based on local support weights and histogram of gradients to provide distinct cluster values; and 2) use of background layer concept to properly segment the foreground. The method also uses variable number of clusters for generalization. The main advantages of the method are implicit use of pixel relationships through distance measure with least modification to the conventional GMM and effective background noise removal through the use of background layer concept with no postprocessing involved. The extensive experimentations on various types of video sequences are performed to validate the improvement in accuracy compared to the GMM and a number of state-of-the-art methods.
---
paper_title: Bayesian Foreground and Shadow Detection in Uncertain Frame Rate Surveillance Videos

paper_content:

In in this paper, we propose a new model regarding foreground and shadow detection in video sequences. The model works without detailed a priori object-shape information, and it is also appropriate for low and unstable frame rate video sources. Contribution is presented in three key issues: 1) we propose a novel adaptive shadow model, and show the improvements versus previous approaches in scenes with difficult lighting and coloring effects; 2) we give a novel description for the foreground based on spatial statistics of the neighboring pixel values, which enhances the detection of background or shadow-colored object parts; 3) we show how microstructure analysis can be used in the proposed framework as additional feature components improving the results. Finally, a Markov random field model is used to enhance the accuracy of the separation. We validate our method on outdoor and indoor sequences including real surveillance videos and well-known benchmark test sets.
---
paper_title: Study on color space selection for detecting cast shadows in video surveillance

paper_content:

In this article, the authors address the color modeling problem of cast shadows in video sequences. It is illustrated that the performance of shadow detection can be improved significantly through appropriate color space selection, if for practical purposes, the number of free parameters of the method should be kept low. Hence, the authors compare several well known color spaces with a six-parameter shadow model embedded into a globally optimal MRF framework. Experimental results are shown regarding the following questions: (1) What is the gain of using color images instead of grayscale ones? (2) What is the gain of using uncorrelated spaces instead of the standard RGB? (3) Chrominance (illumination invariant), luminance, or mixed spaces are more effective? (4) In which scenes are the differences significant? The authors qualified the metrics both in color based clustering of the individual pixels and in the case of Bayesian foreground-background-shadow segmentation. Experimental results on real-life videos show that CIE L*u*v* color space is the most efficient. © 2007 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 17, 190–201, 2007
---
paper_title: A Background Modeling and Foreground Detection Algorithm Using Scaling Coefficients Defined With a Color Model Called Lightness-Red-Green-Blue

paper_content:

This paper presents an algorithm for background modeling and foreground detection that uses scaling coefficients, which are defined with a new color model called lightness-red-green-blue (LRGB). They are employed to compare two images by finding pixels with scaled lightness. Three backgrounds are used: 1) verified background with pixels that are considered as background; 2) testing background with pixels that are tested several times to check if they belong to the background; and 3) final background that is a combination of the testing and verified background (the testing background is used in places, where the verified background is not defined). If a testing background pixel matches pixels from previous frames (the match is tested using scaling coefficients), it is copied to the verified background, otherwise the pixel is set as the weighted average of the corresponding pixels of the last input images. After the background is computed, foreground objects are detected by using the scaling coefficients and additional criteria. The algorithm was evaluated using the SABS data set, Wallflower data set and a subset of the CDnet 2014 data set. The average F measure and sensitivity with the SABS Data set were 0.7109 and 0.8725, respectively. In the Wallflower data set, the total number of errors was 5280 and the total F-measure was 0.9089. In the CDnet 2014 data set, the F-measure for the baseline test case was 0.8887 and for the shadow test case was 0.8300.
---
paper_title: Evaluation of background subtraction techniques for video surveillance

paper_content:

Background subtraction is one of the key techniques for automatic video analysis, especially in the domain of video surveillance. Although its importance, evaluations of recent background subtraction methods with respect to the challenges of video surveillance suffer from various shortcomings. To address this issue, we first identify the main challenges of background subtraction in the field of video surveillance. We then compare the performance of nine background subtraction methods with post-processing according to their ability to meet those challenges. Therefore, we introduce a new evaluation data set with accurate ground truth annotations and shadow masks. This enables us to provide precise in-depth evaluation of the strengths and drawbacks of background subtraction methods.
---
paper_title: Real-time nonparametric background subtraction with tracking-based foreground update

paper_content:

Abstract A nonparametric real-time and high-quality moving object detection strategy in a GPU is proposed. To improve the quality of the results in sequences where the moving objects and the background have similar appearance, not only the background but also the foreground is modelled. Both models are constructed from spatio-temporal reference data to reduce false detections due to small displacements of the background, and to take into consideration the natural displacements of the foreground. To avoid using kernels with too large spatial widths, the spatial positions of the foreground reference data are updated at each new frame using a particle filter that is able to deal with an unknown and variable amount of regions. Additionally, an automatic selection of regions of interest is carried out, which allows reducing drastically the computational cost of both foreground and background models. The proposed strategy has been validated using three databases containing many challenges for motion detection and the results have been compared to those of other state-of-the-art approaches.
---
paper_title: Spatiotemporal GMM for Background Subtraction with Superpixel Hierarchy

paper_content:

We propose a background subtraction algorithm using hierarchical superpixel segmentation, spanning trees and optical flow. First, we generate superpixel segmentation trees using a number of Gaussian Mixture Models (GMMs) by treating each GMM as one vertex to construct spanning trees. Next, we use the $M$ -smoother to enhance the spatial consistency on the spanning trees and estimate optical flow to extend the $M$ -smoother to the temporal domain. Experimental results on synthetic and real-world benchmark datasets show that the proposed algorithm performs favorably for background subtraction in videos against the state-of-the-art methods in spite of frequent and sudden changes of pixel values.
---
paper_title: Moving Person Detection Using Blob Descriptors and Support Vector Machine

paper_content:

In this paper a method dedicated to people detection in videos is presented. We have constructed a system which extracts moving objects using background subtraction, then computes HOG descriptors for each object and classifies the objects into “person” or “non-person” using a Support Vector Machine. We have also analyzed possible improvements for background model updating during background subtraction and the opportunity of using a simplified, faster than HOG, descriptor for moving objects. We present and comment our experimental results, pointing out advantages and disadvantages.
---
paper_title: On the Role and the Importance of Features for Background Modeling and Foreground Detection

paper_content:

Abstract Background modeling has emerged as a popular foreground detection technique for various applications in video surveillance. Background modeling methods have become increasing efficient in robustly modeling the background and hence detecting moving objects in any visual scene. Although several background subtraction and foreground detection have been proposed recently, no traditional algorithm today still seem to be able to simultaneously address all the key challenges of illumination variation, dynamic camera motion, cluttered background and occlusion. This limitation can be attributed to the lack of systematic investigation concerning the role and importance of features within background modeling and foreground detection. With the availability of a rather large set of invariant features, the challenge is in determining the best combination of features that would improve accuracy and robustness in detection. The purpose of this study is to initiate a rigorous and comprehensive survey of features used within background modeling and foreground detection. Further, this paper presents a systematic experimental and statistical analysis of techniques that provide valuable insight on the trends in background modeling and use it to draw meaningful recommendations for practitioners. In this paper, a preliminary review of the key characteristics of features based on the types and sizes is provided in addition to investigating their intrinsic spectral, spatial and temporal properties. Furthermore, improvements using statistical and fuzzy tools are examined and techniques based on multiple features are benchmarked against reliability and selection criterion. Finally, a description of the different resources available such as datasets and codes is provided.
---
paper_title: Foreground detection in camouflaged scenes

paper_content:

Foreground detection has been widely studied for decades due to its importance in many practical applications. Most of the existing methods assume foreground and background show visually distinct characteristics and thus the foreground can be detected once a good background model is obtained. However, there are many situations where this is not the case. Of particular interest in video surveillance is the camouflage case. For example, an active attacker camouflages by intentionally wearing clothes that are visually similar to the background. In such cases, even given a decent background model, it is not trivial to detect foreground objects. This paper proposes a texture guided weighted voting (TGWV) method which can efficiently detect foreground objects in camouflaged scenes. The proposed method employs the stationary wavelet transform to decompose the image into frequency bands. We show that the small and hardly noticeable differences between foreground and background in the image domain can be effectively captured in certain wavelet frequency bands. To make the final foreground decision, a weighted voting scheme is developed based on intensity and texture of all the wavelet bands with weights carefully designed. Experimental results demonstrate that the proposed method achieves superior performance compared to the current state-of-the-art results.
---
paper_title: A Fusion Framework for Camouflaged Moving Foreground Detection in the Wavelet Domain

paper_content:

Detecting camouflaged moving foreground objects has been known to be difficult due to the similarity between the foreground objects and the background. Conventional methods cannot distinguish the foreground from background due to the small differences between them and thus suffer from under-detection of the camouflaged foreground objects. In this paper, we present a fusion framework to address this problem in the wavelet domain. We first show that the small differences in the image domain can be highlighted in certain wavelet bands. Then the likelihood of each wavelet coefficient being foreground is estimated by formulating foreground and background models for each wavelet band. The proposed framework effectively aggregates the likelihoods from different wavelet bands based on the characteristics of the wavelet transform. Experimental results demonstrated that the proposed method significantly outperformed existing methods in detecting camouflaged foreground objects. Specifically, the average F-measure for the proposed algorithm was 0.87 compared with 0.71 to 0.8 for the other state-of-the-art methods.
---
paper_title: A benchmark dataset for outdoor foreground/background extraction

paper_content:

Most of video-surveillance based applications use a foreground extraction algorithm to detect interest objects from videos provided by static cameras. This paper presents a benchmark dataset and evaluation process built from both synthetic and real videos, used in the BMC workshop (Background Models Challenge). This dataset focuses on outdoor situations with weather variations such as wind, sun or rain. Moreover, we propose some evaluation criteria and an associated free software to compute them from several challenging testing videos. The evaluation process has been applied for several state of the art algorithms like gaussian mixture models or codebooks.
---
paper_title: The 3dSOBS+ algorithm for moving object detection

paper_content:

We propose the 3dSOBS+ algorithm, a newly designed approach for moving object detection based on a neural background model automatically generated by a self-organizing method. The algorithm is able to accurately handle scenes containing moving backgrounds, gradual illumination variations, and shadows cast by moving objects, and is robust against false detections for different types of videos taken with stationary cameras. Experimental results and comparisons conducted on the Background Models Challenge benchmark dataset demonstrate the improvements achieved by the proposed algorithm, that compares well with the state-of-the-art methods.
---
paper_title: Background Subtraction Based on Low-Rank and Structured Sparse Decomposition

paper_content:

Low rank and sparse representation based methods, which make few specific assumptions about the background, have recently attracted wide attention in background modeling. With these methods, moving objects in the scene are modeled as pixel-wised sparse outliers. However, in many practical scenarios, the distributions of these moving parts are not truly pixel-wised sparse but structurally sparse. Meanwhile a robust analysis mechanism is required to handle background regions or foreground movements with varying scales. Based on these two observations, we first introduce a class of structured sparsity-inducing norms to model moving objects in videos. In our approach, we regard the observed sequence as being constituted of two terms, a low-rank matrix (background) and a structured sparse outlier matrix (foreground). Next, in virtue of adaptive parameters for dynamic videos, we propose a saliency measurement to dynamically estimate the support of the foreground. Experiments on challenging well known data sets demonstrate that the proposed approach outperforms the state-of-the-art methods and works effectively on a wide range of complex videos.
---
paper_title: Moving Object Detection Using Scanning Camera on a High-Precision Intelligent Holder

paper_content:

During the process of moving object detection in an intelligent visual surveillance system, a scenario with complex background is sure to appear. The traditional methods, such as “frame difference” and “optical flow”, may not able to deal with the problem very well. In such scenarios, we use a modified algorithm to do the background modeling work. In this paper, we use edge detection to get an edge difference image just to enhance the ability of resistance illumination variation. Then we use a “multi-block temporal-analyzing LBP (Local Binary Pattern)” algorithm to do the segmentation. In the end, a connected component is used to locate the object. We also produce a hardware platform, the core of which consists of the DSP (Digital Signal Processor) and FPGA (Field Programmable Gate Array) platforms and the high-precision intelligent holder.
---
paper_title: A benchmark dataset for outdoor foreground/background extraction

paper_content:

Most of video-surveillance based applications use a foreground extraction algorithm to detect interest objects from videos provided by static cameras. This paper presents a benchmark dataset and evaluation process built from both synthetic and real videos, used in the BMC workshop (Background Models Challenge). This dataset focuses on outdoor situations with weather variations such as wind, sun or rain. Moreover, we propose some evaluation criteria and an associated free software to compute them from several challenging testing videos. The evaluation process has been applied for several state of the art algorithms like gaussian mixture models or codebooks.
---
paper_title: CANDID: Robust Change Dynamics and Deterministic Update Policy for Dynamic Background Subtraction

paper_content:

Background subtraction in video provides the preliminary information which is essential for many computer vision applications. In this paper, we propose a sequence of approaches named CANDID to handle the change detection problem in challenging video scenarios. The CANDID adaptively initializes the pixel-level distance threshold and update rate. These parameters are updated by computing the change dynamics at a location. Further, the background model is maintained by formulating a deterministic update policy. The performance of the proposed method is evaluated over various challenging scenarios such as dynamic background and extreme weather conditions. The qualitative and quantitative measures of the proposed method outperform the existing state-of-the-art approaches.
---
paper_title: Changedetection.net: A new change detection benchmark dataset

paper_content:

Change detection is one of the most commonly encountered low-level tasks in computer vision and video processing. A plethora of algorithms have been developed to date, yet no widely accepted, realistic, large-scale video dataset exists for benchmarking different methods. Presented here is a unique change detection benchmark dataset consisting of nearly 90,000 frames in 31 video sequences representing 6 categories selected to cover a wide range of challenges in 2 modalities (color and thermal IR). A distinguishing characteristic of this dataset is that each frame is meticulously annotated for ground-truth foreground, background, and shadow area boundaries — an effort that goes much beyond a simple binary label denoting the presence of change. This enables objective and precise quantitative comparison and ranking of change detection algorithms. This paper presents and discusses various aspects of the new dataset, quantitative performance metrics used, and comparative results for over a dozen previous and new change detection algorithms. The dataset, evaluation tools, and algorithm rankings are available to the public on a website1 and will be updated with feedback from academia and industry in the future.
---
paper_title: CDnet 2014: An Expanded Change Detection Benchmark Dataset

paper_content:

Change detection is one of the most important lowlevel tasks in video analytics. In 2012, we introduced the changedetection.net (CDnet) benchmark, a video dataset devoted to the evalaution of change and motion detection approaches. Here, we present the latest release of the CDnet dataset, which includes 22 additional videos (70; 000 pixel-wise annotated frames) spanning 5 new categories that incorporate challenges encountered in many surveillance settings. We describe these categories in detail and provide an overview of the results of more than a dozen methods submitted to the IEEE Change DetectionWorkshop 2014. We highlight strengths and weaknesses of these methods and identify remaining issues in change detection.
---
paper_title: WeSamBE: A Weight-Sample-Based Method for Background Subtraction

paper_content:

Background subtraction techniques are often treated as fundamental and significant ways to analyze and understand video content. In this paper, we propose a weight-sample-based method for foreground detection. This method allows us to use a few samples with variable weights to achieve effective change detection. To rapidly adapt to changing scenarios, a minimum-weight update policy is first proposed to replace the most inefficient sample instead of the oldest sample or a random sample. In addition, a reward-and-penalty weighting strategy is put forward to reinforce active samples and punish others. In this way, the weights of relatively effective samples are increased and the false updating of effective samples with smaller weights is reduced. Moreover, some other strategies, such as spatial-diffusion policy and random time subsampling, are also incorporated to ensure the flexibility of the proposed method. Finally, in our experiments, an adaptive feedback technique is incorporated into our algorithm to adapt to more challenging videos, and the final results indicate that our method is superior to the state-of-the-art approaches on the challenging CDnet data set.
---
paper_title: Binary Descriptor Based Nonparametric Background Modeling for Foreground Extraction by Using Detection Theory

paper_content:

Recently, most background modeling approaches represent distributions of background changes by using parametric models such as Gaussian mixture models. Because of significant illumination changes and dynamic moving backgrounds with time, variations of background changes are hard to be modeled by parametric background models. Moreover, how to efficiently and effectively update parameters of parametric models to reflect background changes remains a problem. In this paper, we propose a novel coarse-to-fine detection theory algorithm to extract foreground objects on the basis of nonparametric background and foreground models represented by binary descriptors. We update background and foreground models by a first-in–first-out strategy to maintain the most recent observed background and foreground instances. As shown in the experiments, our method can achieve better foreground extraction results and fewer false alarms of surveillance videos with lighting changes and dynamic backgrounds in both collected and CDnet 2012 benchmark data sets.
---
paper_title: Labeled dataset for integral evaluation of moving object detection algorithms: LASIESTA

paper_content:

Complete database for assessing the quality of foreground detection strategies.Indoor and outdoor sequences with many categories addressing different challenges.All the sequences are fully annotated at both pixel and object levels.Information concerning stationary foreground objects.Sequences recorded with static and moving cameras. Display Omitted A public, complete, compact, and well structured database is proposed, which allows to test moving object detection strategies. The database is composed of many real indoor and outdoor sequences organized in different categories, each of one covering a specific challenge. In contrast to other databases, the proposed one is fully annotated at both pixel and object levels. Therefore, it is suitable for strategies exclusively focused on the detection of moving objects and also for those that integrate tracking algorithms in their detection approaches. Additionally, it contains sequences recorded with static and moving cameras and it also provides information about the moving objects remaining temporally static.To test its usefulness, the database has been used to assess the quality of some outstanding moving object detection methods.
---
paper_title: Detection of Stationary Foreground Objects Using Multiple Nonparametric Background-Foreground Models on a Finite State Machine

paper_content:

There is a huge proliferation of surveillance systems that require strategies for detecting different kinds of stationary foreground objects (e.g., unattended packages or illegally parked vehicles). As these strategies must be able to detect foreground objects remaining static in crowd scenarios, regardless of how long they have not been moving, several algorithms for detecting different kinds of such foreground objects have been developed over the last decades. This paper presents an efficient and high-quality strategy to detect stationary foreground objects, which is able to detect not only completely static objects but also partially static ones. Three parallel nonparametric detectors with different absorption rates are used to detect currently moving foreground objects, short-term stationary foreground objects, and long-term stationary foreground objects. The results of the detectors are fed into a novel finite state machine that classifies the pixels among background, moving foreground objects, stationary foreground objects, occluded stationary foreground objects, and uncovered background. Results show that the proposed detection strategy is not only able to achieve high quality in several challenging situations but it also improves upon previous strategies.
---
paper_title: Real-time nonparametric background subtraction with tracking-based foreground update

paper_content:

Abstract A nonparametric real-time and high-quality moving object detection strategy in a GPU is proposed. To improve the quality of the results in sequences where the moving objects and the background have similar appearance, not only the background but also the foreground is modelled. Both models are constructed from spatio-temporal reference data to reduce false detections due to small displacements of the background, and to take into consideration the natural displacements of the foreground. To avoid using kernels with too large spatial widths, the spatial positions of the foreground reference data are updated at each new frame using a particle filter that is able to deal with an unknown and variable amount of regions. Additionally, an automatic selection of regions of interest is carried out, which allows reducing drastically the computational cost of both foreground and background models. The proposed strategy has been validated using three databases containing many challenges for motion detection and the results have been compared to those of other state-of-the-art approaches.
---
paper_title: Background Subtraction with Multi-scale Structured Low-Rank and Sparse Factorization

paper_content:

Abstract Low-rank and sparse factorization, which models the background as a low-rank matrix and the foreground as the contiguously corrupted outliers, exhibits excellent performance in background subtraction, in which the structured constraints of the foreground usually play a very essential role. In this paper, we propose a novel approach with multi-scale structured low-rank and sparse factorization for background subtraction. Different from the conventional methods that only enforce the smoothness between the spatial neighbors, we propose to explore the structured smoothness with both appearance consistency and spatial compactness in the low-rank and sparse factorization framework. Moreover, we integrate structural information at different scales into the formulation for robustness. We also design a low-rank decomposition scheme to improve the computational efficiency of the optimization algorithm. Extensive experiments on benchmark datasets GTFD and CDnet suggest that our approach achieves big superior performance against the state-of-the-art methods.
---
paper_title: Fast Grayscale-Thermal Foreground Detection With Collaborative Low-Rank Decomposition

paper_content:

This paper investigates how to perform efficient and robust foreground detection in challenging scenarios by leveraging multiple source data. We propose a novel approach, called collaborative low-rank decomposition (CLoD), for grayscale-thermal foreground detection. Given two data matrices by accumulating sequential frames from the grayscale and the thermal videos, CLoD detects the foreground objects as sparse noises against the backgrounds with collaborative low rank structure, and also incorporates modality weights to achieve adaptive fusion of different source data. For the optimization, CLoD seeks a sub-optimal solution by making the background matrix rank explicitly determined. In particular, the background matrix with the fixed rank can be decomposed into two sub-matrices of low rank, and then, we iteratively optimize them and the modality weights with closed-form solutions. For improving the efficiency, we design a block-based accelerated algorithm to speed up CLoD while employing the edge-preserving algorithm to keep the accuracy. Extensive experiments on the recently public benchmark grayscale-thermal foreground detection suggest that our approach achieves comparable performance in terms of both accuracy and efficiency against other state-of-the-art methods.
---
paper_title: Background subtraction model based on color and depth cues

paper_content:

Background subtraction consists of segmenting objects in movement in a video captured by a static camera. This is typically performed using color information, but it leads to wrong estimations due to perspective and illumination issues. We show that multimodal approaches based on the integrated use of color and depth cues produce more accurate and robust results than using either data source independently. Depth is less affected by issues such as shadows or foreground objects similar to background. However, objects close to the background may not be detected when using only range information, being color information complementary in those cases. We propose an extension of a well-known background subtraction technique which fuses range and color information, as well as a post-processing mask fusion stage to get the best of each feature. We have evaluated the method proposed using a well-defined dataset and different disparity estimation algorithms, showing the benefits of our method for fusion color and depth cues.
---
paper_title: Background Subtraction Based on Color and Depth Using Active Sensors

paper_content:

Depth information has been used in computer vision for a wide variety of tasks. Since active range sensors are currently available at low cost, high-quality depth maps can be used as relevant input for many applications. Background subtraction and video segmentation algorithms can be improved by fusing depth and color inputs, which are complementary and allow one to solve many classic color segmentation issues. In this paper, we describe one fusion method to combine color and depth based on an advanced color-based algorithm. This technique has been evaluated by means of a complete dataset recorded with Microsoft Kinect, which enables comparison with the original method. The proposed method outperforms the others in almost every test, showing more robustness to illumination changes, shadows, reflections and camouflage.
---
paper_title: Self-Adapting Fuzzy Model for Dynamic Object Detection Using RGB-D Information

paper_content:

This paper describes a novel method for dynamic object detection in RGB-D videos based completely on a fuzzy logic approach. The method is an original contribution because of its self-adapting fuzzy scheme that fuses color and depth information (RGB-D). The fuzzy system analyzes information related to fuzzy color and depth differences as well as fuzzy depth similitude. In order to improve the segmentation results, special cases for incomplete information in the depth pixels are treated by four new fuzzy pixel concepts; complete pixels, empty pixel, new pixel, and color pixel. The fuzzy method also solves flickering and oscillations on edge depth measurements. The proposed system also involves a self-adapting background update mechanism to avoid problems regarding changes in the video scenario sequence, and manual parameter adjustment required in other reported methods. Although the proposed method is self-adapting, it has a better or similar performance in most of the comparative cases used for analysis.
---
paper_title: Efficient local monitoring approach for the task of background subtraction

paper_content:

Abstract We present in this paper a novel and efficient method that will significantly reduce GMM drawbacks in the presence of complex and dynamic scene. The main idea is to combine global and local features to remove local variations and the instant variations in the brightness that, in most cases, decrease the performance of background subtraction models. The first step is to divide the extracted frames into several equal size blocks. Then, we apply an adaptive local monitoring algorithm for each block to control local variation using Pearson similarity measurement. When a significant environment changes are detected in one or more blocks, the parameters of GMM assigned to these blocks are updated and the parameters of the rest remain the same. We also proposed merging adjacent and invariant blocks to reduce processing time and splitting the blocks that have an intense movement to improve accuracy. Experimental results on several datasets demonstrate that the proposed approach is effective and efficient under the common problems found in background modeling, outperforming the most referred state-of-the-art background subtraction methods.
---
paper_title: Background foreground segmentation with RGB-D Kinect data: An efficient combination of classifiers

paper_content:

HighlightsWe use Kinect RGB-D data for foreground/background segmentation.Combination of Classifiers is used to improve segmentation performance.Generation of hand-labeled public available RGB-D benchmark dataset. Low cost RGB-D cameras such as the Microsoft's Kinect or the Asus's Xtion Pro are completely changing the computer vision world, as they are being successfully used in several applications and research areas. Depth data are particularly attractive and suitable for applications based on moving objects detection through foreground/background segmentation approaches; the RGB-D applications proposed in literature employ, in general, state of the art foreground/background segmentation techniques based on the depth information without taking into account the color information. The novel approach that we propose is based on a combination of classifiers that allows improving background subtraction accuracy with respect to state of the art algorithms by jointly considering color and depth data. In particular, the combination of classifiers is based on a weighted average that allows to adaptively modifying the support of each classifier in the ensemble by considering foreground detections in the previous frames and the depth and color edges. In this way, it is possible to reduce false detections due to critical issues that can not be tackled by the individual classifiers such as: shadows and illumination changes, color and depth camouflage, moved background objects and noisy depth measurements. Moreover, we propose, for the best of the author's knowledge, the first publicly available RGB-D benchmark dataset with hand-labeled ground truth of several challenging scenarios to test background/foreground segmentation algorithms.
---
paper_title: Simple Combination of Appearance and Depth for Foreground Segmentation

paper_content:

In foreground segmentation, the depth information is robust to problems of the appearance information such as illumination changes and color camouflage; however, the depth information is not always measured and suffers from depth camouflage. In order to compensate for the disadvantages of the two pieces of information, we define an energy function based on the two likelihoods of depth and appearance backgrounds and minimize the energy using graph cuts to obtain a foreground mask. The two likelihoods are obtained using background subtraction. We use the farthest depth as the depth background in the background subtraction according to the depth information. The appearance background is defined as the appearance with a large likelihood of the depth background to eliminate appearances of foreground objects. In the computation of the likelihood of the appearance background, we also use the likelihood of the depth background for reducing false positives owing to illumination changes. In our experiment, we confirm that our method is sufficiently accurate for indoor environments using the SBM-RGBD 2017 dataset.
---
paper_title: A Benchmarking Framework for Background Subtraction in RGBD Videos

paper_content:

The complementary nature of color and depth synchronized information acquired by low cost RGBD sensors poses new challenges and design opportunities in several applications and research areas. Here, we focus on background subtraction for moving object detection, which is the building block for many computer vision applications, being the first relevant step for subsequent recognition, classification, and activity analysis tasks. The aim of this paper is to describe a novel benchmarking framework that we set up and made publicly available in order to evaluate and compare scene background modeling methods for moving object detection on RGBD videos. The proposed framework involves the largest RGBD video dataset ever made for this specific purpose. The 33 videos span seven categories, selected to include diverse scene background modeling challenges for moving object detection. Seven evaluation metrics, chosen among the most widely used, are adopted to evaluate the results against a wide set of pixel-wise ground truths. Moreover, we present a preliminary analysis of results, devoted to assess to what extent the various background modeling challenges pose troubles to background subtraction methods exploiting color and depth information.
---
paper_title: Efficient and fast multi-modal foreground-background segmentation using RGBD data

paper_content:

Abstract This paper addresses the problem of foreground and background segmentation. Multi-modal data specifically RGBD data has gain many tasks in computer vision recently. However, techniques for background subtraction based only on single-modality still report state-of-the-art results in many benchmarks. Succeeding the fusion of depth and color data for this task requires a robust formalization allowing at the same time higher precision and faster processing. To this end, we propose to make use of kernel density estimation technique to adapt multi-modal data. To speed up kernel density estimation, we explore the fast Gauss transform which allows the summation of a mixture of M kernel at N evaluation points in O(M+N) time as opposed to O(MN) time for a direct evaluation. Extensive experiments have been carried out on four publicly available RGBD foreground/background datasets. Results demonstrate that our proposal outperforms state-of-the-art methods for almost all of the sequences acquired in challenging indoor and outdoor contexts with a fast and non-parametric operation.
---
paper_title: Modelling depth for nonparametric foreground segmentation using RGBD devices

paper_content:

The problem of detecting changes in a scene and segmenting the foreground from background is still challenging, despite previous work. Moreover, new RGBD capturing devices include depth cues, which could be incorporated to improve foreground segmentation. In this work, we present a new nonparametric approach where a unified model mixes the device multiple information cues. In order to unify all the device channel cues, a new probabilistic depth data model is also proposed where we show how handle the inaccurate data to improve foreground segmentation. A new RGBD video dataset is presented in order to introduce a new standard for comparison purposes of this kind of algorithms. Results show that the proposed approach can handle several practical situations and obtain good results in all cases.
---
paper_title: Background Subtraction for Moving Object Detection in RGBD Data: A Survey

paper_content:

The paper provides a specific perspective view on background subtraction for moving object detection, as a building block for many computer vision applications, being the first relevant step for subsequent recognition, classification, and activity analysis tasks. Since color information is not sufficient for dealing with problems like light switches or local gradual changes of illumination, shadows cast by the foreground objects, and color camouflage, new information needs to be caught to deal with these issues. Depth synchronized information acquired by low-cost RGBD sensors is considered in this paper to give evidence about which issues can be solved, but also to highlight new challenges and design opportunities in several applications and research areas.
---
paper_title: Modelling depth for nonparametric foreground segmentation using RGBD devices

paper_content:

The problem of detecting changes in a scene and segmenting the foreground from background is still challenging, despite previous work. Moreover, new RGBD capturing devices include depth cues, which could be incorporated to improve foreground segmentation. In this work, we present a new nonparametric approach where a unified model mixes the device multiple information cues. In order to unify all the device channel cues, a new probabilistic depth data model is also proposed where we show how handle the inaccurate data to improve foreground segmentation. A new RGBD video dataset is presented in order to introduce a new standard for comparison purposes of this kind of algorithms. Results show that the proposed approach can handle several practical situations and obtain good results in all cases.
---
paper_title: Background Subtraction in Real Applications: Challenges, Current Models and Future Directions

paper_content:

Abstract Computer vision applications based on videos often require the detection of moving objects in their first step. Background subtraction is then applied in order to separate the background and the foreground. In literature, background subtraction is surely among the most investigated field in computer vision providing a big amount of publications. Most of them concern the application of mathematical and machine learning models to be more robust to the challenges met in videos. However, the ultimate goal is that the background subtraction methods developed in research could be employed in real applications like traffic surveillance. But looking at the literature, we can remark that there is often a gap between the current methods used in real applications and the current methods in fundamental research. In addition, the videos evaluated in large-scale datasets are not exhaustive in the way that they only covered a part of the complete spectrum of the challenges met in real applications. In this context, we attempt to provide the most exhaustive survey as possible on real applications that used background subtraction in order to identify the real challenges met in practice, the current used background models and to provide future directions. Thus, challenges are investigated in terms of camera (i.e CCD cameras, omnidirectional cameras, …), foreground objects and environments. In addition, we identify the background models that are effectively used in these applications in order to find potential usable recent background models in terms of robustness, time and memory requirements.
---
paper_title: Towards Benchmarking Scene Background Initialization

paper_content:

Given a set of images of a scene taken at different times, the availability of an initial background model that describes the scene without foreground objects is the prerequisite for a wide range of applications, ranging from video surveillance to computational photography. Even though several methods have been proposed for scene background initialization, the lack of a common groundtruthed dataset and of a common set of metrics makes it difficult to compare their performance. To move first steps towards an easy and fair comparison of these methods, we assembled a dataset of sequences frequently adopted for background initialization, selected or created ground truths for quantitative evaluation through a selected suite of metrics, and compared results obtained by some existing methods, making all the material publicly available.
---
paper_title: Scene background initialization: A taxonomy

paper_content:

The availability of an initial background model that describes the scene without foreground objects is the prerequisite for a wide range of applications, ranging from video surveillance to computational photography. Limited attention to the problem is given in the literature on background modeling, that mainly regards model representation and updating. Therefore, we propose a taxonomy study for background initialization, providing the basis for a fair and easy comparison of existing and future methods, on a common dataset of groundtruthed sequences, with a common set of metrics, and based on reproducible results. Experimental results highlight the most promising approaches as well as main open issues for background initialization.
---
paper_title: Background estimation by weightless neural networks

paper_content:

Abstract Initial background estimation in video processing serves as bootstrapping model for moving object detection based on background subtraction. In long-term videos, the initial background model may require a constant update as long as the environmental changes happen (slowly or suddenly). In this paper we approach the background initialization together with its constant updating in time by modeling video background as ever-changing states of weightless neural networks. The result is a background estimation method based on a weightless neural network, called BEWiS. The approach proposed in this work is simple: background estimation at each pixel is carried out by weightless neural networks designed to learn pixel color frequency during video play, and all networks share the same rule for memory retention during training. This approach has the advantage of providing a useful background model at the very beginning of the video, since it operates in unsupervised mode. On the other hand, depending on the video scene, the pixel-level learning rule can be tuned to tackle the specificities and difficulties of the scene. The approach presented in this work has been experimented on the public Scene Background Initialization 2015 dataset and on the Scene Background Modeling Contest 2016 dataset, and it showed a performance comparable or superior to state-of-the-art methods.
---
paper_title: Adaptive bootstrapping management by keypoint clustering for background initialization

paper_content:

A novel adaptive bootstrapping management method for background initialization and updating is proposed.It works with both static and mobile cameras.It works with a small number of frames to initialize the background model. The availability of a background model that describes the scene is a prerequisite for many computer vision applications. In several situations, the model cannot be easily generated when the background contains some foreground objects (i.e., bootstrapping problem). In this letter, an Adaptive Bootstrapping Management (ABM) method, based on keypoint clustering, is proposed to model the background on video sequences acquired by mobile and static cameras. First, keypoints are detected on each frame by the A-KAZE feature extractor, then Density-Based Spatial Clustering of Application with Noise (DBSCAN) is used to find keypoint clusters. These clusters represent the candidate regions of foreground elements inside the scene. The ABM method manages the scene changes generated by foreground elements, both in the background model initialization, managing the bootstrapping problem, and in the background model updating. Moreover, it achieves good results with both mobile and static cameras and it requires a small number of frames to initialize the background model.
---
paper_title: Rejection based multipath reconstruction for background estimation in SBMnet 2016 dataset

paper_content:

Background Estimation in video consists in extracting a foreground-free image from a set of training frames. In this paper, we overview a temporal-spatial block-level approach for background estimation in video and present their results in the SBMnet dataset. First, the employed approach uses a Temporal Analysis module to obtain a compact representation of the training data that is later clustered by a threshold-free technique to generate background candidates at each block location. Then, a Spatial Analysis module iteratively reconstructs the background using a multipath reconstruction guided by background smoothness constraints. The experimental results in the SBMnet dataset demonstrates the utility of the employed approach against stationary objects and its weaknesses when motion information is involved.
---
paper_title: Extensive Benchmark and Survey of Modeling Methods for Scene Background Initialization

paper_content:

Scene background initialization is the process by which a method tries to recover the background image of a video without foreground objects in it. Having a clear understanding about which approach is more robust and/or more suited to a given scenario is of great interest to many end users or practitioners. The aim of this paper is to provide an extensive survey of scene background initialization methods as well as a novel benchmarking framework. The proposed framework involves several evaluation metrics and state-of-the-art methods, as well as the largest video data set ever made for this purpose. The data set consists of several camera-captured videos that: 1) span categories focused on various background initialization challenges; 2) are obtained with different cameras of different lengths, frame rates, spatial resolutions, lighting conditions, and levels of compression; and 3) contain indoor and outdoor scenes. The wide variety of our data set prevents our analysis from favoring a certain family of background initialization methods over others. Our evaluation framework allows us to quantitatively identify solved and unsolved issues related to scene background initialization. We also identify scenarios for which state-of-the-art methods systematically fail.
---
paper_title: A Robust Background Initialization Algorithm with Superpixel Motion Detection

paper_content:

Abstract Scene background initialization allows the recovery of a clear image without foreground objects from a video sequence, which is generally the first step in many computer vision and video processing applications. The process may be strongly affected by some challenges such as illumination changes, foreground cluttering, intermittent movement, etc. In this paper, a robust background initialization approach based on superpixel motion detection is proposed. Both spatial and temporal characteristics of frames are adopted to effectively eliminate foreground objects. A subsequence with stable illumination condition is first selected for background estimation. Images are segmented into superpixels to preserve spatial texture information and foreground objects are eliminated by superpixel motion filtering process. A low-complexity density-based clustering is then performed to generate reliable background candidates for final background determination. The approach has been evaluated on SBMnet dataset and it achieves a performance superior or comparable to other state-of-the-art works with faster processing speed. Moreover, in those complex and dynamic categories, the algorithm produces the best results showing the robustness against very challenging scenarios.
---
paper_title: Extensive Benchmark and Survey of Modeling Methods for Scene Background Initialization

paper_content:

Scene background initialization is the process by which a method tries to recover the background image of a video without foreground objects in it. Having a clear understanding about which approach is more robust and/or more suited to a given scenario is of great interest to many end users or practitioners. The aim of this paper is to provide an extensive survey of scene background initialization methods as well as a novel benchmarking framework. The proposed framework involves several evaluation metrics and state-of-the-art methods, as well as the largest video data set ever made for this purpose. The data set consists of several camera-captured videos that: 1) span categories focused on various background initialization challenges; 2) are obtained with different cameras of different lengths, frame rates, spatial resolutions, lighting conditions, and levels of compression; and 3) contain indoor and outdoor scenes. The wide variety of our data set prevents our analysis from favoring a certain family of background initialization methods over others. Our evaluation framework allows us to quantitatively identify solved and unsolved issues related to scene background initialization. We also identify scenarios for which state-of-the-art methods systematically fail.
---
paper_title: Towards Benchmarking Scene Background Initialization

paper_content:

Given a set of images of a scene taken at different times, the availability of an initial background model that describes the scene without foreground objects is the prerequisite for a wide range of applications, ranging from video surveillance to computational photography. Even though several methods have been proposed for scene background initialization, the lack of a common groundtruthed dataset and of a common set of metrics makes it difficult to compare their performance. To move first steps towards an easy and fair comparison of these methods, we assembled a dataset of sequences frequently adopted for background initialization, selected or created ground truths for quantitative evaluation through a selected suite of metrics, and compared results obtained by some existing methods, making all the material publicly available.
---
paper_title: Comparison of Background Subtraction Methods for a Multimedia Application

paper_content:

This article presents the Aqu@theque application and a comparison of background subtraction methods to improve the performance of this application. The Aqu@theque application consists in elaborating an information system dedicated to aquariums in an interactive learning area. In particular, this article presents our comparison of different background subtraction methods to detect fish in video sequences and the improvement for the Aqu@theque application.
---
paper_title: An innovative web-based collaborative platform for video annotation

paper_content:

Large scale labeled datasets are of key importance for the development of automatic video analysis tools as they, from one hand, allow multi-class classifiers training and, from the other hand, support the algorithms' evaluation phase. This is widely recognized by the multimedia and computer vision communities, as witnessed by the growing number of available datasets; however, the research still lacks in annotation tools able to meet user needs, since a lot of human concentration is necessary to generate high quality ground truth data. Nevertheless, it is not feasible to collect large video ground truths, covering as much scenarios and object categories as possible, by exploiting only the effort of isolated research groups. In this paper we present a collaborative web-based platform for video ground truth annotation. It features an easy and intuitive user interface that allows plain video annotation and instant sharing/integration of the generated ground truths, in order to not only alleviate a large part of the effort and time needed, but also to increase the quality of the generated annotations. The tool has been on-line in the last four months and, at the current date, we have collected about 70,000 annotations. A comparative performance evaluation has also shown that our system outperforms existing state of the art methods in terms of annotation time, annotation quality and system's usability.
---
paper_title: Pixel Modeling Using Histograms Based on Fuzzy Partitions for Dynamic Background Subtraction

paper_content:

We propose a novel pixel-modeling approach for background subtraction using histograms based on strong uniform fuzzy partitions. In the proposed method, the temporal distribution of pixel values is represented by a histogram based on a triangular partition. The threshold for background segmentation is set adaptively according to the shape of the histogram. Histogram accumulation is controlled adaptively by a fuzzy controller under a supervised learning framework. Benefiting from the adaptive scheme, with no parameter tuning, the proposed algorithm functions well across a wide spectrum of challenging environments. The performance of the proposed method is evaluated against more than 20 state-of-the-art methods in complex outdoor environments, particularly in those consisting of highly dynamic backgrounds and camouflaged foregrounds. Experimental results confirm that the proposed method performs effectively in terms of both the true positive rate and the noise suppression ability. Further, it outperforms other state-of-the-art methods by a significant margin.
---
paper_title: A texton-based kernel density estimation approach for background modeling under extreme conditions

paper_content:

Abstract Background modeling is a well-know approach to detect moving objects in video sequences. In recent years, background modeling methods that adopt spatial and texture information have been developed for dealing with complex scenarios. However, none of the investigated approaches have been tested under extreme conditions, such as the underwater domain, on which effects compromising the video quality affect negatively the performance of the background modeling process. In order to overcome such difficulties, more significant features and more robust methods must be found. In this paper, we present a kernel density estimation method which models background and foreground by exploiting textons to describe textures within small and low contrasted regions. Comparison with other texture descriptors, namely, local binary pattern (LBP) and scale invariant local ternary pattern (SILTP) shown improved performance. Besides, quantitative and qualitative performance evaluation carried out on three standard datasets showing very complex conditions revealed that our method outperformed state-of-the-art methods that use different features and modeling techniques and, most importantly, it is able to generalize over different scenarios and targets.
---
paper_title: Recognition in Terra Incognita

paper_content:

It is desirable for detection and classification algorithms to generalize to unfamiliar environments, but suitable benchmarks for quantitatively studying this phenomenon are not yet available. We present a dataset designed to measure recognition generalization to novel environments. The images in our dataset are harvested from twenty camera traps deployed to monitor animal populations. Camera traps are fixed at one location, hence the background changes little across images; capture is triggered automatically, hence there is no human bias. The challenge is learning recognition in a handful of locations, and generalizing animal detection and classification to new locations where no training data is available. In our experiments state-of-the-art algorithms show excellent performance when tested at the same location where they were trained. However, we find that generalization to new locations is poor, especially for classification systems.
---
paper_title: Social behavior recognition in continuous video

paper_content:

We present a novel method for analyzing social behavior. Continuous videos are segmented into action ‘bouts’ by building a temporal context model that combines features from spatio-temporal energy and agent trajectories. The method is tested on an unprecedented dataset of videos of interacting pairs of mice, which was collected as part of a state-of-the-art neurophysiological study of behavior. The dataset comprises over 88 hours (8 million frames) of annotated videos. We find that our novel trajectory features, used in a discriminative framework, are more informative than widely used spatio-temporal features; furthermore, temporal context plays an important role for action recognition in continuous videos. Our approach may be seen as a baseline method on this dataset, reaching a mean recognition rate of 61.2% compared to the expert's agreement rate of about 70%.
---
paper_title: ARGOS-Venice Boat Classification

paper_content:

Detection, classification, and tracking of people and vehicles are fundamental processes in intelligent surveillance systems. The use of publicly available data set is the appropriate way to compare the relative merits of existing methods and to develop and assess new robust solutions. In this paper, we focus on the maritime domain and we describe the generation of boat classification data sets, containing images of boats automatically extracted by the ARGOS system, operating 24/7 in Venice, Italy. The data sets are unique in their nature, since they come from an incomparable environment like Venice, but they present very interesting challenges to vehicle classification, due to changes in the environmental conditions, boat wakes, waves, reflections, etc. We thus believe that robust techniques, validated through the ARGOS Boat Classification data sets, will improve the development and deployment of solutions in similar applications related to vehicle detection and classification.
---
paper_title: MuHAVi: A Multicamera Human Action Video Dataset for the Evaluation of Action Recognition Methods

paper_content:

This paper describes a body of multicamera humanaction video data with manually annotated silhouette datathat has been generated for the purpose of evaluatingsilhouette-based human action recognition methods. Itprovides a realistic challenge to both the segmentationand human action recognition communities and can act asa benchmark to objectively compare proposed algorithms.The public multi-camera, multi-action dataset is animprovement over existing datasets (e.g. PETS, CAVIAR,soccerdataset) that have not been developed specificallyfor human action recognition and complements otheraction recognition datasets (KTH, Weizmann, IXMAS,HumanEva, CMU Motion). It consists of 17 action classes,14 actors and 8 cameras. Each actor performs an actionseveral times in the action zone. The paper describes thedataset and illustrates a possible approach to algorithmevaluation using a previously published action simplerecognition method. In addition to showing an evaluationmethodology, these results establish a baseline for otherresearchers to improve upon.
---
paper_title: HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion

paper_content:

While research on articulated human motion and pose estimation has progressed rapidly in the last few years, there has been no systematic quantitative evaluation of competing methods to establish the current state of the art. We present data obtained using a hardware system that is able to capture synchronized video and ground-truth 3D motion. The resulting HumanEva datasets contain multiple subjects performing a set of predefined actions with a number of repetitions. On the order of 40,000 frames of synchronized motion capture and multi-view video (resulting in over one quarter million image frames in total) were collected at 60 Hz with an additional 37,000 time instants of pure motion capture data. A standard set of error measures is defined for evaluating both 2D and 3D pose estimation and tracking algorithms. We also describe a baseline algorithm for 3D articulated tracking that uses a relatively standard Bayesian framework with optimization in the form of Sequential Importance Resampling and Annealed Particle Filtering. In the context of this baseline algorithm we explore a variety of likelihood functions, prior models of human motion and the effects of algorithm parameters. Our experiments suggest that image observation models and motion priors play important roles in performance, and that in a multi-view laboratory environment, where initialization is available, Bayesian filtering tends to perform well. The datasets and the software are made available to the research community. This infrastructure will support the development of new articulated motion and pose estimation algorithms, will provide a baseline for the evaluation and comparison of new methods, and will help establish the current state of the art in human pose estimation and tracking.
---
paper_title: Toward a kindergarten video surveillance system (KVSS) using background subtraction based Type-2 FGMM model

paper_content:

This paper presents a new video surveillance system called KVSS using background based on Type-2 Fuzzy Gaussian Mixture Models (T2 FGMMs). These techniques are used for resolving some limitations on Gaussian Mixture Models (GMMs) techniques on critical situations like moved camera jitter, illumination changes and objects being introduced or removed from the scene. In this context, we introduce descriptions of T2 GMMs and we present an experimental validation using a new evaluation video dataset which presents various problems. Results demonstrate the relevance of the proposed system.
---
paper_title: Fuzzy Logic Based Human Activity Recognition in Video Surveillance Applications

paper_content:

Automatic fall detection using computer vision is a particular case for real time video analysis, efficient in kindergartens. This paper is focused on the design and implementation of a Human activity analysis system. The multiple cameras sends captured frames to the monitoring system via the local network. Through the use of human silhouette, acquired from a smart camera, a shape representation of the human beings was built in real-time and a fuzzy logic inference system was developed for fall detection. The system also allows tracking and localizing children within an authorized area. The alarm is triggered in case of transgression. Experimental results prove that the fuzzy inference system is efficient.
---
paper_title: A benchmark dataset for outdoor foreground/background extraction

paper_content:

Most of video-surveillance based applications use a foreground extraction algorithm to detect interest objects from videos provided by static cameras. This paper presents a benchmark dataset and evaluation process built from both synthetic and real videos, used in the BMC workshop (Background Models Challenge). This dataset focuses on outdoor situations with weather variations such as wind, sun or rain. Moreover, we propose some evaluation criteria and an associated free software to compute them from several challenging testing videos. The evaluation process has been applied for several state of the art algorithms like gaussian mixture models or codebooks.
---
paper_title: A testing framework for background subtraction algorithms comparison in intrusion detection context

paper_content:

Identifying objects from a video stream is a fundamental and critical task in many computer-vision applications. A popular approach is the background subtraction, which consists in separating foreground (moving objects) from background. Many methodologies have been developed for automatic background segmentation but this fundamental task is still challenging. We focus here on a particular application of computer vision: intrusion detection in video surveillance. We propose in this paper a multi-level methodology for evaluating and comparing background subtraction algorithms. Three levels are studied: first, pixel level to evaluate the accuracy of the segmentation algorithm to attribute the right class to each pixel. Second, image level, measuring the rate of right decision on each frame (intrusion vs no intrusion) and finally sequence level, measuring the accordance with the time span where objects appear. Moreover, we also propose a new similarity measure, called D-Score, adapted to the context of intrusion detection.
---
paper_title: Extensive Benchmark and Survey of Modeling Methods for Scene Background Initialization

paper_content:

Scene background initialization is the process by which a method tries to recover the background image of a video without foreground objects in it. Having a clear understanding about which approach is more robust and/or more suited to a given scenario is of great interest to many end users or practitioners. The aim of this paper is to provide an extensive survey of scene background initialization methods as well as a novel benchmarking framework. The proposed framework involves several evaluation metrics and state-of-the-art methods, as well as the largest video data set ever made for this purpose. The data set consists of several camera-captured videos that: 1) span categories focused on various background initialization challenges; 2) are obtained with different cameras of different lengths, frame rates, spatial resolutions, lighting conditions, and levels of compression; and 3) contain indoor and outdoor scenes. The wide variety of our data set prevents our analysis from favoring a certain family of background initialization methods over others. Our evaluation framework allows us to quantitatively identify solved and unsolved issues related to scene background initialization. We also identify scenarios for which state-of-the-art methods systematically fail.
---
paper_title: Image quality assessment: from error visibility to structural similarity

paper_content:

Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.
---
paper_title: Adaptive Multicue Background Subtraction for Robust Vehicle Counting and Classification

paper_content:

In this paper, we present a robust vision-based system for vehicle tracking and classification devised for traffic flow surveillance. The system performs in real time, achieving good results, even in challenging situations, such as with moving casted shadows on sunny days, headlight reflections on the road, rainy days, and traffic jams, using only a single standard camera. We propose a robust adaptive multicue segmentation strategy that detects foreground pixels corresponding to moving and stopped vehicles, even with noisy images due to compression. First, the approach adaptively thresholds a combination of luminance and chromaticity disparity maps between the learned background and the current frame. It then adds extra features derived from gradient differences to improve the segmentation of dark vehicles with casted shadows and removes headlight reflections on the road. The segmentation is further used by a two-step tracking approach, which combines the simplicity of a linear 2-D Kalman filter and the complexity of a 3-D volume estimation using Markov chain Monte Carlo (MCMC) methods. Experimental results show that our method can count and classify vehicles in real time with a high level of performance under different environmental situations comparable with those of inductive loop detectors.
---
paper_title: Evaluation of Background Subtraction Algorithms with Post-Processing

paper_content:

Processing a video stream to segment foreground objects from the background is a critical first step in many computer vision applications. Background subtraction (BGS) is a commonly used technique for achieving this segmentation. The popularity of BGS largely comes from its computational efficiency, which allows applications such as human-computer interaction, video surveillance, and traffic monitoring to meet their real-time goals. Numerous BGS algorithms and a number of post-processing techniques that aim to improve the results of these algorithms have been proposed. In this paper, we evaluate several popular, state-of-the-art BGS algorithms and examine how post-processing techniques affect their performance. Our experimental results demonstrate that post-processing techniques can significantly improve the foreground segmentation masks produced by a BGS algorithm. We provide recommendations for achieving robust foreground segmentation based on the lessons learned performing this comparative study.
---
paper_title: Static and Moving Object Detection Using Flux Tensor with Split Gaussian Models

paper_content:

In this paper, we present a moving object detection system named Flux Tensor with Split Gaussian models (FTSG) that exploits the benefits of fusing a motion computation method based on spatio-temporal tensor formulation, a novel foreground and background modeling scheme, and a multi-cue appearance comparison. This hybrid system can handle challenges such as shadows, illumination changes, dynamic background, stopped and removed objects. Extensive testing performed on the CVPR 2014 Change Detection benchmark dataset shows that FTSG outperforms state-ofthe-art methods.
---
paper_title: Background subtraction using encoder-decoder structured convolutional neural network

paper_content:

A background subtraction algorithm using an encoderdecoder structured convolutional neural network is proposed in this work, in order to segment out moving objects from the background. A target frame, its previous frame, and a background model are concatenated and fed into the network as the input. Then, the encoder generates a highlevel feature vector, and the decoder converts the feature vector into a segmentation map, which roughly identifies moving object regions. Moreover, we develop background modeling and foreground extraction techniques, which exploit contour information. Experimental results on the CD-net2014 dataset demonstrate that the proposed algorithm outperforms state-of-the-art techniques significantly.
---
paper_title: Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey

paper_content:

Deep learning has recently achieved very promising results in a wide range of areas such as computer vision, speech recognition and natural language processing. It aims to learn hierarchical representations of data by using deep architecture models. In a smart city, a lot of data (e.g. videos captured from many distributed sensors) need to be automatically processed and analyzed. In this paper, we review the deep learning algorithms applied to video analytics of smart city in terms of different research topics: object detection, object tracking, face recognition, image classification and scene labeling.
---
paper_title: SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity

paper_content:

Foreground/background segmentation via change detection in video sequences is often used as a stepping stone in high-level analytics and applications. Despite the wide variety of methods that have been proposed for this problem, none has been able to fully address the complex nature of dynamic scenes in real surveillance tasks. In this paper, we present a universal pixel-level segmentation method that relies on spatiotemporal binary features as well as color information to detect changes. This allows camouflaged foreground objects to be detected more easily while most illumination variations are ignored. Besides, instead of using manually set, frame-wide constants to dictate model sensitivity and adaptation speed, we use pixel-level feedback loops to dynamically adjust our method’s internal parameters without user intervention. These adjustments are based on the continuous monitoring of model fidelity and local segmentation noise levels. This new approach enables us to outperform all 32 previously tested state-of-the-art methods on the 2012 and 2014 versions of the ChangeDetection.net dataset in terms of overall F-Measure. The use of local binary image descriptors for pixel-level modeling also facilitates high-speed parallel implementations: our own version, which used no low-level or architecture-specific instruction, reached real-time processing speed on a midlevel desktop CPU. A complete C++ implementation based on OpenCV is available online.
---
paper_title: A Novel Background Subtraction Algorithm Based on Parallel Vision and Bayesian GANs

paper_content:

Abstract To address the challenges of change detection in the wild, we present a novel background subtraction algorithm based on parallel vision and Bayesian generative adversarial networks (GANs). First, we use the median filtering algorithm for background image extraction. Then, we build the background subtraction model by using Bayesian GANs to classify all pixels into foreground and background, and use parallel vision theory to improve the background subtraction results in complex scenes. The proposed algorithm has been evaluated on the well-known, publicly available changedetection.net dataset. Experiment results show that the proposed algorithm results in better performance than many state-of-the-art ones. In addition, our model trained on CDnet dataset can generalize very well to unseen datasets, outperforming multiple state-of-art methods. The major contribution of this work is to apply parallel vision and Bayesian GANs to solve the difficulties in background subtraction, achieving high detection accuracy.
---
paper_title: Foreground Segmentation Using a Triplet Convolutional Neural Network for Multiscale Feature Encoding

paper_content:

A common approach for moving objects segmentation in a scene is to perform a background subtraction. Several methods have been proposed in this domain. However, they lack the ability of handling various difficult scenarios such as illumination changes, background or camera motion, camouflage effect, shadow etc. To address these issues, we propose a robust and flexible encoder-decoder type neural network based approach. We adapt a pre-trained convolutional network, i.e. VGG-16 Net, under a triplet framework in the encoder part to embed an image in multiple scales into the feature space and use a transposed convolutional network in the decoder part to learn a mapping from feature space to image space. We train this network end-to-end by using only a few training samples. Our network takes an RGB image in three different scales and produces a foreground segmentation probability mask for the corresponding image. In order to evaluate our model, we entered the Change Detection 2014 Challenge (changedetection.net) and our method outperformed all the existing state-of-the-art methods by an average F-Measure of 0.9770. Our source code will be made publicly available at https://github.com/lim-anggun/FgSegNet.
---
paper_title: CDnet 2014: An Expanded Change Detection Benchmark Dataset

paper_content:

Change detection is one of the most important lowlevel tasks in video analytics. In 2012, we introduced the changedetection.net (CDnet) benchmark, a video dataset devoted to the evalaution of change and motion detection approaches. Here, we present the latest release of the CDnet dataset, which includes 22 additional videos (70; 000 pixel-wise annotated frames) spanning 5 new categories that incorporate challenges encountered in many surveillance settings. We describe these categories in detail and provide an overview of the results of more than a dozen methods submitted to the IEEE Change DetectionWorkshop 2014. We highlight strengths and weaknesses of these methods and identify remaining issues in change detection.
---
paper_title: Background Subtraction on Depth Videos with Convolutional Neural Networks

paper_content:

Background subtraction is a significant component of computer vision systems. It is widely used in video surveillance, object tracking, anomaly detection, etc. A new data source for background subtraction appeared as the emergence of low-cost depth sensors like Microsof t Kinect, Asus Xtion PRO, etc. In this paper, we propose a background subtraction approach on depth videos, which is based on convolutional neural networks (CNNs), called BGSNet-D (BackGround Subtraction neural Networks for Depth videos). The method can be used in color unavailable scenarios like poor lighting situations, and can also be applied to combine with existing RGB background subtraction methods. A preprocessing strategy is designed to reduce the influences incurred by noise from depth sensors. The experimental results on the SBM-RGBD dataset show that the proposed method outperforms existing methods on depth data.
---
paper_title: Towards Benchmarking Scene Background Initialization

paper_content:

Given a set of images of a scene taken at different times, the availability of an initial background model that describes the scene without foreground objects is the prerequisite for a wide range of applications, ranging from video surveillance to computational photography. Even though several methods have been proposed for scene background initialization, the lack of a common groundtruthed dataset and of a common set of metrics makes it difficult to compare their performance. To move first steps towards an easy and fair comparison of these methods, we assembled a dataset of sequences frequently adopted for background initialization, selected or created ground truths for quantitative evaluation through a selected suite of metrics, and compared results obtained by some existing methods, making all the material publicly available.
---
paper_title: WeSamBE: A Weight-Sample-Based Method for Background Subtraction

paper_content:

Background subtraction techniques are often treated as fundamental and significant ways to analyze and understand video content. In this paper, we propose a weight-sample-based method for foreground detection. This method allows us to use a few samples with variable weights to achieve effective change detection. To rapidly adapt to changing scenarios, a minimum-weight update policy is first proposed to replace the most inefficient sample instead of the oldest sample or a random sample. In addition, a reward-and-penalty weighting strategy is put forward to reinforce active samples and punish others. In this way, the weights of relatively effective samples are increased and the false updating of effective samples with smaller weights is reduced. Moreover, some other strategies, such as spatial-diffusion policy and random time subsampling, are also incorporated to ensure the flexibility of the proposed method. Finally, in our experiments, an adaptive feedback technique is incorporated into our algorithm to adapt to more challenging videos, and the final results indicate that our method is superior to the state-of-the-art approaches on the challenging CDnet data set.
---
paper_title: Deep background subtraction with scene-specific convolutional neural networks

paper_content:

Background subtraction is usually based on low-level or hand-crafted features such as raw color components, gradients, or local binary patterns. As an improvement, we present a background subtraction algorithm based on spatial features learned with convolutional neural networks (ConvNets). Our algorithm uses a background model reduced to a single background image and a scene-specific training dataset to feed ConvNets that prove able to learn how to subtract the background from an input image patch. Experiments led on 2014 ChangeDetection.net dataset show that our ConvNet based algorithm at least reproduces the performance of state-of-the-art methods, and that it even outperforms them significantly when scene-specific knowledge is considered.
---
paper_title: Background subtraction in highly dynamic scenes

paper_content:

A new algorithm is proposed for background subtraction in highly dynamic scenes. Background subtraction is equated to the dual problem of saliency detection: background points are those considered not salient by suitable comparison of object and background appearance and dynamics. Drawing inspiration from biological vision, saliency is defined locally, using center-surround computations that measure local feature contrast. A discriminant formulation is adopted, where the saliency of a location is the discriminant power of a set of features with respect to the binary classification problem which opposes center to surround. To account for both motion and appearance, and achieve robustness to highly dynamic backgrounds, these features are spatiotemporal patches, which are modeled as dynamic textures. The resulting background subtraction algorithm is fully unsupervised, requires no training stage to learn background parameters, and depends only on the relative disparity of motion between the center and surround regions. This makes it insensitive to camera motion. The algorithm is tested on challenging video sequences, and shown to outperform various state-of-the-art techniques for background subtraction.
---
paper_title: Neural Graph Learning: Training Neural Networks Using Graphs

paper_content:

Label propagation is a powerful and flexible semi-supervised learning technique on graphs. Neural networks, on the other hand, have proven track records in many supervised learning tasks. In this work, we propose a training framework with a graph-regularised objective, namely Neural Graph Machines, that can combine the power of neural networks and label propagation. This work generalises previous literature on graph-augmented training of neural networks, enabling it to be applied to multiple neural architectures (Feed-forward NNs, CNNs and LSTM RNNs) and a wide range of graphs. The new objective allows the neural networks to harness both labeled and unlabeled data by: (a)~allowing the network to train using labeled data as in the supervised setting, (b)~biasing the network to learn similar hidden representations for neighboring nodes on a graph, in the same vein as label propagation. Such architectures with the proposed objective can be trained efficiently using stochastic gradient descent and scaled to large graphs, with a runtime that is linear in the number of edges. The proposed joint training approach convincingly outperforms many existing methods on a wide range of tasks (multi-label classification on social graphs, news categorization, document classification and semantic intent classification), with multiple forms of graph inputs (including graphs with and without node-level features) and using different types of neural networks.
---
paper_title: A deep convolutional neural network for video sequence background subtraction

paper_content:

We propose a novel approach based on deep learning for background subtraction from video sequences.A new algorithm to generate background model has been proposed.Input image patches and their corresponding background images are fed into CNN to do background subtraction.We utilized median filter to enhance the segmentation results.Experiments of Change detection results confirm the performance of the proposed approach. In this work, we present a novel background subtraction from video sequences algorithm that uses a deep Convolutional Neural Network (CNN) to perform the segmentation. With this approach, feature engineering and parameter tuning become unnecessary since the network parameters can be learned from data by training a single CNN that can handle various video scenes. Additionally, we propose a new approach to estimate background model from video sequences. For the training of the CNN, we employed randomly 5% video frames and their ground truth segmentations taken from the Change Detection challenge 2014 (CDnet 2014). We also utilized spatial-median filtering as the post-processing of the network outputs. Our method is evaluated with different data-sets, and it (so-called DeepBS) outperforms the existing algorithms with respect to the average ranking over different evaluation metrics announced in CDnet 2014. Furthermore, due to the network architecture, our CNN is capable of real time processing.
---
paper_title: Deep Learning in Visual Computing and Signal Processing

paper_content:

Deep learning is a subfield of machine learning, which aims to learn a hierarchy of features from input data. Nowadays, researchers have intensively investigated deep learning algorithms for solving challenging problems in many areas such as image classification, speech recognition, signal processing, and natural language processing. In this study, we not only review typical deep learning algorithms in computer vision and signal processing but also provide detailed information on how to apply deep learning to specific areas such as road crack detection, fault diagnosis, and human activity detection. Besides, this study also discusses the challenges of designing and training deep neural networks.
---
paper_title: Learning multi-scale features for foreground segmentation

paper_content:

Foreground segmentation algorithms aim at segmenting moving objects from the background in a robust way under various challenging scenarios. Encoder–decoder-type deep neural networks that are used in this domain recently perform impressive segmentation results. In this work, we propose a variation of our formerly proposed method (Anonymous 2018) that can be trained end-to-end using only a few training examples. The proposed method extends the feature pooling module of FgSegNet by introducing fusion of features inside this module, which is capable of extracting multi-scale features within images, resulting in a robust feature pooling against camera motion, which can alleviate the need of multi-scale inputs to the network. Sample visualizations highlight the regions in the images on which the model is specially focused. It can be seen that these regions are also the most semantically relevant. Our method outperforms all existing state-of-the-art methods in CDnet2014 datasets by an average overall F-measure of 0.9847. We also evaluate the effectiveness of our method on SBI2015 and UCSD Background Subtraction datasets. The source code of the proposed method is made available at https://github.com/lim-anggun/FgSegNet_v2.
---
paper_title: Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation

paper_content:

Abstract Conventional neural networks have been demonstrated to be a powerful framework for background subtraction in video acquired by static cameras. Indeed, the well-known Self-Organizing Background Subtraction (SOBS) method and its variants based on neural networks have long been the leading methods on the large-scale CDnet 2012 dataset during a long time. Convolutional neural networks, which are used in deep learning, have been recently and excessively employed for background initialization, foreground detection, and deep learned features. The top background subtraction methods currently used in CDnet 2014 are based on deep neural networks, and have demonstrated a large performance improvement in comparison to conventional unsupervised approaches based on multi-feature or multi-cue strategies. Furthermore, since the seminal work of Braham and Van Droogenbroeck in 2016, a large number of studies on convolutional neural networks applied to background subtraction have been published, and a continual gain of performance has been achieved. In this context, we provide the first review of deep neural network concepts in background subtraction for novices and experts in order to analyze this success and to provide further directions. To do so, we first surveyed the background initialization and background subtraction methods based on deep neural networks concepts, and also deep learned features. We then discuss the adequacy of deep neural networks for the task of background subtraction. Finally, experimental results are presented for the CDnet 2014 dataset.
---
paper_title: Very Deep Convolutional Networks for Large-Scale Image Recognition

paper_content:

In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.
---
paper_title: Interactive deep learning method for segmenting moving objects

paper_content:

The first machine learning method for ground truthing a video is proposed.We devise different sampling strategies for training different categories.Our results can be considered within the error margin of a human. With the increasing number of machine learning methods used for segmenting images and analyzing videos, there has been a growing need for large datasets with pixel accurate ground truth. In this letter, we propose a highly accurate semi-automatic method for segmenting foreground moving objects pictured in surveillance videos. Given a limited number of user interventions, the goal of the method is to provide results sufficiently accurate to be used as ground truth. In this paper, we show that by manually outlining a small number of moving objects, we can get our model to learn the appearance of the background and the foreground moving objects. Since the background and foreground moving objects are highly redundant from one image to another (videos come from surveillance cameras) the model does not need a large number of examples to accurately fit the data. Our end-to-end model is based on a multi-resolution convolutional neural network (CNN) with a cascaded architecture. Tests performed on the largest publicly-available video dataset with pixel accurate groundtruth (changdetection.net) reveal that on videos from 11 categories, our approach has an average F-measure of 0.95 which is within the error margin of a human being. With our model, the amount of manual work for ground truthing a video gets reduced by a factor of up to 40. Code is made publicly available at: https://github.com/zhimingluo/MovingObjectSegmentation
---
paper_title: Spatiotemporal GMM for Background Subtraction with Superpixel Hierarchy

paper_content:

We propose a background subtraction algorithm using hierarchical superpixel segmentation, spanning trees and optical flow. First, we generate superpixel segmentation trees using a number of Gaussian Mixture Models (GMMs) by treating each GMM as one vertex to construct spanning trees. Next, we use the $M$ -smoother to enhance the spatial consistency on the spanning trees and estimate optical flow to extend the $M$ -smoother to the temporal domain. Experimental results on synthetic and real-world benchmark datasets show that the proposed algorithm performs favorably for background subtraction in videos against the state-of-the-art methods in spite of frequent and sudden changes of pixel values.
---

Title: A Comprehensive Survey of Video Datasets for Background Subtraction

Section 1: INTRODUCTION
Description 1: Introduce the topic of automated video surveillance and the importance of background subtraction in intelligent video analytics. Discuss the challenges and the objectives of the survey.

Section 2: VIDEO DATASETS FOR BACKGROUND SUBTRACTION
Description 2: Provide an overview and detailed description of various video datasets released for evaluating background subtraction methods, categorized by small-scale and large-scale datasets.

Section 3: RGB-D VIDEO DATASETS FOR BACKGROUND SUBTRACTION
Description 3: Discuss the datasets that utilize RGB-D (color and depth) data for background subtraction, providing details on the specific challenges they address.

Section 4: VIDEO DATASETS FOR BACKGROUND INITIALIZATION
Description 4: Cover the datasets specifically designed to benchmark background initialization algorithms, detailing the typical challenges they include.

Section 5: DISCUSSION
Description 5: Summarize the key attributes of different video datasets, comparing them on various parameters and discussing the challenges they address.

Section 6: APPLICATION SPECIFIC VIDEO DATASETS
Description 6: Summarize video datasets that are designed for specific applications beyond general surveillance, such as underwater detection and animal monitoring.

Section 7: PERFORMANCE MEASURE IN BACKGROUND SUBTRACTION
Description 7: Outline the standard evaluation metrics used for assessing the quality of background subtraction algorithms in the literature.

Section 8: BACKGROUND SUBTRACTION LIBRARIES
Description 8: Discuss the libraries available for implementing background subtraction algorithms, describing their offerings and usage.

Section 9: CURRENT RESEARCH TRENDS AND FUTURE RESEARCH DIRECTIONS
Description 9: Highlight recent advancements in the field, especially the use of deep neural networks for background subtraction, and propose future research directions.

Section 10: CONCLUSION
Description 10: Provide concluding remarks on the state of video datasets for background subtraction and the need for further research and comprehensive datasets.