A survey of datasets for visual tracking
---
paper_title: The BEHAVE video dataset: ground truthed video for multi-person

paper_content:

Although there is much research on behaviour recognition in time-varying video, there are few ground truthed datasets for assessing multi-person behavioral interactions. This short paper presents the BEHAVE project’s dataset, which has around 90,000 frames of humans identified by bounding boxes, with interacting groups classified into one of 5 different behaviors. An example of its use is also presented.
---
paper_title: A Review of Visual Tracking

paper_content:

This report contains a review of visual tracking in monocular video sequences. For the purpose of this review, the majority of the visual trackers in the literature are divided into three tracking categories: discrete feature trackers, contour trackers, and region-based trackers. This categorization was performed based on the features used and the algorithms employed by the various visual trackers. The first class of trackers represents targets as discrete features (e.g. points, sets of points, lines) and performs data association using a distance metric that accommodates the particular feature. Contour trackers provide precise outlines of the target boundaries, meaning that they must not only uncover the position of the target, but its shape as well. Contour trackers often make use of gradient edge information during the tracking process. Region trackers represent the target with area-based descriptors that define its support and attempt to locate the image region in the current frame that best matches an object template. Trackers that are not in agreement with the abovementioned categorization, including those that combine methods from the three defined classes, are also considered in this review. In addition to categorizing and describing the various visual trackers in the literature, this review also provides a commentary on the current state of the field as well as a comparative analysis of the various approaches. The paper concludes with an outline of open problems in visual tracking.
---
paper_title: Online Object Tracking: A Benchmark

paper_content:

Object tracking is one of the most important components in numerous applications of computer vision. While much progress has been made in recent years with efforts on sharing code and datasets, it is of great importance to develop a library and benchmark to gauge the state of the art. After briefly reviewing recent advances of online object tracking, we carry out large scale experiments with various evaluation criteria to understand how these algorithms perform. The test image sequences are annotated with different attributes for performance evaluation and analysis. By analyzing quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field.
---
paper_title: A Review of Visual Tracking

paper_content:

This report contains a review of visual tracking in monocular video sequences. For the purpose of this review, the majority of the visual trackers in the literature are divided into three tracking categories: discrete feature trackers, contour trackers, and region-based trackers. This categorization was performed based on the features used and the algorithms employed by the various visual trackers. The first class of trackers represents targets as discrete features (e.g. points, sets of points, lines) and performs data association using a distance metric that accommodates the particular feature. Contour trackers provide precise outlines of the target boundaries, meaning that they must not only uncover the position of the target, but its shape as well. Contour trackers often make use of gradient edge information during the tracking process. Region trackers represent the target with area-based descriptors that define its support and attempt to locate the image region in the current frame that best matches an object template. Trackers that are not in agreement with the abovementioned categorization, including those that combine methods from the three defined classes, are also considered in this review. In addition to categorizing and describing the various visual trackers in the literature, this review also provides a commentary on the current state of the field as well as a comparative analysis of the various approaches. The paper concludes with an outline of open problems in visual tracking.
---
paper_title: Performance Evaluation Software: Moving Object Detection and Tracking in Videos

paper_content:

Performance Evaluation Software: Moving Object Detection and Tracking in Videos introduces a software approach for the real-time evaluation and performance comparison of the methods specializinginmoving object detection and/or tracking (D&T) in video processing. Digital video content analysis is an important item for multimedia content-based indexing (MCBI), content-based video retrieval (CBVR) and visual surveillance systems. There are some frequently-used generic algorithms for video object D&T in the literature, such as Background Subtraction (BS), Continuously Adaptive Mean-shift (CMS), Optical Flow (OF), etc. An important problem for performance evaluation is the absence of any stable and flexible software for comparison of different algorithms. In this frame, we have designed and implemented the software forcomparing andevaluating thewell-known video object D&T algorithms on the same platform. This software is able to compare them with the same metrics in real-time and on the same platform. It also works as an automatic and/or semi-automatic test environment in real-time, which uses the image and video processing essentials, e.g. morphological operations and filters, and ground-truth (GT) XML data files, charting/plotting capabilities, etc. Along with the comprehensive literature survey of the abovementioned video object D&T algorithms, this book also covers the technical details of our performance benchmark software as well as a case study on people D&T for the functionality of the software.
---
paper_title: Heuristic particle filter: applying abstraction techniques to the design of visual tracking algorithms

paper_content:

: Many real-world visual tracking applications have a high dimensionality, i.e. the system state is defined by a large number of variables. This kind of problem can be modelled as a dynamic optimization problem, which involves dynamic variables whose values change in time. Most applied research on optimization methods have focused on static optimization problems but these static methods often lack explicit adaptive methodologies. Heuristics are specific methods for solving problems in the absence of an algorithm for formal proof. Metaheuristics are approximate optimization methods which have been applied to more general problems with significant success. However, particle filters are Monte Carlo algorithms which solve the sequential estimation problem by approximating the theoretical distributions in the state space by simulated random measures called particles. However, particle filters lack efficient search strategies. In this paper, we propose a general framework to hybridize heuristics/metaheuristics with particle filters properly. The aim of this framework is to devise effective hybrid visual tracking algorithms naturally, guided by the use of abstraction techniques. Resulting algorithms exploit the benefits of both complementary approaches. As a particular example, a memetic algorithm particle filter is derived from the proposed hybridization framework. Finally, we show the performance of the memetic algorithm particle filter when it is applied to a multiple object tracking problem.
---
paper_title: The BEHAVE video dataset: ground truthed video for multi-person

paper_content:

Although there is much research on behaviour recognition in time-varying video, there are few ground truthed datasets for assessing multi-person behavioral interactions. This short paper presents the BEHAVE project’s dataset, which has around 90,000 frames of humans identified by bounding boxes, with interacting groups classified into one of 5 different behaviors. An example of its use is also presented.
---
paper_title: Visual tracking by fusing multiple cues with context-sensitive reliabilities

paper_content:

Many researchers argue that fusing multiple cues increases the reliability and robustness of visual tracking. However, how the multi-cue integration is realized during tracking is still an open issue. In this work, we present a novel data fusion approach for multi-cue tracking using particle filter. Our method differs from previous approaches in a number of ways. First, we carry out the integration of cues both in making predictions about the target object and in verifying them through observations. Our second and more significant contribution is that both stages of integration directly depend on the dynamically changing reliabilities of visual cues. These two aspects of our method allow the tracker to easily adapt itself to the changes in the context, and accordingly improve the tracking accuracy by resolving the ambiguities.
---
paper_title: Visual tracking using the Earth Mover's Distance between Gaussian mixtures and Kalman filtering

paper_content:

In this paper, we demonstrate how the differential Earth Mover's Distance (EMD) may be used for visual tracking in synergy with Gaussian mixtures models (GMM). According to our model, motion between adjacent frames results in variations of the mixing proportions of the Gaussian components representing the object to be tracked. These variations are computed in closed form by minimizing the differential EMD between Gaussian mixtures, yielding a very fast algorithm with high accuracy, without recurring to the EM algorithm in each frame. Moreover, we also propose a framework to handle occlusions, where the prediction for the object's location is forwarded to an adaptive Kalman filter whose parameters are estimated on line by the motion model already observed. Experimental results show significant improvement in tracking performance in the presence of occlusion.
---
paper_title: Online Object Tracking: A Benchmark

paper_content:

Object tracking is one of the most important components in numerous applications of computer vision. While much progress has been made in recent years with efforts on sharing code and datasets, it is of great importance to develop a library and benchmark to gauge the state of the art. After briefly reviewing recent advances of online object tracking, we carry out large scale experiments with various evaluation criteria to understand how these algorithms perform. The test image sequences are annotated with different attributes for performance evaluation and analysis. By analyzing quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field.
---
paper_title: Real time robust L1 tracker using accelerated proximal gradient approach

paper_content:

Recently sparse representation has been applied to visual tracker by modeling the target appearance using a sparse approximation over a template set, which leads to the so-called L1 trackers as it needs to solve an l 1 norm related minimization problem for many times. While these L1 trackers showed impressive tracking accuracies, they are very computationally demanding and the speed bottleneck is the solver to l 1 norm minimizations. This paper aims at developing an L1 tracker that not only runs in real time but also enjoys better robustness than other L1 trackers. In our proposed L1 tracker, a new l 1 norm related minimization model is proposed to improve the tracking accuracy by adding an l 1 norm regularization on the coefficients associated with the trivial templates. Moreover, based on the accelerated proximal gradient approach, a very fast numerical solver is developed to solve the resulting l 1 norm related minimization problem with guaranteed quadratic convergence. The great running time efficiency and tracking accuracy of the proposed tracker is validated with a comprehensive evaluation involving eight challenging sequences and five alternative state-of-the-art trackers.
---
paper_title: Visual tracking by proto-objects

paper_content:

In this paper, we propose a biologically inspired framework of visual tracking based on proto-objects. Given an image sequence, proto-objects are first detected by combining saliency map and topic model. Then the target is tracked based on spatial and saliency information of the proto-objects. In the proposed Bayesian approach, states of the target and proto-objects are jointly estimated over time. Gibbs sampling has been used to optimize the estimation during the tracking process. The proposed method robustly handles occlusion, distraction, and illumination change in the experiments. Experimental results also demonstrate that the proposed method outperforms the state-of-the-art methods in challenging tracking tasks.
---
paper_title: Robust visual tracking using discriminative stable regions and K-means clustering

paper_content:

This paper presents a method of extracting discriminative stable regions (DSRs) from image, and applies them for object tracking. These DSRs obtained by using the criterion of maximal entropy and spatial discrimination present high appearance stability and strong spatial discriminative power, which enables them to tolerate more appearance variations and to effectively resist spatial distracters. Meanwhile, the adaptive fusion tracking incorporated k-means clustering can handle severe occlusion as well as disturbance of motion noise during target localization. In addition, an effective local update scheme is designed to adapt to the object change for ensuring the tracking robustness. Experiments are carried out on several challenging sequences and results show that our method performs well in terms of object tracking, even in the presence of occlusion, deformation, illumination change, moving camera and spatial distracter.
---
paper_title: Dynamic appearance model for particle filter based visual tracking

paper_content:

This paper made major research on the target representation problem, which plays a significant role in visual tracking, but has received little attention in most researches. In order to fulfill the requirements of tracking robustness and effectiveness in practical conditions, a dynamic appearance model is constructed. Due to particle filter's excellent characteristics, it is employed in this paper not only to estimate target's state, but also to construct the dynamic observation model integrated by multiple cues. In the proposed method, a dynamic multi-cue integration model is constructed for particle filter framework. And a systematic study is done on evaluating cue's weight. Specially, a particle filter based weight tracker is designed to update multi-cue's integrating manner online, so as to adapt the observation model to target's appearance changes. In such a way, a double-particle-filter based tracking framework is formed, and it is field tested on a variety of videos in different tracking conditions. In the experiments and comparisons, the applicable conditions of the proposed dynamic model are discussed, and its robustness and effectiveness are demonstrated.
---
paper_title: Elliptical head tracking using intensity gradients and color histograms

paper_content:

An algorithm for tracking a person's head is presented. The head's projection onto the image plane is modeled as an ellipse whose position and size are continually updated by a local search combining the output of a module concentrating on the intensity gradient around the ellipse's perimeter with that of another module focusing on the color histogram of the ellipse's interior. Since these two modules have roughly orthogonal failure modes, they serve to complement one another. The result is a robust, real-time system that is able to track a person's head with enough accuracy to automatically control the camera's pan, tilt, and zoom in order to keep the person centered in the field of view at a desired size. Extensive experimentation shows the algorithm's robustness with respect to full 360-degree out-of-plane rotation, up to 90-degree tilting, severe but brief occlusion, arbitrary camera movement, and multiple moving people in the background.
---
paper_title: Locally Orderless Tracking

paper_content:

Locally Orderless Tracking (LOT) is a visual tracking algorithm that automatically estimates the amount of local (dis)order in the target. This lets the tracker specialize in both rigid and deformable objects on-line and with no prior assumptions. We provide a probabilistic model of the target variations over time. We then rigorously show that this model is a special case of the Earth Mover's Distance optimization problem where the ground distance is governed by some underlying noise model. This noise model has several parameters that control the cost of moving pixels and changing their color. We develop two such noise models and demonstrate how their parameters can be estimated on-line during tracking to account for the amount of local (dis)order in the target. We also discuss the significance of this on-line parameter update and demonstrate its contribution to the performance. Finally we show LOT's tracking capabilities on challenging video sequences, both commonly used and new, displaying performance comparable to state-of-the-art methods.
---
paper_title: Visual tracking with online Multiple Instance Learning

paper_content:

In this paper, we address the problem of learning an adaptive appearance model for object tracking. In particular, a class of tracking techniques called “tracking by detection” have been shown to give promising results at real-time speeds. These methods train a discriminative classifier in an online manner to separate the object from the background. This classifier bootstraps itself by using the current tracker state to extract positive and negative examples from the current frame. Slight inaccuracies in the tracker can therefore lead to incorrectly labeled training examples, which degrades the classifier and can cause further drift. In this paper we show that using Multiple Instance Learning (MIL) instead of traditional supervised learning avoids these problems, and can therefore lead to a more robust tracker with fewer parameter tweaks. We present a novel online MIL algorithm for object tracking that achieves superior results with real-time performance.
---
paper_title: Blurred target tracking by Blur-driven Tracker

paper_content:

Visual tracking plays an important role in many computer vision tasks. A common assumption in previous methods is that the video frames are blur free. In reality, motion blurs are pervasive in the real videos. In this paper we present a novel BLUr-driven Tracker (BLUT) framework for tracking motion-blurred targets. BLUT actively uses the information from blurs without performing debluring. Specifically, we integrate the tracking problem with the motion-from-blur problem under a unified sparse approximation framework. We further use the motion information inferred by blurs to guide the sampling process in the particle filter based tracking. To evaluate our method, we have collected a large number of video sequences with significatcant motion blurs and compared BLUT with state-of-the-art trackers. Experimental results show that, while many previous methods are sensitive to motion blurs, BLUT can robustly and reliably track severely blurred targets.
---
paper_title: Visual tracking in the presence of motion blur

paper_content:

We consider the problem of visual tracking of regions of interest in a sequence of motion blurred images. Traditional methods couple tracking with deblurring in order to correctly account for the effects of motion blur. Such coupling is usually appropriate, but computationally wasteful when visual tracking is the lone objective. Instead of deblurring images, we propose to match regions by blurring them. The matching score for two image regions is governed by a cost function that only involves the region deformation parameters and two motion blur vectors. We present an efficient algorithm to minimize the proposed cost function and demonstrate it on sequences of real blurred images.
---
paper_title: Boosting scalable gradient features for adaptive real-time tracking

paper_content:

Recently, several image gradient and edge based features have been introduced. In unison, they all discovered that object shape is a strong cue for recognition and tracking. Generally their basic feature extraction relies on pixel-wise gradient or edge computation using discrete filter masks, while scale invariance is later achieved by higher level operations like accumulating histograms or abstracting edgels to line segments. In this paper we show a novel and fast way to compute region based gradient features which are scale invariant themselves. We developed specialized, quick learnable weak classifiers that are integrated into our adaptively boosted observation model for particle filter based tracking. With an ensemble of region based gradient features this observation model is able to reliably capture the shape of the tracked object. The observation model is adapted to new object and background appearances while tracking. Thus we developed advanced methods to decide when to update the model, or in other words, if the filter is on target or not. We evaluated our approach using the BoBoT1 as well as the PROST2 datasets.
---
paper_title: Target Lock: robust real time adaptive visual tracker

paper_content:

Visual tracking is still an open problem because one needs to discriminate between the target object and background ::: under long duration. There is a major problem with conventional adaptive tracking where the target object is incorrectly ::: learnt (adapted) during runtime, resulting in poor performance of tracker. In this paper, we address this problem by ::: proposing validation-update strategy to minimize the error of false patches updating. The classifier we use is based on ::: boosted ensemble of Local Dominant Orientation (LDO). However, since LDO features contain binary values which are ::: unsuitable for classification, we have added a process to the online boosting learning algorithm that permits the two ::: binary values of "0" and "1". We elevate the tracker performance by pairing the classifier with normalized crosscorrelation ::: of patches tracked by Lukas-Kanade tracker. In the experiment conducted, we compare our method with two ::: other state-of-the-art adaptive trackers using BoBot dataset. Our method yields good tracking performance under variety ::: of scenarios set by BoBot dataset.
---
paper_title: Visual tracking using a pixelwise spatiotemporal oriented energy representation

paper_content:

This paper presents a novel pixelwise representation for visual tracking that models both the spatial structure and dynamics of a target in a unified fashion. The representation is derived from spatiotemporal energy measurements that capture underlying local spacetime orientation structure at multiple scales. For interframe motion estimation, the feature representation is instantiated within a pixelwise template warping framework; thus, the spatial arrangement of the pixelwise energy measurements remains intact. The proposed target representation is extremely rich, including appearance and motion information as well as information about how these descriptors are spatially arranged. Qualitative and quantitative empirical evaluation on challenging sequences demonstrates that the resulting tracker outperforms several alternative state-of-the-art systems.
---
paper_title: Online multiple support instance tracking

paper_content:

We propose an online tracking algorithm in which the support instances are selected adaptively within the multiple instance learning framework. The support instances are selected from training 1-norm support vector machines in a feature space, thereby learning large margin classifiers for visual tracking. An algorithm is presented to update the support instances by taking image data obtained previously and recently into account. In addition, a forgetting factor is introduced to weigh the contribution of support instances obtained at different time stamps. Experimental results demonstrate that our tracking algorithm is robust in handling occlusion, abrupt motion and illumination.
---
paper_title: Real time robust L1 tracker using accelerated proximal gradient approach

paper_content:

Recently sparse representation has been applied to visual tracker by modeling the target appearance using a sparse approximation over a template set, which leads to the so-called L1 trackers as it needs to solve an l 1 norm related minimization problem for many times. While these L1 trackers showed impressive tracking accuracies, they are very computationally demanding and the speed bottleneck is the solver to l 1 norm minimizations. This paper aims at developing an L1 tracker that not only runs in real time but also enjoys better robustness than other L1 trackers. In our proposed L1 tracker, a new l 1 norm related minimization model is proposed to improve the tracking accuracy by adding an l 1 norm regularization on the coefficients associated with the trivial templates. Moreover, based on the accelerated proximal gradient approach, a very fast numerical solver is developed to solve the resulting l 1 norm related minimization problem with guaranteed quadratic convergence. The great running time efficiency and tracking accuracy of the proposed tracker is validated with a comprehensive evaluation involving eight challenging sequences and five alternative state-of-the-art trackers.
---
paper_title: Visual tracking by proto-objects

paper_content:

In this paper, we propose a biologically inspired framework of visual tracking based on proto-objects. Given an image sequence, proto-objects are first detected by combining saliency map and topic model. Then the target is tracked based on spatial and saliency information of the proto-objects. In the proposed Bayesian approach, states of the target and proto-objects are jointly estimated over time. Gibbs sampling has been used to optimize the estimation during the tracking process. The proposed method robustly handles occlusion, distraction, and illumination change in the experiments. Experimental results also demonstrate that the proposed method outperforms the state-of-the-art methods in challenging tracking tasks.
---
paper_title: Visual tracking via adaptive structural local sparse appearance model

paper_content:

Sparse representation has been applied to visual tracking by finding the best candidate with minimal reconstruction error using target templates. However most sparse representation based trackers only consider the holistic representation and do not make full use of the sparse coefficients to discriminate between the target and the background, and hence may fail with more possibility when there is similar object or occlusion in the scene. In this paper we develop a simple yet robust tracking method based on the structural local sparse appearance model. This representation exploits both partial information and spatial information of the target based on a novel alignment-pooling method. The similarity obtained by pooling across the local patches helps not only locate the target more accurately but also handle occlusion. In addition, we employ a template update strategy which combines incremental subspace learning and sparse representation. This strategy adapts the template to the appearance change of the target with less possibility of drifting and reduces the influence of the occluded target template as well. Both qualitative and quantitative evaluations on challenging benchmark image sequences demonstrate that the proposed tracking algorithm performs favorably against several state-of-the-art methods.
---
paper_title: Robust object tracking via sparsity-based collaborative model

paper_content:

In this paper we propose a robust object tracking algorithm using a collaborative model. As the main challenge for object tracking is to account for drastic appearance change, we propose a robust appearance model that exploits both holistic templates and local representations. We develop a sparsity-based discriminative classifier (SD-C) and a sparsity-based generative model (SGM). In the S-DC module, we introduce an effective method to compute the confidence value that assigns more weights to the foreground than the background. In the SGM module, we propose a novel histogram-based method that takes the spatial information of each patch into consideration with an occlusion handing scheme. Furthermore, the update scheme considers both the latest observations and the original template, thereby enabling the tracker to deal with appearance change effectively and alleviate the drift problem. Numerous experiments on various challenging videos demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms.
---
paper_title: Game-theoretical occlusion handling for multi-target visual tracking

paper_content:

Multi-target visual tracking is a challenge because of data association and mutual occlusion in the interacting targets. This paper presents a Gaussian mixture probability hypothesis density based multi-target visual tracking system with game-theoretical occlusion handling. Firstly, the spatial constraint based appearance model with other interacting targets' interferences is modeled. Then, a two-step occlusion reasoning algorithm is proposed. Finally, an n-person, non-zero-sum, non-cooperative game is constructed to handle the mutual occlusion problem. The individual targets within the occlusion region are regarded as the players in the constructed game to compete for the maximum utilities by using the certain strategies. A Nash Equilibrium of the game is the optimal estimation of the locations of the players within the occlusion region. Experiments on video sequences demonstrate the good performance of the proposed occlusion handling algorithm.
---
paper_title: Adaptive earth movers distance-based Bayesian multi-target tracking

paper_content:

This study describes a complete system for multiple-target tracking in image sequences. The target appearance is represented as a set of weighted clusters in colour space. This is in contrast to the more typical use of colour histograms to model target appearance. The use of clusters allows a more flexible and accurate representation of the target, which demonstrates the benefits for tracking. However, it also introduces a number of computational difficulties, as calculating and matching cluster signatures are both computationally intensive tasks. To overcome this, the authors introduce a new formulation of incremental medoid-shift clustering that operates faster than mean shift in multi-target tracking scenarios. This matching scheme is integrated into a Bayesian tracking framework. Particle filters, a special case of Bayesian filters where the state variables are non-linear and non-Gaussian, are used in this study. An adaptive model update procedure is proposed for the cluster signature representation to handle target changes with time. The model update procedure is demonstrated to work successfully on a synthetic dataset and then on real datasets. Successful tracking results are shown on public datasets. Both qualitative and quantitative evaluations have been carried out to demonstrate the improved performance of the proposed multi-target tracking system. A higher tracking accuracy in long image sequences has been achieved compared to other standard tracking methods.
---
paper_title: Robust visual tracking with discriminative sparse learning

paper_content:

Recently, sparse representation in the task of visual tracking has been obtained increasing attention and many algorithms are proposed based on it. In these algorithms for visual tracking, each candidate target is sparsely represented by a set of target templates. However, these algorithms fail to consider the structural information of the space of the target templates, i.e., target template set. In this paper, we propose an algorithm named non-local self-similarity (NLSS) based sparse coding algorithm (NLSSC) to learn the sparse representations, which considers the geometrical structure of the set of target candidates. By using non-local self-similarity (NLSS) as a smooth operator, the proposed method can turn the tracking into sparse representations problems, in which the information of the set of target candidates is exploited. Extensive experimental results on visual tracking have demonstrated the effectiveness of the proposed algorithm.
---
paper_title: Locally Orderless Tracking

paper_content:

Locally Orderless Tracking (LOT) is a visual tracking algorithm that automatically estimates the amount of local (dis)order in the target. This lets the tracker specialize in both rigid and deformable objects on-line and with no prior assumptions. We provide a probabilistic model of the target variations over time. We then rigorously show that this model is a special case of the Earth Mover's Distance optimization problem where the ground distance is governed by some underlying noise model. This noise model has several parameters that control the cost of moving pixels and changing their color. We develop two such noise models and demonstrate how their parameters can be estimated on-line during tracking to account for the amount of local (dis)order in the target. We also discuss the significance of this on-line parameter update and demonstrate its contribution to the performance. Finally we show LOT's tracking capabilities on challenging video sequences, both commonly used and new, displaying performance comparable to state-of-the-art methods.
---
paper_title: Minimum Uncertainty Gap for Robust Visual Tracking

paper_content:

We propose a novel tracking algorithm that robustly tracks the target by finding the state which minimizes uncertainty of the likelihood at current state. The uncertainty of the likelihood is estimated by obtaining the gap between the lower and upper bounds of the likelihood. By minimizing the gap between the two bounds, our method finds the confident and reliable state of the target. In the paper, the state that gives the Minimum Uncertainty Gap (MUG) between likelihood bounds is shown to be more reliable than the state which gives the maximum likelihood only, especially when there are severe illumination changes, occlusions, and pose variations. A rigorous derivation of the lower and upper bounds of the likelihood for the visual tracking problem is provided to address this issue. Additionally, an efficient inference algorithm using Interacting Markov Chain Monte Carlo is presented to find the best state that maximizes the average of the lower and upper bounds of the likelihood and minimizes the gap between two bounds simultaneously. Experimental results demonstrate that our method successfully tracks the target in realistic videos and outperforms conventional tracking methods.
---
paper_title: Robust Visual Tracking Using an Adaptive Coupled-Layer Visual Model

paper_content:

This paper addresses the problem of tracking objects which undergo rapid and significant appearance changes. We propose a novel coupled-layer visual model that combines the target's global and local appearance by interlacing two layers. The local layer in this model is a set of local patches that geometrically constrain the changes in the target's appearance. This layer probabilistically adapts to the target's geometric deformation, while its structure is updated by removing and adding the local patches. The addition of these patches is constrained by the global layer that probabilistically models the target's global visual properties, such as color, shape, and apparent local motion. The global visual properties are updated during tracking using the stable patches from the local layer. By this coupled constraint paradigm between the adaptation of the global and the local layer, we achieve a more robust tracking through significant appearance changes. We experimentally compare our tracker to 11 state-of-the-art trackers. The experimental results on challenging sequences confirm that our tracker outperforms the related trackers in many cases by having a smaller failure rate as well as better accuracy. Furthermore, the parameter analysis shows that our tracker is stable over a range of parameter values.
---
paper_title: Linear Regression and Adaptive Appearance Models for Fast Simultaneous Modelling and Tracking

paper_content:

This work proposes an approach to tracking by regression that uses no hard-coded models and no offline learning stage. The Linear Predictor (LP) tracker has been shown to be highly computationally efficient, resulting in fast tracking. Regression tracking techniques tend to require offline learning to learn suitable regression functions. This work removes the need for offline learning and therefore increases the applicability of the technique. The online-LP tracker can simply be seeded with an initial target location, akin to the ubiquitous Lucas-Kanade algorithm that tracks by registering an image template via minimisation. ::: ::: A fundamental issue for all trackers is the representation of the target appearance and how this representation is able to adapt to changes in target appearance over time. The two proposed methods, LP-SMAT and LP-MED, demonstrate the ability to adapt to large appearance variations by incrementally building an appearance model that identifies modes or aspects of the target appearance and associates these aspects to the Linear Predictor trackers to which they are best suited. Experiments comparing and evaluating regression and registration techniques are presented along with performance evaluations favourably comparing the proposed tracker and appearance model learning methods to other state of the art simultaneous modelling and tracking approaches.
---
paper_title: Online Learning and Partitioning of Linear Displacement Predictors for Tracking

paper_content:

A novel approach to learning and tracking arbitrary image features is presented. Tracking is tackled by learning the mapping from image intensity differences to displacements. Linear regression is used, resulting in low computational cost. An appearance model of the target is built on-the-fly by clustering sub-sampled image templates. The medoidshift algorithm is used to cluster the templates thus identifying various modes or aspects of the target appearance, each mode is associated to the most suitable set of linear predictors allowing piecewise linear regression from image intensity differences to warp updates. Despite no hard-coding or offline learning, excellent results are shown on three publicly available video sequences and comparisons with related approaches made.
---
paper_title: Visual tracking via adaptive structural local sparse appearance model

paper_content:

Sparse representation has been applied to visual tracking by finding the best candidate with minimal reconstruction error using target templates. However most sparse representation based trackers only consider the holistic representation and do not make full use of the sparse coefficients to discriminate between the target and the background, and hence may fail with more possibility when there is similar object or occlusion in the scene. In this paper we develop a simple yet robust tracking method based on the structural local sparse appearance model. This representation exploits both partial information and spatial information of the target based on a novel alignment-pooling method. The similarity obtained by pooling across the local patches helps not only locate the target more accurately but also handle occlusion. In addition, we employ a template update strategy which combines incremental subspace learning and sparse representation. This strategy adapts the template to the appearance change of the target with less possibility of drifting and reduces the influence of the occluded target template as well. Both qualitative and quantitative evaluations on challenging benchmark image sequences demonstrate that the proposed tracking algorithm performs favorably against several state-of-the-art methods.
---
paper_title: Minimum Uncertainty Gap for Robust Visual Tracking

paper_content:

We propose a novel tracking algorithm that robustly tracks the target by finding the state which minimizes uncertainty of the likelihood at current state. The uncertainty of the likelihood is estimated by obtaining the gap between the lower and upper bounds of the likelihood. By minimizing the gap between the two bounds, our method finds the confident and reliable state of the target. In the paper, the state that gives the Minimum Uncertainty Gap (MUG) between likelihood bounds is shown to be more reliable than the state which gives the maximum likelihood only, especially when there are severe illumination changes, occlusions, and pose variations. A rigorous derivation of the lower and upper bounds of the likelihood for the visual tracking problem is provided to address this issue. Additionally, an efficient inference algorithm using Interacting Markov Chain Monte Carlo is presented to find the best state that maximizes the average of the lower and upper bounds of the likelihood and minimizes the gap between two bounds simultaneously. Experimental results demonstrate that our method successfully tracks the target in realistic videos and outperforms conventional tracking methods.
---
paper_title: Superpixel tracking

paper_content:

While numerous algorithms have been proposed for object tracking with demonstrated success, it remains a challenging problem for a tracker to handle large change in scale, motion, shape deformation with occlusion. One of the main reasons is the lack of effective image representation to account for appearance variation. Most trackers use high-level appearance structure or low-level cues for representing and matching target objects. In this paper, we propose a tracking method from the perspective of mid-level vision with structural information captured in superpixels. We present a discriminative appearance model based on superpixels, thereby facilitating a tracker to distinguish the target and the background with mid-level cues. The tracking task is then formulated by computing a target-background confidence map, and obtaining the best candidate by maximum a posterior estimate. Experimental results demonstrate that our tracker is able to handle heavy occlusion and recover from drifts. In conjunction with online update, the proposed algorithm is shown to perform favorably against existing methods for object tracking.
---
paper_title: Visual tracking with online Multiple Instance Learning

paper_content:

In this paper, we address the problem of learning an adaptive appearance model for object tracking. In particular, a class of tracking techniques called “tracking by detection” have been shown to give promising results at real-time speeds. These methods train a discriminative classifier in an online manner to separate the object from the background. This classifier bootstraps itself by using the current tracker state to extract positive and negative examples from the current frame. Slight inaccuracies in the tracker can therefore lead to incorrectly labeled training examples, which degrades the classifier and can cause further drift. In this paper we show that using Multiple Instance Learning (MIL) instead of traditional supervised learning avoids these problems, and can therefore lead to a more robust tracker with fewer parameter tweaks. We present a novel online MIL algorithm for object tracking that achieves superior results with real-time performance.
---
paper_title: Object tracking using firefly algorithm

paper_content:

Firefly algorithm (FA) is a new meta-heuristic optimisation algorithm that mimics the social behaviour of fireflies flying in the tropical and temperate summer sky. In this study, a novel application of FA is presented as it is applied to solve tracking problem. A general optimisation-based tracking architecture is proposed and the parameters' sensitivity and adjustment of the FA in tracking system are studied. Experimental results show that the FA-based tracker can robustly track an arbitrary target in various challenging conditions. The authors compare the speed and accuracy of the FA with three typical tracking algorithms including the particle filter, meanshift and particle swarm optimisation. Comparative results show that the FA-based tracker outperforms the other three trackers.
---
paper_title: Real-time visual tracking via online weighted multiple instance learning

paper_content:

Adaptive tracking-by-detection methods have been widely studied with promising results. These methods first train a classifier in an online manner. Then, a sliding window is used to extract some samples from the local regions surrounding the former object location at the new frame. The classifier is then applied to these samples where the location of sample with maximum classifier score is the new object location. However, such classifier may be inaccurate when the training samples are imprecise which causes drift. Multiple instance learning (MIL) method is recently introduced into the tracking task, which can alleviate drift to some extent. However, the MIL tracker may detect the positive sample that is less important because it does not discriminatively consider the sample importance in its learning procedure. In this paper, we present a novel online weighted MIL (WMIL) tracker. The WMIL tracker integrates the sample importance into an efficient online learning procedure by assuming the most important sample (i.e., the tracking result in current frame) is known when training the classifier. A new bag probability function combining the weighted instance probability is proposed via which the sample importance is considered. Then, an efficient online approach is proposed to approximately maximize the bag likelihood function, leading to a more robust and much faster tracker. Experimental results on various benchmark video sequences demonstrate the superior performance of our algorithm to state-of-the-art tracking algorithms.
---
paper_title: Robust object tracking via sparsity-based collaborative model

paper_content:

In this paper we propose a robust object tracking algorithm using a collaborative model. As the main challenge for object tracking is to account for drastic appearance change, we propose a robust appearance model that exploits both holistic templates and local representations. We develop a sparsity-based discriminative classifier (SD-C) and a sparsity-based generative model (SGM). In the S-DC module, we introduce an effective method to compute the confidence value that assigns more weights to the foreground than the background. In the SGM module, we propose a novel histogram-based method that takes the spatial information of each patch into consideration with an occlusion handing scheme. Furthermore, the update scheme considers both the latest observations and the original template, thereby enabling the tracker to deal with appearance change effectively and alleviate the drift problem. Numerous experiments on various challenging videos demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms.
---
paper_title: Visual tracking decomposition

paper_content:

We propose a novel tracking algorithm that can work robustly in a challenging scenario such that several kinds of appearance and motion changes of an object occur at the same time. Our algorithm is based on a visual tracking decomposition scheme for the efficient design of observation and motion models as well as trackers. In our scheme, the observation model is decomposed into multiple basic observation models that are constructed by sparse principal component analysis (SPCA) of a set of feature templates. Each basic observation model covers a specific appearance of the object. The motion model is also represented by the combination of multiple basic motion models, each of which covers a different type of motion. Then the multiple basic trackers are designed by associating the basic observation models and the basic motion models, so that each specific tracker takes charge of a certain change in the object. All basic trackers are then integrated into one compound tracker through an interactive Markov Chain Monte Carlo (IMCMC) framework in which the basic trackers communicate with one another interactively while run in parallel. By exchanging information with others, each tracker further improves its performance, which results in increasing the whole performance of tracking. Experimental results show that our method tracks the object accurately and reliably in realistic videos where the appearance and motion are drastically changing over time.
---
paper_title: Visual tracking by proto-objects

paper_content:

In this paper, we propose a biologically inspired framework of visual tracking based on proto-objects. Given an image sequence, proto-objects are first detected by combining saliency map and topic model. Then the target is tracked based on spatial and saliency information of the proto-objects. In the proposed Bayesian approach, states of the target and proto-objects are jointly estimated over time. Gibbs sampling has been used to optimize the estimation during the tracking process. The proposed method robustly handles occlusion, distraction, and illumination change in the experiments. Experimental results also demonstrate that the proposed method outperforms the state-of-the-art methods in challenging tracking tasks.
---
paper_title: Tracking by Sampling Trackers

paper_content:

We propose a novel tracking framework called visual tracker sampler that tracks a target robustly by searching for the appropriate trackers in each frame. Since the real-world tracking environment varies severely over time, the trackers should be adapted or newly constructed depending on the current situation. To do this, our method obtains several samples of not only the states of the target but also the trackers themselves during the sampling process. The trackers are efficiently sampled using the Markov Chain Monte Carlo method from the predefined tracker space by proposing new appearance models, motion models, state representation types, and observation types, which are the basic important components of visual trackers. Then, the sampled trackers run in parallel and interact with each other while covering various target variations efficiently. The experiment demonstrates that our method tracks targets accurately and robustly in the real-world tracking environments and outperforms the state-of-the-art tracking methods.
---
paper_title: Robust object tracking via sparsity-based collaborative model

paper_content:

In this paper we propose a robust object tracking algorithm using a collaborative model. As the main challenge for object tracking is to account for drastic appearance change, we propose a robust appearance model that exploits both holistic templates and local representations. We develop a sparsity-based discriminative classifier (SD-C) and a sparsity-based generative model (SGM). In the S-DC module, we introduce an effective method to compute the confidence value that assigns more weights to the foreground than the background. In the SGM module, we propose a novel histogram-based method that takes the spatial information of each patch into consideration with an occlusion handing scheme. Furthermore, the update scheme considers both the latest observations and the original template, thereby enabling the tracker to deal with appearance change effectively and alleviate the drift problem. Numerous experiments on various challenging videos demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms.
---
paper_title: Structure-aware keypoint tracking for partial occlusion handling

paper_content:

This paper introduces a novel keypoint-based method for visual object tracking. To represent the target, we use a new model combining color distribution with keypoints. The appearance model also incorporates the spatial layout of the keypoints, encoding the object structure learned during tracking. With this multi-feature appearance model, our Structure-Aware Tracker (SAT) estimates accurately the target location using three main steps. First, the search space is reduced to the most likely image regions with a probabilistic approach. Second, the target location is estimated in the reduced search space using deterministic keypoint matching. Finally, the location prediction is corrected by exploiting the keypoint structural model with a voting-based method. By applying our SAT on several tracking problems, we show that location correction based on structural constraints is a key technique to improve prediction in moderately crowded scenes, even if only a small part of the target is visible. We also conduct comparison with a number of state-of-the-art trackers and demonstrate the competitiveness of the proposed method.
---
paper_title: Online visual tracking with histograms and articulating blocks

paper_content:

We propose an algorithm for accurate tracking of articulated objects using online update of appearance and shape. The challenge here is to model foreground appearance with histograms in a way that is both efficient and accurate. In this algorithm, the constantly changing foreground shape is modeled as a small number of rectangular blocks, whose positions within the tracking window are adaptively determined. Under the general assumption of stationary foreground appearance, we show that robust object tracking is possible by adaptively adjusting the locations of these blocks. Implemented in MATLAB without substantial optimization, our tracker runs already at 3.7 frames per second on a 3GHz machine. Experimental results have demonstrated that the algorithm is able to efficiently track articulated objects undergoing large variation in appearance and shape.
---
paper_title: Wang-Landau Monte Carlo-Based Tracking Methods for Abrupt Motions

paper_content:

We propose a novel tracking algorithm based on the Wang-Landau Monte Carlo (WLMC) sampling method for dealing with abrupt motions efficiently. Abrupt motions cause conventional tracking methods to fail because they violate the motion smoothness constraint. To address this problem, we introduce the Wang-Landau sampling method and integrate it into a Markov Chain Monte Carlo (MCMC)-based tracking framework. By employing the novel density-of-states term estimated by the Wang-Landau sampling method into the acceptance ratio of MCMC, our WLMC-based tracking method alleviates the motion smoothness constraint and robustly tracks the abrupt motions. Meanwhile, the marginal likelihood term of the acceptance ratio preserves the accuracy in tracking smooth motions. The method is then extended to obtain good performance in terms of scalability, even on a high-dimensional state space. Hence, it covers drastic changes in not only position but also scale of a target. To achieve this, we modify our method by combining it with the N-fold way algorithm and present the N-Fold Wang-Landau (NFWL)-based tracking method. The N-fold way algorithm helps estimate the density-of-states with a smaller number of samples. Experimental results demonstrate that our approach efficiently samples the states of the target, even in a whole state space, without loss of time, and tracks the target accurately and robustly when position and scale are changing severely.
---
paper_title: Real time robust L1 tracker using accelerated proximal gradient approach

paper_content:

Recently sparse representation has been applied to visual tracker by modeling the target appearance using a sparse approximation over a template set, which leads to the so-called L1 trackers as it needs to solve an l 1 norm related minimization problem for many times. While these L1 trackers showed impressive tracking accuracies, they are very computationally demanding and the speed bottleneck is the solver to l 1 norm minimizations. This paper aims at developing an L1 tracker that not only runs in real time but also enjoys better robustness than other L1 trackers. In our proposed L1 tracker, a new l 1 norm related minimization model is proposed to improve the tracking accuracy by adding an l 1 norm regularization on the coefficients associated with the trivial templates. Moreover, based on the accelerated proximal gradient approach, a very fast numerical solver is developed to solve the resulting l 1 norm related minimization problem with guaranteed quadratic convergence. The great running time efficiency and tracking accuracy of the proposed tracker is validated with a comprehensive evaluation involving eight challenging sequences and five alternative state-of-the-art trackers.
---
paper_title: Game-theoretical occlusion handling for multi-target visual tracking

paper_content:

Multi-target visual tracking is a challenge because of data association and mutual occlusion in the interacting targets. This paper presents a Gaussian mixture probability hypothesis density based multi-target visual tracking system with game-theoretical occlusion handling. Firstly, the spatial constraint based appearance model with other interacting targets' interferences is modeled. Then, a two-step occlusion reasoning algorithm is proposed. Finally, an n-person, non-zero-sum, non-cooperative game is constructed to handle the mutual occlusion problem. The individual targets within the occlusion region are regarded as the players in the constructed game to compete for the maximum utilities by using the certain strategies. A Nash Equilibrium of the game is the optimal estimation of the locations of the players within the occlusion region. Experiments on video sequences demonstrate the good performance of the proposed occlusion handling algorithm.
---
paper_title: Robust visual tracking with discriminative sparse learning

paper_content:

Recently, sparse representation in the task of visual tracking has been obtained increasing attention and many algorithms are proposed based on it. In these algorithms for visual tracking, each candidate target is sparsely represented by a set of target templates. However, these algorithms fail to consider the structural information of the space of the target templates, i.e., target template set. In this paper, we propose an algorithm named non-local self-similarity (NLSS) based sparse coding algorithm (NLSSC) to learn the sparse representations, which considers the geometrical structure of the set of target candidates. By using non-local self-similarity (NLSS) as a smooth operator, the proposed method can turn the tracking into sparse representations problems, in which the information of the set of target candidates is exploited. Extensive experimental results on visual tracking have demonstrated the effectiveness of the proposed algorithm.
---
paper_title: Locally Orderless Tracking

paper_content:

Locally Orderless Tracking (LOT) is a visual tracking algorithm that automatically estimates the amount of local (dis)order in the target. This lets the tracker specialize in both rigid and deformable objects on-line and with no prior assumptions. We provide a probabilistic model of the target variations over time. We then rigorously show that this model is a special case of the Earth Mover's Distance optimization problem where the ground distance is governed by some underlying noise model. This noise model has several parameters that control the cost of moving pixels and changing their color. We develop two such noise models and demonstrate how their parameters can be estimated on-line during tracking to account for the amount of local (dis)order in the target. We also discuss the significance of this on-line parameter update and demonstrate its contribution to the performance. Finally we show LOT's tracking capabilities on challenging video sequences, both commonly used and new, displaying performance comparable to state-of-the-art methods.
---
paper_title: Robust Fragments-based Tracking using the Integral Histogram

paper_content:

We present a novel algorithm (which we call "Frag- Track") for tracking an object in a video sequence. The template object is represented by multiple image fragments or patches. The patches are arbitrary and are not based on an object model (in contrast with traditional use of modelbased parts e.g. limbs and torso in human tracking). Every patch votes on the possible positions and scales of the object in the current frame, by comparing its histogram with the corresponding image patch histogram. We then minimize a robust statistic in order to combine the vote maps of the multiple patches. A key tool enabling the application of our algorithm to tracking is the integral histogram data structure [18]. Its use allows to extract histograms of multiple rectangular regions in the image in a very efficient manner. Our algorithm overcomes several difficulties which cannot be handled by traditional histogram-based algorithms [8, 6]. First, by robustly combining multiple patch votes, we are able to handle partial occlusions or pose change. Second, the geometric relations between the template patches allow us to take into account the spatial distribution of the pixel intensities - information which is lost in traditional histogram-based algorithms. Third, as noted by [18], tracking large targets has the same computational cost as tracking small targets. We present extensive experimental results on challenging sequences, which demonstrate the robust tracking achieved by our algorithm (even with the use of only gray-scale (noncolor) information).
---
paper_title: Boosting scalable gradient features for adaptive real-time tracking

paper_content:

Recently, several image gradient and edge based features have been introduced. In unison, they all discovered that object shape is a strong cue for recognition and tracking. Generally their basic feature extraction relies on pixel-wise gradient or edge computation using discrete filter masks, while scale invariance is later achieved by higher level operations like accumulating histograms or abstracting edgels to line segments. In this paper we show a novel and fast way to compute region based gradient features which are scale invariant themselves. We developed specialized, quick learnable weak classifiers that are integrated into our adaptively boosted observation model for particle filter based tracking. With an ensemble of region based gradient features this observation model is able to reliably capture the shape of the tracked object. The observation model is adapted to new object and background appearances while tracking. Thus we developed advanced methods to decide when to update the model, or in other words, if the filter is on target or not. We evaluated our approach using the BoBoT1 as well as the PROST2 datasets.
---
paper_title: Minimum Uncertainty Gap for Robust Visual Tracking

paper_content:

We propose a novel tracking algorithm that robustly tracks the target by finding the state which minimizes uncertainty of the likelihood at current state. The uncertainty of the likelihood is estimated by obtaining the gap between the lower and upper bounds of the likelihood. By minimizing the gap between the two bounds, our method finds the confident and reliable state of the target. In the paper, the state that gives the Minimum Uncertainty Gap (MUG) between likelihood bounds is shown to be more reliable than the state which gives the maximum likelihood only, especially when there are severe illumination changes, occlusions, and pose variations. A rigorous derivation of the lower and upper bounds of the likelihood for the visual tracking problem is provided to address this issue. Additionally, an efficient inference algorithm using Interacting Markov Chain Monte Carlo is presented to find the best state that maximizes the average of the lower and upper bounds of the likelihood and minimizes the gap between two bounds simultaneously. Experimental results demonstrate that our method successfully tracks the target in realistic videos and outperforms conventional tracking methods.
---
paper_title: PROST: Parallel robust online simple tracking

paper_content:

Tracking-by-detection is increasingly popular in order to tackle the visual tracking problem. Existing adaptive methods suffer from the drifting problem, since they rely on self-updates of an on-line learning method. In contrast to previous work that tackled this problem by employing semi-supervised or multiple-instance learning, we show that augmenting an on-line learning method with complementary tracking approaches can lead to more stable results. In particular, we use a simple template model as a non-adaptive and thus stable component, a novel optical-flow-based mean-shift tracker as highly adaptive element and an on-line random forest as moderately adaptive appearance-based learner. We combine these three trackers in a cascade. All of our components run on GPUs or similar multi-core systems, which allows for real-time performance. We show the superiority of our system over current state-of-the-art tracking methods in several experiments on publicly available data.
---
paper_title: Visual tracking with online Multiple Instance Learning

paper_content:

In this paper, we address the problem of learning an adaptive appearance model for object tracking. In particular, a class of tracking techniques called “tracking by detection” have been shown to give promising results at real-time speeds. These methods train a discriminative classifier in an online manner to separate the object from the background. This classifier bootstraps itself by using the current tracker state to extract positive and negative examples from the current frame. Slight inaccuracies in the tracker can therefore lead to incorrectly labeled training examples, which degrades the classifier and can cause further drift. In this paper we show that using Multiple Instance Learning (MIL) instead of traditional supervised learning avoids these problems, and can therefore lead to a more robust tracker with fewer parameter tweaks. We present a novel online MIL algorithm for object tracking that achieves superior results with real-time performance.
---
paper_title: Real time robust L1 tracker using accelerated proximal gradient approach

paper_content:

Recently sparse representation has been applied to visual tracker by modeling the target appearance using a sparse approximation over a template set, which leads to the so-called L1 trackers as it needs to solve an l 1 norm related minimization problem for many times. While these L1 trackers showed impressive tracking accuracies, they are very computationally demanding and the speed bottleneck is the solver to l 1 norm minimizations. This paper aims at developing an L1 tracker that not only runs in real time but also enjoys better robustness than other L1 trackers. In our proposed L1 tracker, a new l 1 norm related minimization model is proposed to improve the tracking accuracy by adding an l 1 norm regularization on the coefficients associated with the trivial templates. Moreover, based on the accelerated proximal gradient approach, a very fast numerical solver is developed to solve the resulting l 1 norm related minimization problem with guaranteed quadratic convergence. The great running time efficiency and tracking accuracy of the proposed tracker is validated with a comprehensive evaluation involving eight challenging sequences and five alternative state-of-the-art trackers.
---
paper_title: Robust visual tracking using discriminative stable regions and K-means clustering

paper_content:

This paper presents a method of extracting discriminative stable regions (DSRs) from image, and applies them for object tracking. These DSRs obtained by using the criterion of maximal entropy and spatial discrimination present high appearance stability and strong spatial discriminative power, which enables them to tolerate more appearance variations and to effectively resist spatial distracters. Meanwhile, the adaptive fusion tracking incorporated k-means clustering can handle severe occlusion as well as disturbance of motion noise during target localization. In addition, an effective local update scheme is designed to adapt to the object change for ensuring the tracking robustness. Experiments are carried out on several challenging sequences and results show that our method performs well in terms of object tracking, even in the presence of occlusion, deformation, illumination change, moving camera and spatial distracter.
---
paper_title: Struck: Structured Output Tracking with Kernels

paper_content:

Adaptive tracking-by-detection methods are widely used in computer vision for tracking arbitrary objects. Current approaches treat the tracking problem as a classification task and use online learning techniques to update the object model. However, for these updates to happen one needs to convert the estimated object position into a set of labelled training examples, and it is not clear how best to perform this intermediate step. Furthermore, the objective for the classifier (label prediction) is not explicitly coupled to the objective for the tracker (estimation of object position). In this paper, we present a framework for adaptive visual object tracking based on structured output prediction. By explicitly allowing the output space to express the needs of the tracker, we avoid the need for an intermediate classification step. Our method uses a kernelised structured output support vector machine (SVM), which is learned online to provide adaptive tracking. To allow our tracker to run at high frame rates, we (a) introduce a budgeting mechanism that prevents the unbounded growth in the number of support vectors that would otherwise occur during tracking, and (b) show how to implement tracking on the GPU. Experimentally, we show that our algorithm is able to outperform state-of-the-art trackers on various benchmark videos. Additionally, we show that we can easily incorporate additional features and kernels into our framework, which results in increased tracking performance.
---
paper_title: Visual tracking via adaptive structural local sparse appearance model

paper_content:

Sparse representation has been applied to visual tracking by finding the best candidate with minimal reconstruction error using target templates. However most sparse representation based trackers only consider the holistic representation and do not make full use of the sparse coefficients to discriminate between the target and the background, and hence may fail with more possibility when there is similar object or occlusion in the scene. In this paper we develop a simple yet robust tracking method based on the structural local sparse appearance model. This representation exploits both partial information and spatial information of the target based on a novel alignment-pooling method. The similarity obtained by pooling across the local patches helps not only locate the target more accurately but also handle occlusion. In addition, we employ a template update strategy which combines incremental subspace learning and sparse representation. This strategy adapts the template to the appearance change of the target with less possibility of drifting and reduces the influence of the occluded target template as well. Both qualitative and quantitative evaluations on challenging benchmark image sequences demonstrate that the proposed tracking algorithm performs favorably against several state-of-the-art methods.
---
paper_title: Visual tracking and learning using speeded up robust features

paper_content:

A speeded up robust features (SURF) based optical flow algorithm is presented for visual tracking in real scenarios. SURF construct invariant features to correspond the blobs of interest across frames. Meanwhile, new feature-based optical flow algorithm is used to compute the warp matrix of a region centered on SURF key points. Furthermore, on-line visual learning for long-term tracking is performed using incremental object subspace method, which includes the correct update of the sample mean and appearance model. The proposed SURF based tracking and learning method contributes measurably to improving overall tracking performance. Experimental work demonstrates that the proposed strategy improves the performance of the classical optical flow algorithms in complicated real scenarios.
---
paper_title: Robust object tracking via sparsity-based collaborative model

paper_content:

In this paper we propose a robust object tracking algorithm using a collaborative model. As the main challenge for object tracking is to account for drastic appearance change, we propose a robust appearance model that exploits both holistic templates and local representations. We develop a sparsity-based discriminative classifier (SD-C) and a sparsity-based generative model (SGM). In the S-DC module, we introduce an effective method to compute the confidence value that assigns more weights to the foreground than the background. In the SGM module, we propose a novel histogram-based method that takes the spatial information of each patch into consideration with an occlusion handing scheme. Furthermore, the update scheme considers both the latest observations and the original template, thereby enabling the tracker to deal with appearance change effectively and alleviate the drift problem. Numerous experiments on various challenging videos demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms.
---
paper_title: Robust visual tracking with discriminative sparse learning

paper_content:

Recently, sparse representation in the task of visual tracking has been obtained increasing attention and many algorithms are proposed based on it. In these algorithms for visual tracking, each candidate target is sparsely represented by a set of target templates. However, these algorithms fail to consider the structural information of the space of the target templates, i.e., target template set. In this paper, we propose an algorithm named non-local self-similarity (NLSS) based sparse coding algorithm (NLSSC) to learn the sparse representations, which considers the geometrical structure of the set of target candidates. By using non-local self-similarity (NLSS) as a smooth operator, the proposed method can turn the tracking into sparse representations problems, in which the information of the set of target candidates is exploited. Extensive experimental results on visual tracking have demonstrated the effectiveness of the proposed algorithm.
---
paper_title: Locally Orderless Tracking

paper_content:

Locally Orderless Tracking (LOT) is a visual tracking algorithm that automatically estimates the amount of local (dis)order in the target. This lets the tracker specialize in both rigid and deformable objects on-line and with no prior assumptions. We provide a probabilistic model of the target variations over time. We then rigorously show that this model is a special case of the Earth Mover's Distance optimization problem where the ground distance is governed by some underlying noise model. This noise model has several parameters that control the cost of moving pixels and changing their color. We develop two such noise models and demonstrate how their parameters can be estimated on-line during tracking to account for the amount of local (dis)order in the target. We also discuss the significance of this on-line parameter update and demonstrate its contribution to the performance. Finally we show LOT's tracking capabilities on challenging video sequences, both commonly used and new, displaying performance comparable to state-of-the-art methods.
---
paper_title: Visual tracking with online Multiple Instance Learning

paper_content:

In this paper, we address the problem of learning an adaptive appearance model for object tracking. In particular, a class of tracking techniques called “tracking by detection” have been shown to give promising results at real-time speeds. These methods train a discriminative classifier in an online manner to separate the object from the background. This classifier bootstraps itself by using the current tracker state to extract positive and negative examples from the current frame. Slight inaccuracies in the tracker can therefore lead to incorrectly labeled training examples, which degrades the classifier and can cause further drift. In this paper we show that using Multiple Instance Learning (MIL) instead of traditional supervised learning avoids these problems, and can therefore lead to a more robust tracker with fewer parameter tweaks. We present a novel online MIL algorithm for object tracking that achieves superior results with real-time performance.
---
paper_title: High-Speed Tracking with Kernelized Correlation Filters

paper_content:

The core component of most modern trackers is a discriminative classifier, tasked with distinguishing between the target and the surrounding environment. To cope with natural image changes, this classifier is typically trained with translated and scaled sample patches. Such sets of samples are riddled with redundancies—any overlapping pixels are constrained to be the same. Based on this simple observation, we propose an analytic model for datasets of thousands of translated patches. By showing that the resulting data matrix is circulant, we can diagonalize it with the discrete Fourier transform, reducing both storage and computation by several orders of magnitude. Interestingly, for linear regression our formulation is equivalent to a correlation filter, used by some of the fastest competitive trackers. For kernel regression, however, we derive a new kernelized correlation filter (KCF), that unlike other kernel algorithms has the exact same complexity as its linear counterpart. Building on it, we also propose a fast multi-channel extension of linear correlation filters, via a linear kernel, which we call dual correlation filter (DCF). Both KCF and DCF outperform top-ranking trackers such as Struck or TLD on a 50 videos benchmark, despite running at hundreds of frames-per-second, and being implemented in a few lines of code (Algorithm 1). To encourage further developments, our tracking framework was made open-source.
---
paper_title: Robust visual tracking using discriminative stable regions and K-means clustering

paper_content:

This paper presents a method of extracting discriminative stable regions (DSRs) from image, and applies them for object tracking. These DSRs obtained by using the criterion of maximal entropy and spatial discrimination present high appearance stability and strong spatial discriminative power, which enables them to tolerate more appearance variations and to effectively resist spatial distracters. Meanwhile, the adaptive fusion tracking incorporated k-means clustering can handle severe occlusion as well as disturbance of motion noise during target localization. In addition, an effective local update scheme is designed to adapt to the object change for ensuring the tracking robustness. Experiments are carried out on several challenging sequences and results show that our method performs well in terms of object tracking, even in the presence of occlusion, deformation, illumination change, moving camera and spatial distracter.
---
paper_title: Structure Preserving Object Tracking

paper_content:

Model-free trackers can track arbitrary objects based on a single (bounding-box) annotation of the object. Whilst the performance of model-free trackers has recently improved significantly, simultaneously tracking multiple objects with similar appearance remains very hard. In this paper, we propose a new multi-object model-free tracker (based on tracking-by-detection) that resolves this problem by incorporating spatial constraints between the objects. The spatial constraints are learned along with the object detectors using an online structured SVM algorithm. The experimental evaluation of our structure-preserving object tracker (SPOT) reveals significant performance improvements in multi-object tracking. We also show that SPOT can improve the performance of single-object trackers by simultaneously tracking different parts of the object.
---
paper_title: Minimum Uncertainty Gap for Robust Visual Tracking

paper_content:

We propose a novel tracking algorithm that robustly tracks the target by finding the state which minimizes uncertainty of the likelihood at current state. The uncertainty of the likelihood is estimated by obtaining the gap between the lower and upper bounds of the likelihood. By minimizing the gap between the two bounds, our method finds the confident and reliable state of the target. In the paper, the state that gives the Minimum Uncertainty Gap (MUG) between likelihood bounds is shown to be more reliable than the state which gives the maximum likelihood only, especially when there are severe illumination changes, occlusions, and pose variations. A rigorous derivation of the lower and upper bounds of the likelihood for the visual tracking problem is provided to address this issue. Additionally, an efficient inference algorithm using Interacting Markov Chain Monte Carlo is presented to find the best state that maximizes the average of the lower and upper bounds of the likelihood and minimizes the gap between two bounds simultaneously. Experimental results demonstrate that our method successfully tracks the target in realistic videos and outperforms conventional tracking methods.
---
paper_title: Visual Tracking via Locality Sensitive Histograms

paper_content:

This paper presents a novel locality sensitive histogram algorithm for visual tracking. Unlike the conventional image histogram that counts the frequency of occurrences of each intensity value by adding ones to the corresponding bin, a locality sensitive histogram is computed at each pixel location and a floating-point value is added to the corresponding bin for each occurrence of an intensity value. The floating-point value declines exponentially with respect to the distance to the pixel location where the histogram is computed, thus every pixel is considered but those that are far away can be neglected due to the very small weights assigned. An efficient algorithm is proposed that enables the locality sensitive histograms to be computed in time linear in the image size and the number of bins. A robust tracking framework based on the locality sensitive histograms is proposed, which consists of two main components: a new feature for tracking that is robust to illumination changes and a novel multi-region tracking algorithm that runs in real time even with hundreds of regions. Extensive experiments demonstrate that the proposed tracking framework outperforms the state-of-the-art methods in challenging scenarios, especially when the illumination changes dramatically.
---
paper_title: Real-time visual tracking via online weighted multiple instance learning

paper_content:

Adaptive tracking-by-detection methods have been widely studied with promising results. These methods first train a classifier in an online manner. Then, a sliding window is used to extract some samples from the local regions surrounding the former object location at the new frame. The classifier is then applied to these samples where the location of sample with maximum classifier score is the new object location. However, such classifier may be inaccurate when the training samples are imprecise which causes drift. Multiple instance learning (MIL) method is recently introduced into the tracking task, which can alleviate drift to some extent. However, the MIL tracker may detect the positive sample that is less important because it does not discriminatively consider the sample importance in its learning procedure. In this paper, we present a novel online weighted MIL (WMIL) tracker. The WMIL tracker integrates the sample importance into an efficient online learning procedure by assuming the most important sample (i.e., the tracking result in current frame) is known when training the classifier. A new bag probability function combining the weighted instance probability is proposed via which the sample importance is considered. Then, an efficient online approach is proposed to approximately maximize the bag likelihood function, leading to a more robust and much faster tracker. Experimental results on various benchmark video sequences demonstrate the superior performance of our algorithm to state-of-the-art tracking algorithms.
---
paper_title: Context tracker: Exploring supporters and distracters in unconstrained environments

paper_content:

Visual tracking in unconstrained environments is very challenging due to the existence of several sources of varieties such as changes in appearance, varying lighting conditions, cluttered background, and frame-cuts. A major factor causing tracking failure is the emergence of regions having similar appearance as the target. It is even more challenging when the target leaves the field of view (FoV) leading the tracker to follow another similar object, and not reacquire the right target when it reappears. This paper presents a method to address this problem by exploiting the context on-the-fly in two terms: Distracters and Supporters. Both of them are automatically explored using a sequential randomized forest, an online template-based appearance model, and local features. Distracters are regions which have similar appearance as the target and consistently co-occur with high confidence score. The tracker must keep tracking these distracters to avoid drifting. Supporters, on the other hand, are local key-points around the target with consistent co-occurrence and motion correlation in a short time span. They play an important role in verifying the genuine target. Extensive experiments on challenging real-world video sequences show the tracking improvement when using this context information. Comparisons with several state-of-the-art approaches are also provided.
---
paper_title: Locally discriminative stable model for visual tracking with clustering and principle component analysis

paper_content:

The challenge of visual tracking mainly comes from intrinsic appearance variations of the target and extrinsic environment changes around the target in a long duration, so the tracker that can simultaneously tolerate these variabilities is largely expected. In this study, the authors propose a new tracking approach based on discriminative stable regions (DSRs). The DSRs are obtained based on the criterion of maximal local entropy and spatial discrimination, which enables the tracker to handle well distractors and appearance variations. The collaborative tracking incorporated hierarchical clustering can tolerate motion noise and occlusions. In addition, as an efficient tool, the principle component analysis is used to discover the potential affine relation between DSR and the target, which timely adapts to the shape deformation of the target. Extensive experiments show that the proposed method achieves superior performance in many challenging target tracking tasks.
---
paper_title: Visual tracking via adaptive structural local sparse appearance model

paper_content:

Sparse representation has been applied to visual tracking by finding the best candidate with minimal reconstruction error using target templates. However most sparse representation based trackers only consider the holistic representation and do not make full use of the sparse coefficients to discriminate between the target and the background, and hence may fail with more possibility when there is similar object or occlusion in the scene. In this paper we develop a simple yet robust tracking method based on the structural local sparse appearance model. This representation exploits both partial information and spatial information of the target based on a novel alignment-pooling method. The similarity obtained by pooling across the local patches helps not only locate the target more accurately but also handle occlusion. In addition, we employ a template update strategy which combines incremental subspace learning and sparse representation. This strategy adapts the template to the appearance change of the target with less possibility of drifting and reduces the influence of the occluded target template as well. Both qualitative and quantitative evaluations on challenging benchmark image sequences demonstrate that the proposed tracking algorithm performs favorably against several state-of-the-art methods.
---
paper_title: Robust object tracking via sparsity-based collaborative model

paper_content:

In this paper we propose a robust object tracking algorithm using a collaborative model. As the main challenge for object tracking is to account for drastic appearance change, we propose a robust appearance model that exploits both holistic templates and local representations. We develop a sparsity-based discriminative classifier (SD-C) and a sparsity-based generative model (SGM). In the S-DC module, we introduce an effective method to compute the confidence value that assigns more weights to the foreground than the background. In the SGM module, we propose a novel histogram-based method that takes the spatial information of each patch into consideration with an occlusion handing scheme. Furthermore, the update scheme considers both the latest observations and the original template, thereby enabling the tracker to deal with appearance change effectively and alleviate the drift problem. Numerous experiments on various challenging videos demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms.
---
paper_title: Segmentation based particle filtering for real-time 2d object tracking

paper_content:

We address the problem of visual tracking of arbitrary objects that undergo significant scale and appearance changes. The classical tracking methods rely on the bounding box surrounding the target object. Regardless of the tracking approach, the use of bounding box quite often introduces background information. This information propagates in time and its accumulation quite often results in drift and tracking failure. This is particularly the case with the particle filtering approach that is often used for visual tracking. However, it always uses a bounding box around the object to compute features of the particle samples. Since this causes the drift, we propose to use segmentation for sampling. Relying on segmentation and computing the colour and gradient orientation histograms from these segmented particle samples allows the tracker to easily adapt to the object's deformations, occlusions, orientation, scale and appearance changes. We propose two particle sampling strategies based on segmentation. In the first, segmentation is done for every propagated particle sample, while in the second only the strongest particle sample is segmented. Depending on this decision there is obviously a trade-off between speed and performance. ::: ::: We perform an exhaustive quantitative evaluation on a number of challenging sequences and compare our method with the number of state-of-the-art methods previously evaluated on those sequences. The results we obtain outperform majority of the related work, both in terms of the performance and speed.
---
paper_title: Visual tracking decomposition

paper_content:

We propose a novel tracking algorithm that can work robustly in a challenging scenario such that several kinds of appearance and motion changes of an object occur at the same time. Our algorithm is based on a visual tracking decomposition scheme for the efficient design of observation and motion models as well as trackers. In our scheme, the observation model is decomposed into multiple basic observation models that are constructed by sparse principal component analysis (SPCA) of a set of feature templates. Each basic observation model covers a specific appearance of the object. The motion model is also represented by the combination of multiple basic motion models, each of which covers a different type of motion. Then the multiple basic trackers are designed by associating the basic observation models and the basic motion models, so that each specific tracker takes charge of a certain change in the object. All basic trackers are then integrated into one compound tracker through an interactive Markov Chain Monte Carlo (IMCMC) framework in which the basic trackers communicate with one another interactively while run in parallel. By exchanging information with others, each tracker further improves its performance, which results in increasing the whole performance of tracking. Experimental results show that our method tracks the object accurately and reliably in realistic videos where the appearance and motion are drastically changing over time.
---
paper_title: Struck: Structured Output Tracking with Kernels

paper_content:

Adaptive tracking-by-detection methods are widely used in computer vision for tracking arbitrary objects. Current approaches treat the tracking problem as a classification task and use online learning techniques to update the object model. However, for these updates to happen one needs to convert the estimated object position into a set of labelled training examples, and it is not clear how best to perform this intermediate step. Furthermore, the objective for the classifier (label prediction) is not explicitly coupled to the objective for the tracker (estimation of object position). In this paper, we present a framework for adaptive visual object tracking based on structured output prediction. By explicitly allowing the output space to express the needs of the tracker, we avoid the need for an intermediate classification step. Our method uses a kernelised structured output support vector machine (SVM), which is learned online to provide adaptive tracking. To allow our tracker to run at high frame rates, we (a) introduce a budgeting mechanism that prevents the unbounded growth in the number of support vectors that would otherwise occur during tracking, and (b) show how to implement tracking on the GPU. Experimentally, we show that our algorithm is able to outperform state-of-the-art trackers on various benchmark videos. Additionally, we show that we can easily incorporate additional features and kernels into our framework, which results in increased tracking performance.
---
paper_title: Robust object tracking via sparsity-based collaborative model

paper_content:

In this paper we propose a robust object tracking algorithm using a collaborative model. As the main challenge for object tracking is to account for drastic appearance change, we propose a robust appearance model that exploits both holistic templates and local representations. We develop a sparsity-based discriminative classifier (SD-C) and a sparsity-based generative model (SGM). In the S-DC module, we introduce an effective method to compute the confidence value that assigns more weights to the foreground than the background. In the SGM module, we propose a novel histogram-based method that takes the spatial information of each patch into consideration with an occlusion handing scheme. Furthermore, the update scheme considers both the latest observations and the original template, thereby enabling the tracker to deal with appearance change effectively and alleviate the drift problem. Numerous experiments on various challenging videos demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms.
---
paper_title: Online Object Tracking: A Benchmark

paper_content:

Object tracking is one of the most important components in numerous applications of computer vision. While much progress has been made in recent years with efforts on sharing code and datasets, it is of great importance to develop a library and benchmark to gauge the state of the art. After briefly reviewing recent advances of online object tracking, we carry out large scale experiments with various evaluation criteria to understand how these algorithms perform. The test image sequences are annotated with different attributes for performance evaluation and analysis. By analyzing quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field.
---
paper_title: Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review

paper_content:

The use of hand gestures provides an attractive alternative to cumbersome interface devices for human-computer interaction (HCI). In particular, visual interpretation of hand gestures can help in achieving the ease and naturalness desired for HCI. This has motivated a very active research area concerned with computer vision-based analysis and interpretation of hand gestures. We survey the literature on visual interpretation of hand gestures in the context of its role in HCI. This discussion is organized on the basis of the method used for modeling, analyzing, and recognizing gestures. Important differences in the gesture interpretation approaches arise depending on whether a 3D model of the human hand or an image appearance model of the human hand is used. 3D hand models offer a way of more elaborate modeling of hand gestures but lead to computational hurdles that have not been overcome given the real-time requirements of HCI. Appearance-based models lead to computationally efficient "purposive" approaches that work well under constrained situations but seem to lack the generality desirable for HCI. We also discuss implemented gestural systems as well as other potential applications of vision-based gesture recognition. Although the current progress is encouraging, further theoretical as well as computational advances are needed before gestures can be widely used for HCI. We discuss directions of future research in gesture recognition, including its integration with other natural modes of human-computer interaction.
---
paper_title: Hollywood 3D: Recognizing Actions in 3D Natural Scenes

paper_content:

Action recognition in unconstrained situations is a difficult task, suffering from massive intra-class variations. It is made even more challenging when complex 3D actions are projected down to the image plane, losing a great deal of information. The recent emergence of 3D data, both in broadcast content, and commercial depth sensors, provides the possibility to overcome this issue. This paper presents a new dataset, for benchmarking action recognition algorithms in natural environments, while making use of 3D information. The dataset contains around 650 video clips, across 14 classes. In addition, two state of the art action recognition algorithms are extended to make use of the 3D data, and five new interest point detection strategies are also proposed, that extend to the 3D data. Our evaluation compares all 4 feature descriptors, using 7 different types of interest point, over a variety of threshold levels, for the Hollywood3D dataset. We make the dataset including stereo video, estimated depth maps and all code required to reproduce the benchmark results, available to the wider community.
---
paper_title: Multi-modal gesture recognition challenge 2013: dataset and results

paper_content:

The recognition of continuous natural gestures is a complex and challenging problem due to the multi-modal nature of involved visual cues (e.g. fingers and lips movements, subtle facial expressions, body pose, etc.), as well as technical limitations such as spatial and temporal resolution and unreliable depth cues. In order to promote the research advance on this field, we organized a challenge on multi-modal gesture recognition. We made available a large video database of 13,858 gestures from a lexicon of 20 Italian gesture categories recorded with a Kinect™ camera, providing the audio, skeletal model, user mask, RGB and depth images. The focus of the challenge was on user independent multiple gesture learning. There are no resting positions and the gestures are performed in continuous sequences lasting 1-2 minutes, containing between 8 and 20 gesture instances in each sequence. As a result, the dataset contains around 1.720.800 frames. In addition to the 20 main gesture categories, "distracter" gestures are included, meaning that additional audio and gestures out of the vocabulary are included. The final evaluation of the challenge was defined in terms of the Levenshtein edit distance, where the goal was to indicate the real order of gestures within the sequence. 54 international teams participated in the challenge, and outstanding results were obtained by the first ranked participants.
---
paper_title: The Terrascope Dataset: Scripted Multi-Camera Indoor Video Surveillance with Ground-truth

paper_content:

This paper introduces a new video surveillance dataset that was captured by a network of synchronized cameras placed throughout an indoor setting and augmented with groundtruth data. The dataset includes ten minutes of individuals who are moving throughout the sensor network. In addition, three scripted scenarios that contain behaviors exhibtied over a wide-area, such as "gathering for a meeting" or "stealing an object" are included to assist researchers who are interested in wide-area surveillance and behavior recognition. In addition to the video data, a face and gait database for all twelve individuals observed by the network of cameras is supplied. Hand-segmented ground-truth foreground regions are provided for every 500th frame in all cameras and for many sequential frames in two overlapping views. The entrance and exit time of each individual in each camera for one of the scenarios is provided in an XML database. We believe that the dataset will help provide a common development and verifcation framework for the increasing number of research efforts related to video surveillance in multiple, potentially non-overlapping, camera networks.
---
paper_title: The BEHAVE video dataset: ground truthed video for multi-person

paper_content:

Although there is much research on behaviour recognition in time-varying video, there are few ground truthed datasets for assessing multi-person behavioral interactions. This short paper presents the BEHAVE project’s dataset, which has around 90,000 frames of humans identified by bounding boxes, with interacting groups classified into one of 5 different behaviors. An example of its use is also presented.
---
paper_title: HMDB: A large video database for human motion recognition

paper_content:

With nearly one billion online videos viewed everyday, an emerging new frontier in computer vision research is recognition and search in video. While much effort has been devoted to the collection and annotation of large scalable static image datasets containing thousands of image categories, human action datasets lag far behind. Current action recognition databases contain on the order of ten different action categories collected under fairly controlled conditions. State-of-the-art performance on these datasets is now near ceiling and thus there is a need for the design and creation of new benchmarks. To address this issue we collected the largest action video database to-date with 51 action categories, which in total contain around 7,000 manually annotated clips extracted from a variety of sources ranging from digitized movies to YouTube. We use this database to evaluate the performance of two representative computer vision systems for action recognition and explore the robustness of these methods under various conditions such as camera motion, viewpoint, video quality and occlusion.
---
paper_title: The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition

paper_content:

This paper describes the world's largest gait database-the “OU-ISIR Gait Database, Large Population Dataset”-and its application to a statistically reliable performance evaluation of vision-based gait recognition. Whereas existing gait databases include at most 185 subjects, we construct a larger gait database that includes 4007 subjects (2135 males and 1872 females) with ages ranging from 1 to 94 years. The dataset allows us to determine statistically significant performance differences between currently proposed gait features. In addition, the dependences of gait-recognition performance on gender and age group are investigated and the results provide several novel insights, such as the gradual change in recognition performance with human growth.
---
paper_title: The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition

paper_content:

This paper describes the world's largest gait database-the “OU-ISIR Gait Database, Large Population Dataset”-and its application to a statistically reliable performance evaluation of vision-based gait recognition. Whereas existing gait databases include at most 185 subjects, we construct a larger gait database that includes 4007 subjects (2135 males and 1872 females) with ages ranging from 1 to 94 years. The dataset allows us to determine statistically significant performance differences between currently proposed gait features. In addition, the dependences of gait-recognition performance on gender and age group are investigated and the results provide several novel insights, such as the gradual change in recognition performance with human growth.
---
paper_title: Hollywood 3D: Recognizing Actions in 3D Natural Scenes

paper_content:

Action recognition in unconstrained situations is a difficult task, suffering from massive intra-class variations. It is made even more challenging when complex 3D actions are projected down to the image plane, losing a great deal of information. The recent emergence of 3D data, both in broadcast content, and commercial depth sensors, provides the possibility to overcome this issue. This paper presents a new dataset, for benchmarking action recognition algorithms in natural environments, while making use of 3D information. The dataset contains around 650 video clips, across 14 classes. In addition, two state of the art action recognition algorithms are extended to make use of the 3D data, and five new interest point detection strategies are also proposed, that extend to the 3D data. Our evaluation compares all 4 feature descriptors, using 7 different types of interest point, over a variety of threshold levels, for the Hollywood3D dataset. We make the dataset including stereo video, estimated depth maps and all code required to reproduce the benchmark results, available to the wider community.
---
paper_title: HMDB: A large video database for human motion recognition

paper_content:

With nearly one billion online videos viewed everyday, an emerging new frontier in computer vision research is recognition and search in video. While much effort has been devoted to the collection and annotation of large scalable static image datasets containing thousands of image categories, human action datasets lag far behind. Current action recognition databases contain on the order of ten different action categories collected under fairly controlled conditions. State-of-the-art performance on these datasets is now near ceiling and thus there is a need for the design and creation of new benchmarks. To address this issue we collected the largest action video database to-date with 51 action categories, which in total contain around 7,000 manually annotated clips extracted from a variety of sources ranging from digitized movies to YouTube. We use this database to evaluate the performance of two representative computer vision systems for action recognition and explore the robustness of these methods under various conditions such as camera motion, viewpoint, video quality and occlusion.
---
paper_title: A survey of video datasets for human action and activity recognition

paper_content:

Highlights? Description of datasets for video-based human activity and action recognition. ? 68 datasets reported: 28 for heterogeneous and 40 for specific human actions. ? Useful data, such as web for dowloading, published works or ground truth, are provided. ? Datasets are compared and classified. Vision-based human action and activity recognition has an increasing importance among the computer vision community with applications to visual surveillance, video retrieval and human-computer interaction. In recent years, more and more datasets dedicated to human action and activity recognition have been created. The use of these datasets allows us to compare different recognition systems with the same input data. The survey introduced in this paper tries to cover the lack of a complete description of the most important public datasets for video-based human activity and action recognition and to guide researchers in the election of the most suitable dataset for benchmarking their algorithms.
---
paper_title: The BEHAVE video dataset: ground truthed video for multi-person

paper_content:

Although there is much research on behaviour recognition in time-varying video, there are few ground truthed datasets for assessing multi-person behavioral interactions. This short paper presents the BEHAVE project’s dataset, which has around 90,000 frames of humans identified by bounding boxes, with interacting groups classified into one of 5 different behaviors. An example of its use is also presented.
---
paper_title: Dynamic scene understanding by improved sparse topical coding

paper_content:

The explosive growth of cameras in public areas demands a technique which develops a fully automated surveillance and monitoring system. In this paper, we propose a novel unsupervised approach to automatically explore motion patterns occurring in dynamic scenes under an improved sparse topical coding (STC) framework. Given an input video, it is segmented into a sequence of clips without overlapping. Optical flow features are extracted from each pair of consecutive frames, and quantized into discrete visual flow words. Each video clip is interpreted as a document and visual flow words as words within the document. Then the improved STC is applied to explore latent patterns which represent the common motion distributions of the scene. Finally, each video clip is represented as a weighted summation of these patterns with only a few non-zero coefficients. The proposed approach is purely data-driven and scene independent, which make it suitable for very large range applications of scenarios, such as rule mining and abnormal event detection. Experimental results and comparisons on various public datasets demonstrate the promise of the proposed approach.
---
paper_title: The Terrascope Dataset: Scripted Multi-Camera Indoor Video Surveillance with Ground-truth

paper_content:

This paper introduces a new video surveillance dataset that was captured by a network of synchronized cameras placed throughout an indoor setting and augmented with groundtruth data. The dataset includes ten minutes of individuals who are moving throughout the sensor network. In addition, three scripted scenarios that contain behaviors exhibtied over a wide-area, such as "gathering for a meeting" or "stealing an object" are included to assist researchers who are interested in wide-area surveillance and behavior recognition. In addition to the video data, a face and gait database for all twelve individuals observed by the network of cameras is supplied. Hand-segmented ground-truth foreground regions are provided for every 500th frame in all cameras and for many sequential frames in two overlapping views. The entrance and exit time of each individual in each camera for one of the scenarios is provided in an XML database. We believe that the dataset will help provide a common development and verifcation framework for the increasing number of research efforts related to video surveillance in multiple, potentially non-overlapping, camera networks.
---
paper_title: Online Object Tracking: A Benchmark

paper_content:

Object tracking is one of the most important components in numerous applications of computer vision. While much progress has been made in recent years with efforts on sharing code and datasets, it is of great importance to develop a library and benchmark to gauge the state of the art. After briefly reviewing recent advances of online object tracking, we carry out large scale experiments with various evaluation criteria to understand how these algorithms perform. The test image sequences are annotated with different attributes for performance evaluation and analysis. By analyzing quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field.
---

Title: A Survey of Datasets for Visual Tracking

Section 1: Introduction
Description 1: Introduce the concept of visual tracking, its challenges, and the goal of the paper.

Section 2: Visual tracking: which dataset for which need?
Description 2: Define visual tracking tasks and goals, and discuss the selection of datasets for evaluating visual tracking algorithms.

Section 3: Visual tracking issues
Description 3: Address issues raised by visual tracking and detail different metrics used to evaluate visual tracking algorithms.

Section 4: Problems inherent to visual tracking
Description 4: Provide a list of challenging problems in visual tracking such as illumination effects, scene clutter, changes in object appearance, abrupt changes in motion, occlusions, similar appearances, camera motions, appearance/disappearance, and quality of frames.

Section 5: Quantitative evaluation of visual tracking
Description 5: Discuss quantitative evaluation metrics including error score, accuracy score, success score, and detection scores.

Section 6: Robustness of evaluation
Description 6: Explain the importance of evaluating the robustness of visual trackers to initialization conditions and provide methods for such evaluation.

Section 7: Datasets for visual tracking
Description 7: Present a selection of publicly available datasets for visual tracking, describe their features, and list the key properties and difficulties they address.

Section 8: Datasets for scene analysis and understanding based on visual tracking
Description 8: Provide an overview of datasets used for dynamic scene analysis and understanding, particularly focusing on human motion analysis and global scene motion understanding.

Section 9: Human understanding
Description 9: Discuss datasets for analyzing human gestures, faces, body motion, individual actions, behaviors, interactions, and social activities.

Section 10: Scene understanding
Description 10: Explain datasets for applications such as video surveillance, event detection, crowd analysis, and sports games description.

Section 11: Conclusion
Description 11: Summarize the findings, discuss the importance of choosing appropriate datasets for visual tracking, and suggest future directions for creating benchmarks and universal measures of tracking quality.