Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis
---
paper_title: Object-oriented Bayesian networks for detection of lane change maneuvers

paper_content:

In this paper we introduce a novel approach towards the recognition of typical driving maneuvers in structured highway scenarios and identify some of the key benefits of traffic scene modeling with object-oriented Bayesian networks (OOBNs). The approach exploits the advantages of an introduced lane-related coordinate system together with individual occupancy grids for all vehicles. This combination allows for an efficient classification of the existing vehiclelane and vehicle-vehicle relations in a traffic scene and thus substantially improves the understanding of complex traffic scenes. We systematically propagate probabilities and variances within our network which results in probabilistic sets of the modeled driving maneuvers. Using this generic approach, we are able to classify a total of 27 driving maneuvers including merging and object following.
---
paper_title: On-road vehicle detection: a review

paper_content:

Developing on-board automotive driver assistance systems aiming to alert drivers about driving environments, and possible collision with other vehicles has attracted a lot of attention lately. In these systems, robust and reliable vehicle detection is a critical step. This paper presents a review of recent vision-based on-road vehicle detection systems. Our focus is on systems where the camera is mounted on the vehicle rather than being fixed such as in traffic/driveway monitoring systems. First, we discuss the problem of on-road vehicle detection using optical sensors followed by a brief review of intelligent vehicle research worldwide. Then, we discuss active and passive sensors to set the stage for vision-based vehicle detection. Methods aiming to quickly hypothesize the location of vehicles in an image as well as to verify the hypothesized locations are reviewed next. Integrating detection with tracking is also reviewed to illustrate the benefits of exploiting temporal continuity for vehicle detection. Finally, we present a critical overview of the methods discussed, we assess their potential for future deployment, and we present directions for future research.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: Combining monocular and stereo-vision for real-time vehicle ranging and tracking on multilane highways

paper_content:

In this paper, we introduce a novel stereo-monocular fusion approach to on-road localization and tracking of vehicles. Utilizing a calibrated stereo-vision rig, the proposed approach combines monocular detection with stereo-vision for on-road vehicle localization and tracking for driver assistance. The system initially acquires synchronized monocular frames and calculates depth maps from the stereo rig. The system then detects vehicles in the image plane using an active learning-based monocular vision approach. Using the image coordinates of detected vehicles, the system then localizes the vehicles in real-world coordinates using the calculated depth map. The vehicles are tracked both in the image plane, and in real-world coordinates, fusing information from both the monocular and stereo modalities. Vehicles' states are estimated and tracked using Kalman filtering. Quantitative analysis of tracks is provided. The full system takes 46ms to process a single frame.
---
paper_title: Learning multi-lane trajectories using vehicle-based vision

paper_content:

Safe operation of a motor vehicle requires awareness of the current traffic situation as well as the ability to predict future maneuvers. In order to provide an intelligent vehicle the ability to make predictions, this work proposes a framework for understanding the driving situation based on vehicle mounted vision sensors. Vehicles are tracked using Kalman filtering based on a vision-based system that detects and tracks using a combination of monocular and stereo-vision. The vehicles' full trajectories are recorded, and a data-driven learning framework has been applied to automatically learn surround behaviors. By learning based on observations, the ADAS system is being trained by experience. Learned trajectories have been compared between dense and free-flowing traffic conditions. Preliminary experimental results using real-world multi-lane highways show the basic promise of this approach. Future research directions are discussed.
---
paper_title: Amplitude-Modulated Laser Radar for Range and Speed Measurement in Car Applications

paper_content:

Doppler laser radar can improve the precision of speed measurement by about two orders of magnitude compared with time-of-flight range finders, which obtain target speeds by range differentiation. However, in a car environment, the usage of traditional Doppler laser radar schemes is limited, because they do not satisfy the requirement of simultaneously measuring the target range together with speed with high precision. First, in this paper, we describe a new in-car laser radar system and show a new modulation scheme that enables the in-car laser radar to simultaneously measure the target range and speed with high precision. Then, we perform simulations and experiments to verify the accuracy of the proposed method. In the Appendix, a brief review of current widely used laser radar schemes is given. The limitations of these schemes to their employment in car applications are also discussed.
---
paper_title: Sensorfusion Using Spatio-Temporal Aligned Video and Lidar for Improved Vehicle Detection

paper_content:

This contribution introduces a novel approach to cross-calibrate automotive vision and ranging sensors. The resulting sensor alignment allows the incorporation of multiple sensor data into a detection and tracking framework. Exemplarily, we show how a realtime vehicle detection system, intended for emergency breaking or ACC applications, benefits from the low level fusion of multibeam lidar and vision sensor measurements in discrimination performance and computational complexity.
---
paper_title: Vehicle Detection Based on Laser Radar

paper_content:

This paper describes the detection of moving obstacles using laser radar in road environments. This application is designed to be implemented in further research on data fusion technologies. The developed application uses only a laser radar which provides information to sort objects according to their shape and movement. The subsequent detection and classification provide higher level tracking.
---
paper_title: Towards fully autonomous driving: Systems and algorithms

paper_content:

In order to achieve autonomous operation of a vehicle in urban situations with unpredictable traffic, several realtime systems must interoperate, including environment perception, localization, planning, and control. In addition, a robust vehicle platform with appropriate sensors, computational hardware, networking, and software infrastructure is essential.
---
paper_title: Multilayer lidar-based pedestrian tracking in urban environments

paper_content:

This paper presents a method for pedestrian tracking in urban environments using in-vehicle multilayer laser lidar (MLLR). The MLLR that we developed irradiates the laser in six scanning planes by a polygon mirror mechanism, and thus objects with height are observed with the plural scanning planes. MLLR outputs are modified by GPS data and are mapped onto a grid map. Pedestrians are found based on the occupancy grid method, and they are tracked via Kalman filter in conjunction with global nearest neighboring (GNN) based data association. A track management method improves tracking accuracy in real worlds. Our tracking algorithm works well in a low-performance computer environment. The experimental results in different scenarios such as intersection and on the community road validate the proposed method.
---
paper_title: Integrated Lane and Vehicle Detection, Localization, and Tracking: A Synergistic Approach

paper_content:

In this paper, we introduce a synergistic approach to integrated lane and vehicle tracking for driver assistance. The approach presented in this paper results in a final system that improves on the performance of both lane tracking and vehicle tracking modules. Further, the presented approach introduces a novel approach to localizing and tracking other vehicles on the road with respect to lane position, which provides information on higher contextual relevance that neither the lane tracker nor vehicle tracker can provide by itself. Improvements in lane tracking and vehicle tracking have been extensively quantified. Integrated system performance has been validated on real-world highway data. Without specific hardware and software optimizations, the fully implemented system runs at near-real-time speeds of 11 frames per second.
---
paper_title: Electronically scanned millimeter-wave radar for pre-crash safety and adaptive cruise control system

paper_content:

An objective of developing intelligent transport systems (ITS) is to enhance driving convenience. As an ITS technology that meets this objective, an adaptive cruise control (ACC) system has already been commercialized. Another objective is to enhance the vehicle safety performance. With the aim of attaining this objective, we have developed a Pre-crash Safety system by evolving the ITS technology used in ACC. The Pre-crash Safety system is designed to predict an unavoidable collision and activate Pre-crash Seatbelt and Pre-crash Brake Assist well in advance of the collision to mitigate occupant injury caused by the collision. A millimeter-wave radar sensor is indispensable to the Pre-crash Safety system. This present paper describes the electronically scanned millimeter-wave radar sensor developed for use in the system. This radar is physically compact while providing high object recognition performance. This is due to the phased array technology employed for the first time in the world for automotive radar. Other features of the radar include a "high-efficiency planar antenna, "high-power and low-loss monolithic microwave ICs (MMICs) and millimeter-wave circuit" and a "high-speed digital-beam-forming (DBF) algorithm".
---
paper_title: On-road vehicle detection: a review

paper_content:

Developing on-board automotive driver assistance systems aiming to alert drivers about driving environments, and possible collision with other vehicles has attracted a lot of attention lately. In these systems, robust and reliable vehicle detection is a critical step. This paper presents a review of recent vision-based on-road vehicle detection systems. Our focus is on systems where the camera is mounted on the vehicle rather than being fixed such as in traffic/driveway monitoring systems. First, we discuss the problem of on-road vehicle detection using optical sensors followed by a brief review of intelligent vehicle research worldwide. Then, we discuss active and passive sensors to set the stage for vision-based vehicle detection. Methods aiming to quickly hypothesize the location of vehicles in an image as well as to verify the hypothesized locations are reviewed next. Integrating detection with tracking is also reviewed to illustrate the benefits of exploiting temporal continuity for vehicle detection. Finally, we present a critical overview of the methods discussed, we assess their potential for future deployment, and we present directions for future research.
---
paper_title: Learning Algorithm for Real-Time Vehicle Tracking

paper_content:

This article presents a learning algorithm for real-time object tracking in video sequences which uses an improvement of a feature selection method known from object detection. But in contrast to trackers based on object detection methods, our approach explicitly selects the features which are best suited to track an object, which are different from the best features for object detection. The used features are constructed from pairs of image patches and related to Haar features. Besides the automatic selection of features according to their discriminative (tracking) power, the advantage of this approach is that the resulting tracker is very fast, allowing it to run in addition to a detector to robustify the object position estimation and to compensate for dropouts of the detector. A comparison of the proposed tracking algorithm with other tracking methods is presented which shows the accuracy of the proposed algorithm
---
paper_title: On-road vehicle tracking using deformable object model and particle filter with integrated likelihoods

paper_content:

This paper proposes a novel method for vehicle detection and tracking using a vehicle-mounted monocular camera. In this method, features of vehicles are learned as a deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOG). The vehicle detector uses both global and local features as the deformable object model. Detected vehicles are tracked by using a particle filter with integrated likelihoods, such as the probability of vehicles estimated from the deformable object model and the intensity correlation between different picture frames. Tracking likelihoods are iteratively used as the a priori probability for the next frame. The experimental results showed that the proposed method can achieve an average vehicle detection rate of 98% and an average vehicle tracking rate of 87% with a false positive rate of less than 0.3%.
---
paper_title: Context-Adaptive Approach for Vehicle Detection Under Varying Lighting Conditions

paper_content:

This paper presents a vision-based vehicle detection method, taking into account the lighting context of the images. The adaptability of a vehicle detection system to lighting conditions is an important characteristic on which little research has been carried out. The scheme presented here categorizes the scenes according to their lighting conditions and switches between specialized classifiers for different scene contexts. In our implementation, four categories of lighting conditions have been identified using a clustering algorithm in the space of image histograms: Daylight, Low Light, Night, and Saturation. Classifiers trained with AdaBoost are used for both Daylight and Low Light categories, and a tail-light detector is used for the Night category. No detection is made for the Saturation case. Experiments have shown a considerate improvement in the detection performance when using the proposed context-adaptive scheme compared to a single vehicle detector for all lighting conditions.
---
paper_title: Object Detection with Discriminatively Trained Part-Based Models

paper_content:

We describe an object detection system based on mixtures of multiscale deformable part models. Our system is able to represent highly variable object classes and achieves state-of-the-art results in the PASCAL object detection challenges. While deformable part models have become quite popular, their value had not been demonstrated on difficult benchmarks such as the PASCAL data sets. Our system relies on new methods for discriminative training with partially labeled data. We combine a margin-sensitive approach for data-mining hard negative examples with a formalism we call latent SVM. A latent SVM is a reformulation of MI--SVM in terms of latent variables. A latent SVM is semiconvex, and the training problem becomes convex once latent information is specified for the positive examples. This leads to an iterative training algorithm that alternates between fixing latent values for positive examples and optimizing the latent SVM objective function.
---
paper_title: Estimating surrounding vehicles' pose using computer vision

paper_content:

This paper presents a computer vision-based approach to tracking surrounding vehicles and estimating their trajectories, in order to detect potentially dangerous situations. Images are acquired using a camera mounted in the egovehicle. Estimations of the distance, velocity and orientation of other vehicles on the road are obtained by detecting their lights and shadow. Because 3D information is not readily available in a mono-camera system, several sets of constraints and assumptions on the geometry of both road and vehicles are proposed and tested in this paper. Kalman filters are used to track the detected vehicles. We also study the advantages of tracking the vehicles in road space (world coordinates), or tracking the position of the lights and shadows on the image. The performance of the approaches is evaluated on video recorded in urban environment.
---
paper_title: Parametric ego-motion estimation for vehicle surround analysis using an omnidirectional camera

paper_content:

Abstract.Omnidirectional cameras that give a 360° panoramic view of the surroundings have recently been used in many applications such as robotics, navigation, and surveillance. This paper describes the application of parametric ego-motion estimation for vehicle detection to perform surround analysis using an automobile-mounted camera. For this purpose, the parametric planar motion model is integrated with the transformations to compensate distortion in omnidirectional images. The framework is used to detect objects with independent motion or height above the road. Camera calibration as well as the approximate vehicle speed obtained from a CAN bus are integrated with the motion information from spatial and temporal gradients using a Bayesian approach. The approach is tested for various configurations of an automobile-mounted omni camera as well as a rectilinear camera. Successful detection and tracking of moving vehicles and generation of a surround map are demonstrated for application to intelligent driver support.
---
paper_title: Fusing multiple 2D visual features for vehicle detection

paper_content:

This contribution presents a multi sensor fusion approach for vehicle detection. Shadow and Symmetry features, abstracted to 3D sensors by including street surface information, are combined in an Interacting Multiple Model filter with two system models, one for constant velocity, one for constant acceleration. Measurements are associated to tracks by means of a Cheap Joint Probabilistic Data Association technique. Virtual prediction steps are used to incorporate multiple sensors and to achieve a versatile fusion architecture that allows easy integration of further sensors.
---
paper_title: Object recognition from local scale-invariant features

paper_content:

An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.
---
paper_title: Integrating Appearance and Edge Features for Sedan Vehicle Detection in the Blind-Spot Area

paper_content:

Changing lanes while having no information about the blind spot area can be dangerous. We propose a vision-based vehicle detection system for a lane changing assistance system to monitor the potential sedan vehicle in the blind-spot area. To serve our purpose, we select adequate features, which are directly obtained from vehicle images, to detect possible vehicles in the blind-spot area. This is challenging due to the significant change in the view angle of a vehicle along with its location throughout the blind-spot area. To cope with this problem, we propose a method to combine two kinds of part-based features that are related to the characteristics of the vehicle, and we build multiple models based on different viewpoints of a vehicle. The location information of each feature is incorporated to help construct the detector and estimate the reasonable position of the presence of the vehicle. The experiments show that our system is reliable in detecting various sedan vehicles in the blind-spot area.
---
paper_title: Learning based symmetric features selection for vehicle detection

paper_content:

This paper describes a symmetric features selection strategy based on statistical learning method for detecting vehicles with a single moving camera for autonomous driving. Symmetry is a good class of feature for vehicle detection, but the areas with high symmetry and threshold for segmentation is hard to be decided. Usually, the additional supposition is added artificially, and this will decrease the robustness of algorithms. In this paper, we focus on the problem of symmetric features selection using learning method for autonomous driving environment. Global symmetry and local symmetry are defined and used to construct a cascaded structure with a one-class classifier followed by a two-class classifier. Especially for local symmetric features, different symmetric areas in the rear view image of vehicles are searched through Adaboost based learning, and most useful symmetric features are extracted. The threshold for classification is also found through learning. The effective features selection strategy shows that the integration of global symmetry and local symmetry helps to improve the robustness of algorithms. Experimental results indicate the robustness and real-time performance of the algorithm.
---
paper_title: On-board robust vehicle detection and tracking using adaptive quality evaluation

paper_content:

This paper presents a robust method for real-time vehicle detection and tracking in dynamic traffic environments. The proposed strategy aims to find a trade-off between the robustness shown by time-uncorrelated detection techniques and the speed-up obtained with tracking algorithms. It combines both advantages by continuously evaluating the quality of the tracking results along time and triggering new detections to restart the tracking process when quality falls behind a certain quality requirement. Robustness is also ensured within the tracking algorithm with an outlier rejection stage and the use of stochastic filtering. Several sequences from real traffic situations have been tested, obtaining highly accurate multiple vehicle detections.
---
paper_title: Legendre and gabor moments for vehicle recognition in forward collision warning

paper_content:

Collision warning remains an active research field due to the increasing complexities of on-road traffic worldwide. Vision-based warning systems are of particular interest because of the extensive information contained in images. This paper proposes the combination of Legendre moments and Gabor features for monocular vision-based vehicle recognition. We focus on vehicle recognition within a region of interest (ROI) in an image by assuming that the ROI has been detected by a radar sensor. Two classifiers including a support vector machine (SVM) and a neural network have been investigated to verify the effectiveness of the features. We have tested the proposed approaches on real-world video sequences acquired under various weather conditions for a wide range of vehicles and non-vehicles at up to 70 meters. The proposed combination of Legendre moments and Gabor features has yielded a correct classification rate of 99.1% and a false alarm rate of 1.9%. We have compared the proposed features with the over-complete Haar wavelets in the literature
---
paper_title: Vehicle Detection and Tracking in Car Video Based on Motion Model

paper_content:

This paper aims at real-time in-car video analysis to detect and track vehicles ahead for safety, autodriving, and target tracing. This paper describes a comprehensive approach to localizing target vehicles in video under various environmental conditions. The extracted geometry features from the video are continuously projected onto a 1-D profile and are constantly tracked. We rely on temporal information of features and their motion behaviors for vehicle identification, which compensates for the complexity in recognizing vehicle shapes, colors, and types. We probabilistically model the motion in the field of view according to the scene characteristic and the vehicle motion model. The hidden Markov model (HMM) is used to separate target vehicles from the background and track them probabilistically. We have investigated videos of day and night on different types of roads, showing that our approach is robust and effective in dealing with changes in environment and illumination and that real-time processing becomes possible for vehicle-borne cameras.
---
paper_title: A Short Introduction to Boosting

paper_content:

Boosting is a general method for improving the accuracy of any given learning algorithm. This short overview paper introduces the boosting algorithm AdaBoost, and explains the underlying theory of boosting, including an explanation of why boosting often does not suffer from overfitting as well as boosting’s relationship to support-vector machines. Some examples of recent applications of boosting are also described.
---
paper_title: Surf: Speeded up robust features

paper_content:

In this paper, we present a novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features). It approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. ::: ::: This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (in casu, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps. The paper presents experimental results on a standard evaluation set, as well as on imagery obtained in the context of a real-life object recognition application. Both show SURF's strong performance.
---
paper_title: Improving the Generalization Properties of Neural Networks: an Application to Vehicle Detection

paper_content:

In this paper a multilayer feedforward neural network based approach for vehicle detection is proposed. The main idea is to use such network to perform both feature extraction and classification. This simplicity enables real time applications. In order to achieve such capabilities, the network is trained by a new algorithm, proposed in this paper, named minimization of inter-class interference (MCI). Such algorithm aims to create a hidden space (i.e. feature space) where the patterns have a desirable statistical distribution. Regarding the neural architecture, the linear output layer is replaced by the Mahalanobis kernel, in order to improve generalization. Experiments are performed by means of a dataset that includes two standard datasets from Caltech car rear. Finally, disturbed images are used, in order to evaluate the robustness of the neural-network based vehicle detection. The proposed method reveals low miss rate, low false alarm rate and high area under ROC curve. In Matlab environment, the algorithm spends only 3.280e-4 seconds per image. These facts encourage this research line.
---
paper_title: Monocular precrash vehicle detection: features and classifiers

paper_content:

Robust and reliable vehicle detection from images acquired by a moving vehicle (i.e., on-road vehicle detection) is an important problem with applications to driver assistance systems and autonomous, self-guided vehicles. The focus of this work is on the issues of feature extraction and classification for rear-view vehicle detection. Specifically, by treating the problem of vehicle detection as a two-class classification problem, we have investigated several different feature extraction methods such as principal component analysis, wavelets, and Gabor filters. To evaluate the extracted features, we have experimented with two popular classifiers, neural networks and support vector machines (SVMs). Based on our evaluation results, we have developed an on-board real-time monocular vehicle detection system that is capable of acquiring grey-scale images, using Ford's proprietary low-light camera, achieving an average detection rate of 10 Hz. Our vehicle detection algorithm consists of two main steps: a multiscale driven hypothesis generation step and an appearance-based hypothesis verification step. During the hypothesis generation step, image locations where vehicles might be present are extracted. This step uses multiscale techniques not only to speed up detection, but also to improve system robustness. The appearance-based hypothesis verification step verifies the hypotheses using Gabor features and SVMs. The system has been tested in Ford's concept vehicle under different traffic conditions (e.g., structured highway, complex urban streets, and varying weather conditions), illustrating good performance.
---
paper_title: Online Boosting for Vehicle Detection

paper_content:

This paper presents a real-time vision-based vehicle detection system employing an online boosting algorithm. It is an online AdaBoost approach for a cascade of strong classifiers instead of a single strong classifier. Most existing cascades of classifiers must be trained offline and cannot effectively be updated when online tuning is required. The idea is to develop a cascade of strong classifiers for vehicle detection that is capable of being online trained in response to changing traffic environments. To make the online algorithm tractable, the proposed system must efficiently tune parameters based on incoming images and up-to-date performance of each weak classifier. The proposed online boosting method can improve system adaptability and accuracy to deal with novel types of vehicles and unfamiliar environments, whereas existing offline methods rely much more on extensive training processes to reach comparable results and cannot further be updated online. Our approach has been successfully validated in real traffic environments by performing experiments with an onboard charge-coupled-device camera in a roadway vehicle.
---
paper_title: Learning a Family of Detectors via Multiplicative Kernels

paper_content:

Object detection is challenging when the object class exhibits large within-class variations. In this work, we show that foreground-background classification (detection) and within-class classification of the foreground class (pose estimation) can be jointly learned in a multiplicative form of two kernel functions. Model training is accomplished via standard SVM learning. When the foreground object masks are provided in training, the detectors can also produce object segmentations. A tracking-by-detection framework to recover foreground state in video sequences is also proposed with our model. The advantages of our method are demonstrated on tasks of object detection, view angle estimation, and tracking. Our approach compares favorably to existing methods on hand and vehicle detection tasks. Quantitative tracking results are given on sequences of moving vehicles and human faces.
---
paper_title: Cascade object detection with deformable part models

paper_content:

We describe a general method for building cascade classifiers from part-based deformable models such as pictorial structures. We focus primarily on the case of star-structured models and show how a simple algorithm based on partial hypothesis pruning can speed up object detection by more than one order of magnitude without sacrificing detection accuracy. In our algorithm, partial hypotheses are pruned with a sequence of thresholds. In analogy to probably approximately correct (PAC) learning, we introduce the notion of probably approximately admissible (PAA) thresholds. Such thresholds provide theoretical guarantees on the performance of the cascade method and can be computed from a small sample of positive examples. Finally, we outline a cascade detection algorithm for a general class of models defined by a grammar formalism. This class includes not only tree-structured pictorial structures but also richer models that can represent each part recursively as a mixture of other parts.
---
paper_title: Lateral Driving Assistance Using Optical Flow and Scene Analysis

paper_content:

This paper presents a novel algorithm of image-based motion detection for lateral driving assistance. We propose a feature-based motion detection algorithm, which combines optical-flow estimation and road plane segmentation using FOE concept. The developed algorithm can segment moving objects from a changing background efficiently. Both cars and motorcycles on the lateral side of a driving vehicle can be detected. We have realized the system in a stand-alone image processing platform, which can operate independently on a vehicle using a CMOS image sensor and embedded processor. Experimental results show that the proposed methods are effective to segment moving vehicles from a moving background for lateral safety assistance.
---
paper_title: Histograms of oriented gradients for human detection

paper_content:

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.
---
paper_title: Increasing driving safety with a multiple vehicle detection and tracking system using ongoing vehicle shadow information

paper_content:

This paper proposes a vehicle detection and tracking system based on processing monochrome images captured by a single camera. The work has mainly been focused on detecting and tracking vehicles in daylight conditions, viewed from inside a vehicle. Unlike previous work, this approach uses vehicle shadow clues and vehicle edge information to obtain cost effective and fast estimation. The proposed method includes road area finding which has been implemented by a lane detection algorithm to avoid false detections of vehicles caused by the distraction of background objects. Assuming that lanes are successfully detected, vehicle presence inside the road area is hypothesized by using “shadow” as a cue. Hypothesized vehicle locations are verified using vertical edges. After extracting vehicles, the algorithm effectively tracks them using a Kalman filter based tracking algorithm. A vehicle has been instrumented with various sensors for the experiments. Several sequences from real traffic situations have been tested, obtaining highly accurate multiple vehicle detections. Tracking information is used to estimate time-to-collision (TTC) and warn the driver for a possible collision.
---
paper_title: Vision-Based Vehicle Detection System With Consideration of the Detecting Location

paper_content:

In this paper, we propose a vision-based vehicle detection system. We use a method composed of a hypothesis generation (HG) step and a hypothesis verification (HV) step, following the general approach to vision-based vehicle detection systems. In the HG step, the system extracts hypotheses using shadow regions that appear under vehicles. In the HV step, the system classifies feature vectors extracted from hypotheses to determine whether those hypotheses are vehicles. Along with the histogram of oriented gradients (HOG), we propose and implement a new type of feature vector, i.e., HOG symmetry vectors, in this paper. We also propose a new classification method that uses data importance in the HV step. The data importance value is based on the locations of hypotheses to prioritize hypotheses that have greater risks of accident. Experimental results show the strong performance of our proposed system.
---
paper_title: Vehicle Detection by Independent Parts for Urban Driver Assistance

paper_content:

In this paper, we introduce vehicle detection by independent parts (VDIP) for urban driver assistance. In urban environments, vehicles appear in a variety of orientations, i.e., oncoming, preceding, and sideview. Additionally, partial vehicle occlusions are common at intersections, during entry and exit from the camera's field of view, or due to scene clutter. VDIP provides a lightweight robust framework for detecting oncoming, preceding, sideview, and partially occluded vehicles in urban driving. In this paper, we use active learning to train independent-part detectors. A semisupervised approach is used for training part-matching classification, which forms sideview vehicles from independently detected parts. The hierarchical learning process yields VDIP, featuring efficient evaluation and robust performance. Parts and vehicles are tracked using Kalman filtering. The fully implemented system is lightweight and runs in real time. Extensive quantitative analysis on real-world on-road data sets is provided.
---
paper_title: LaRASideCam: a Fast and Robust Vision-Based Blindspot Detection System

paper_content:

While shifting lane on the road, the presence of a car in the blindspot can cause many accidents, since the driver does not always turn his head. Therefore, a blindspot car detection is likely to become an essential part of modern vehicles. We developed a program that detects cars in the particular configuration of blindspot using video data taken from the left or right mirror of a car, using on the one hand edge detection and support vector machine (SVM) learning and on the other hand template matching. This makes this program simple, fast and adaptative thanks to SVM learning. The program only uses basical functions of the Ecole des Mines' Camellia open-source image processing library [1], which is close to Intel's IPL library. Thus the program is easy to adapt to another API; it has already been adapted to an embedded system currently in development at NXP Semiconductors (formerly Philips Semiconductors). The source code was tested using the valgrind code checking tool [3] and was validated on real-world video sequences.
---
paper_title: Rapid object detection using a boosted cascade of simple features

paper_content:

This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the "integral image" which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers. The third contribution is a method for combining increasingly more complex classifiers in a "cascade" which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. The cascade can be viewed as an object specific focus-of-attention mechanism which unlike previous approaches provides statistical guarantees that discarded regions are unlikely to contain the object of interest. In the domain of face detection the system yields detection rates comparable to the best previous systems. Used in real-time applications, the detector runs at 15 frames per second without resorting to image differencing or skin color detection.
---
paper_title: Lateral vehicles detection using monocular high resolution cameras on TerraMax™

paper_content:

Autonomous driving in complex urban environments, including traffic merge, four-ways stop, overtaking, etc., requires a very wide range sensorial capabilities, both in angle and distance. This paper presents a vision system, designed to help merging into traffic on two-ways intersections, and able to provide a long detection distance (over 100 m) for incoming vehicles. The system is made of two high resolution wide angle cameras, each one looking laterally (70 degrees) with respect of the moving direction, performing a specific background subtraction based technique, along with tracking and speed estimation. The system works when the vehicle is stopped at intersections, and is triggered by the high-level vehicle manager. The system has been developed and tested on the Oshkosh Teampsilas vehicle TerraMaxtrade, one of the 11 robots admitted to the DARPA Urban Challenge 2007 Final Event.
---
paper_title: GPU & CPU cooperative accelerated pedestrian and vehicle detection

paper_content:

This paper presents a fast pedestrian and vehicle detection framework that integrates GPU (graphics processing unit) and CPU implementations. We employ the Histograms of Oriented Gradients (HoG) and the Feature Interaction Descriptor (FIND) as object descriptors. FIND describes the high-level properties of an object's appearance by computing pair-wise interactions of adjacent region-level features. We also employ the cascade approach in a sliding-window manner with multi-classifiers specialized for both the direction of a pedestrian and the distance of the pedestrian to a camera installed on a vehicle. Although our detection framework can detect pedestrians and vehicles in images, it wastes computational cost on calculating high-dimensional FIND features. Therefore, to realize real-time processing, we utilize the NVIDIA CUDA framework for pedestrian and vehicle detection. Three parallelization techniques implemented on both a GPU and a CPU are incorporated in our detection framework. As a result, our proposed implementation can perform detection more than 30 times faster than can a conventional implementation running on a CPU. For a 640×480 image, our parallel techniques attain a processing speed of 23.8 fps (42 [ms]) and detects both pedestrians and vehicles.
---
paper_title: On-Road Multivehicle Tracking Using Deformable Object Model and Particle Filter With Improved Likelihood Estimation

paper_content:

This paper proposes a novel method for multivehicle detection and tracking using a vehicle-mounted monocular camera. In the proposed method, the features of vehicles are learned as a deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOGs). The detection algorithm combines both global and local features of the vehicle as a deformable object model. Detected vehicles are tracked through a particle filter, which estimates the particles' likelihood by using a detection scores map and template compatibility for both root and parts of the vehicle while considering the deformation cost caused by the movement of vehicle parts. Tracking likelihoods are iteratively used as a priori probability to generate vehicle hypothesis regions and update the detection threshold to reduce false negatives of the algorithm presented before. Extensive experiments in urban scenarios showed that the proposed method can achieve an average vehicle detection rate of 97% and an average vehicle-tracking rate of 86% with a false positive rate of less than 0.26%.
---
paper_title: Realtime Obstacle Detection and Tracking Based on Constrained Delaunay Triangulation

paper_content:

We introduce a realtime vehicle detection and tracking algorithm for in-vehicle video images. Although various vehicle detection approaches have been proposed, it is difficult to find a fast and reliable algorithm for realtime applications, such as for vehicle collision warning. We introduce a realtime appearance-based vehicle detection approach. It uses constrained Delaunay triangulation to make image evidence collection for multiple overlapping hypotheses computationally efficient. We also propose an integrated detection and tracking approach where redundant detection and tracking can compensate each other's shortcomings. The experiment was performed on various video clips of highways and local roads with various traffic and illumination conditions, and the resulting performance is competitive compared to those of other active sensors
---
paper_title: Realtime Vision Based Multi-Target-Tracking with Particle Filters in Automotive Applications

paper_content:

In the following we present a multi-instance-based multi-target-tracking method using particle filters. We developed a robust and flexible system capable of tracking an unknown and changing number of objects (vehicles) using early-vision image features. The system works in real time.
---
paper_title: An evolutionary optimized vehicle tracker in collaboration with a detection system

paper_content:

In this work a learning algorithm for visual object tracking is presented. As object representation a fast computable set of Haar-like features is used and a weighted correlation is applied for the matching process. The object tracker utilizes the same set of features that is already calculated for object detection and thus it is possible to reuse features for detection and tracking. The feature's weight values are optimized for the tracking purpose by means of evolutionary strategies. Different tests of the object tracker on real-world sequences are presented using vehicles as example objects. Additionally, an object detection system and the integration of the object tracker into that system is described. Besides the system is based on a cascade of boosted classifiers, Haar and Triangle features, an adaptive sliding window and finally a Kalman filter.
---
paper_title: Real-time side vehicle tracking using parts-based boosting

paper_content:

This paper presents a real-time vision-based side vehicle detection system employing a parts-based boosting algorithm. Working at the part level, as opposed to the whole object level, enables a more flexible class representation and allows scenes in which the query object is significantly occluded to be classified. Therefore, a parts-based learning approach is proposed in order to better deal with side vehicle variability, illumination conditions, partial occlusions, and rotations. Most existing boosting learning algorithms usually select weak classifiers by minimizing a cost directly associated with the error rate, where the learned strong classifier may be sub-optimal for applications in terms of error rate. Nevertheless, the proposed Adaboost approach selects weak classifiers by minimizing multiple types of error functions. The idea is to define multiple types of error functions based on current strong classifier and each selected weak classifier results to represent different effects of each weak classifier. Therefore, weak classifiers can be selected with different requirements at the same time to avoid a sub-optimal solution. To reduce system computation, window-based tracking is employed. Moreover, Kalman filtering is used to predict the position of each part of vehicles in the image plane to effectively relocate the tracking windows. Compared with existing approaches, the proposed system appears to be capable of improving system efficiency and accuracy under varying lighting conditions, changing vehicle poses, and in the presence of partial occlusions. Our approach has been successfully validated in real traffic environments by performing experiments with a CCD camera mounted onboard a highway vehicle.
---
paper_title: A General Active-Learning Framework for On-Road Vehicle Recognition and Tracking

paper_content:

This paper introduces a general active-learning framework for robust on-road vehicle recognition and tracking. This framework takes a novel active-learning approach to building vehicle-recognition and tracking systems. A passively trained recognition system is built using conventional supervised learning. Using the query and archiving interface for active learning (QUAIL), the passively trained vehicle-recognition system is evaluated on an independent real-world data set, and informative samples are queried and archived to perform selective sampling. A second round of learning is then performed to build an active-learning-based vehicle recognizer. Particle filter tracking is integrated to build a complete multiple-vehicle tracking system. The active-learning-based vehicle-recognition and tracking (ALVeRT) system has been thoroughly evaluated on static images and roadway video data captured in a variety of traffic, illumination, and weather conditions. Experimental results show that this framework yields a robust efficient on-board vehicle recognition and tracking system with high precision, high recall, and good localization.
---
paper_title: Monocular Video serves RADAR-based Emergency Braking

paper_content:

Much work was carried out recently for emergency braking based on radar signals. The key step for emergency braking is the reliable detection of obstacles. Moving objects are verified as such by tracking the radar signal. However, discarding so-called phantom objects remains a challenge for stationary objects. This leads to the question of sensor fusion for more reliable verification of obstacles. In this paper we propose a novel method using a monocular camera, such as the night view camera in the Mercedes S class. Our two goals in this paper are the verification of obstacles and the detection of obstacle boundaries. This allows analysis of the situation for carrying out emergency braking. The verification of obstacles is done by analyzing the scaling of obstacles as they get closer to the camera. The perspective image motion of the ground plane serves as a counter hypothesis to detect phantom objects. Obstacle boundaries are found by graph cut segmentation on these two motion fields.
---
paper_title: Vehicle detection using an extended Hidden Random Field model

paper_content:

Prevent collision with other vehicles is crucial for developing advanced driver assistance systems. Vision-based approaches for vehicle detection attract more attention than those using other sensors. In this study, we address the problem of detecting front vehicles in still images. Unlike traditional methods which mainly based on the holistic appearance of vehicles, we adopted a local part based model. We extended the Hidden Random Field (HRF) model to incorporate logistic regression classifiers into unary potentials. The proposed model was trained and tested on a set of real images captured by an on-board camera. The results showed that the effectiveness of the approach, and a better performance could be found when the vehicle was occluded by other vehicles.
---
paper_title: Rear Vehicle Detection and Tracking for Lane Change Assist

paper_content:

A monocular vision based rear vehicle detection and tracking system is presented for Lane Change Assist (LCA), which does not need road boundary and lane information. Our algorithm extracts regions of interest (ROI) using the shadow underneath a vehicle, and accurately localizes vehicle regions in ROI by vehicle features such as symmetry, edge and shadow underneath vehicles. The algorithm realizes vehicle verification by combining knowledge-based and learning-based methods. During vehicle tracking, templates are dynamically created on-line, tracking window is adaptively adjusted with motion estimation, and confidence is determined for tracked vehicle. The algorithm was tested under various traffic scenes at different daytime, the result illustrated good performance.
---
paper_title: Active learning for on-road vehicle detection: a comparative study

paper_content:

In recent years, active learning has emerged as a powerful tool in building robust systems for object detection using computer vision. Indeed, active learning approaches to on-road vehicle detection have achieved impressive results. While active learning approaches for object detection have been explored and presented in the literature, few studies have been performed to comparatively assess costs and merits. In this study, we provide a cost-sensitive analysis of three popular active learning methods for on-road vehicle detection. The generality of active learning findings is demonstrated via learning experiments performed with detectors based on histogram of oriented gradient features and SVM classification (HOG---SVM), and Haar-like features and Adaboost classification (Haar---Adaboost). Experimental evaluation has been performed on static images and real-world on-road vehicle datasets. Learning approaches are assessed in terms of the time spent annotating, data required, recall, and precision.
---
paper_title: Support-vector networks

paper_content:

Thesupport-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data.High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
---
paper_title: Real-time vehicle detection using parts at intersections

paper_content:

In this study, we propose a novel, lightweight approach to real-time detection of vehicles using parts at intersections. Intersections feature oncoming, preceding, and cross traffic, which presents challenges for vision-based vehicle detection. Ubiquitous partial occlusions further complicate the vehicle detection task, and occur when vehicles enter and leave the camera's field of view. To confront these issues, we independently detect vehicle parts using strong classifiers trained with active learning. We match part responses using a learned matching classification. The learning process for part configurations leverages user input regarding full vehicle configurations. Part configurations are evaluated using Support Vector Machine classification. We present a comparison of detection results using geometric image features and appearance-based features. The full vehicle detection by parts has been evaluated on real-world data, runs in real time, and shows promise for future work in urban driver assistance.
---
paper_title: Automatic Vehicle Detection Using Local Features—A Statistical Approach

paper_content:

This paper develops a novel statistical approach for automatic vehicle detection based on local features that are located within three significant subregions of the image. In the detection process, each subregion is projected onto its associated eigenspace and independent basis space to generate a principal components analysis (PCA) weight vector and an independent component analysis (ICA) coefficient vector, respectively. A likelihood evaluation process is then performed based on the estimated joint probability of the projection weight vectors and the coefficient vectors of the subregions with position information. The use of subregion position information minimizes the risk of false acceptances, whereas the use of PCA to model the low-frequency components of the eigenspace and ICA to model the high-frequency components of the residual space improves the tolerance of the detection process toward variations in the illumination conditions and vehicle pose. The use of local features not only renders the system more robust toward partial occlusions but also reduces the computational overhead. The computational costs are further reduced by eliminating the requirement for an ICA residual image reconstruction process and by computing the likelihood probability using a weighted Gaussian mixture model, whose parameters and weights are iteratively estimated using an expectation-maximization algorithm.
---
paper_title: Vehicle detection combining gradient analysis and AdaBoost classification

paper_content:

This paper presents a real-time vision-based vehicle's rear detection system using gradient based methods and Adaboost classification, for ACC applications. Our detection algorithm consists of two main steps: gradient driven hypothesis generation and appearance based hypothesis verification. In the hypothesis generation step, possible target locations are hypothesized. This step uses an adaptive range-dependant threshold and symmetry for gradient maxima localization. Appearance-based hypothesis validation verifies those hypothesis using AdaBoost for classification with illumination independent classifiers. The monocular system was tested under different traffic scenarios (e.g., simply structured highway, complex urban environments, varying lightening conditions), illustrating good performance.
---
paper_title: Vehicle Detection under Various Lighting Conditions by Incorporating Particle Filter

paper_content:

We propose an automatic system to detect preceding vehicles on the highway under various lighting and different weather conditions based on the computer vision technologies. To adapt to different characteristics of vehicle appearance at daytime and nighttime, four cues including underneath, vertical edge, symmetry and taillight are fused for the preceding vehicle detection. By using particle filter with four cues through the processes including initial sampling, propagation, observation and cue fusion and evaluation, particle filter accurately generates the vehicle distribution. Thus, the proposed system can successfully detect and track preceding vehicles and be robust to different lighting conditions. Unlike normal particle filter focuses on a single target distribution in a discrete state space, we detect multiple vehicles with particle filter through a high-level tracking strategy using clustering technique called basic sequential algorithmic scheme (BSAS). Finally, experimental results for several videos from different scenes are provided to demonstrate the effectiveness of our proposed system.
---
paper_title: Unsupervised sub-categorization for object detection: Finding cars from a driving vehicle

paper_content:

We present a novel algorithm for unsupervised subcategorization of an object class, in the context of object detection. Dividing the detection problem into smaller subproblems simplifies the object vs. background classification.
---
paper_title: Symmetry-based monocular vehicle detection system

paper_content:

In this paper, we describe the development of a symmetry-based vehicle detection system. The system uses a single forward looking camera to capture the road scene. Vehicles are detected based on their edges and symmetrical characteristics. A method to extract the symmetric regions in the image using a multi-sized window and clustering technique is introduced. We hypothesize the vehicle’s locations in the image from the detected symmetric regions and the regions are then further processed to enhance their symmetrical edges. A bounding box of a vehicle is detected from the projection maps of the enhanced vertical and horizontal edges. The hypothesized vehicles are then verified using a two-class classifier, which consists of an edge oriented histogram (EOH) feature extractor and a support vector machine (SVM). Once a vehicle is verified, a tracking process based on a Kalman filter and a reliability point system is used to track the movement of the vehicle in consecutive video frames. The system was successfully implemented and tested on a standard PC. Experimental results on live video feed and pre-recorded video sequences for various road scenes showed that the system is able to detect multiple vehicles in real time.
---
paper_title: Real-time vision-based preceding vehicle tracking and recognition

paper_content:

This paper proposes a preceding vehicle tracking and detection system which includes road area finding, vehicle footprint extraction, and vehicle bounding box extraction. Our method detects the footprint of the vehicle inside the road area to avoid the false detection of delineators. By extracting the preceding vehicle, we effectively track the vehicles, and detect the passing car and distant car. Then, we also propose the distance measure formula. Through camera calibration process and vanishing point detection process, we can calculate the distance to the preceding vehicle accurately.
---
paper_title: Vehicle Surround Capture: Survey of Techniques and a Novel Omni-Video-Based Approach for Dynamic Panoramic Surround Maps

paper_content:

Awareness of what surrounds a vehicle directly affects the safe driving and maneuvering of an automobile. This paper focuses on the capture of vehicle surroundings using video inputs. Surround information or maps can help in studies of driver behavior as well as provide critical input in the development of effective driver assistance systems. A survey of literature related to surround analysis is presented, emphasizing detecting objects such as vehicles, pedestrians, and other obstacles. Omni cameras, which give a panoramic view of the surroundings, can be useful for visualizing and analyzing the nearby surroundings of the vehicle. The concept of Dynamic Panoramic Surround (DPS) map that shows the nearby surroundings of the vehicle and detects the objects of importance on the road is introduced. A novel approach for synthesizing the DPS using stereo and motion analysis of video images from a pair of omni cameras on the vehicle is developed. Successful generation of the DPS in experimental runs on an instrumented vehicle test bed is demonstrated. These experiments prove the basic feasibility and show promise of omni-camera-based DPS capture algorithm to provide useful semantic descriptors of the state of moving vehicles and obstacles in the vicinity of a vehicle
---
paper_title: Multi-class Object Detection in Vision Systems Using a Hierarchy of Cascaded Classifiers

paper_content:

Boosted cascades for fast and reliable object detection for one object class were introduced by Viola et al. (2001). Using this scheme for multi-class detection requires parallel usage of multiple cascades and increases computation time. We present an extension to the cascade which examines multiple classes jointly in the first stages of the cascade. Adaboost is selecting common features for all considered object classes, which are then computed only once and thus reduce the computation time of the overall system. We also show how to define the search-window, as it needs to be adjusted to the specific objects. The multi-class capable cascade is applied to traffic scenes on rural roads where pedestrians and reflection posts are detected
---
paper_title: Vision-based detection and labelling of multiple vehicle parts

paper_content:

This paper presents a method for the visual detection of parts of interest on the outer surface of vehicles. The proposed method combines computer vision techniques and machine learning algorithms to process images of lateral views of automobiles. The aim of this approach is to determine the location of a set of car parts in ordinary scenes. The approach can be used in the intelligent transportation industry to construct advanced monitoring and security applications. The key contributions of this work are the introduction of a methodology to locate multiple patterns in cluttered scenes of vehicles which makes use of a probabilistic technique to reduce false detection, and the proposal of a method for inferring the location of regions of interest using a priori knowledge. The results demonstrate excellent performance in the task of detecting up to fourteen different car parts over a vehicle.
---
paper_title: Vehicle tracking and motion prediction in complex urban scenarios

paper_content:

The recognition of potentially hazardous situations on road intersections is an indispensable skill of future driver assistance systems. In this context, this study focuses on the task of vehicle tracking in combination with a long-term motion prediction (1–2 s into the future) in a dynamic scenario. A motion-attributed stereo point cloud obtained using computationally efficient feature-based methods represents the scene, relying on images of a stereo camera system mounted on a vehicle. A two-stage mean-shift algorithm is used for detection and tracking of the traffic participants. A hierarchical setup depending on the history of the tracked object is applied for prediction. This includes prediction by optical flow, a standard kinematic prediction, and a particle filter based motion pattern method relying on learned object trajectories. The evaluation shows that the proposed system is able to track the road users in a stable manner and predict their positions at least one order of magnitude more accurately than a standard kinematic prediction method.
---
paper_title: Fast detection of moving objects in complex scenarios

paper_content:

More than one third of all traffic accidents with injuries occur in urban areas, especially at intersections. A suitable driver assistance system for such complex situations requires the understanding of the scene, in particular a reliable detection of other moving traffic participants. This contribution shows how a robust and fast detection of relevant moving objects is obtained by a smart combination of stereo vision and motion analysis. This approach, called 6D Vision, estimates location and motion of pixels simultaneously which enables the detection of moving objects on a pixel level. Using a Kalman filter attached to each tracked pixel, the algorithm propagates the current interpretation to the next image. In addition, a Kalman filter based ego-motion compensation is described that takes advantage of the 6D information. This precise information enables us to discriminate between static and moving objects exactly and to obtain a better prediction. This speeds up tracking and a real-time implementation is achieved. Examples of critical situations in urban areas exhibit the potential of the 6D Vision concept which can also be extended to robotics applications.
---
paper_title: Object tracking from stereo sequences using particle filter

paper_content:

In this paper we present a vehicle tracking particle filter system based on gray histogram and sparse optical flow detection in stereo images. The proposed approach is based on the fact that for 2D tracked features we can compute their 3D correspondences, which are used for particle filter tracking improvement. The goal of this paper is to show how vision based particle filter tracking, optical flow and stereovision can be integrated to work together in order to achieve a robust object tracking algorithm.
---
paper_title: Parametric ego-motion estimation for vehicle surround analysis using an omnidirectional camera

paper_content:

Abstract.Omnidirectional cameras that give a 360° panoramic view of the surroundings have recently been used in many applications such as robotics, navigation, and surveillance. This paper describes the application of parametric ego-motion estimation for vehicle detection to perform surround analysis using an automobile-mounted camera. For this purpose, the parametric planar motion model is integrated with the transformations to compensate distortion in omnidirectional images. The framework is used to detect objects with independent motion or height above the road. Camera calibration as well as the approximate vehicle speed obtained from a CAN bus are integrated with the motion information from spatial and temporal gradients using a Bayesian approach. The approach is tested for various configurations of an automobile-mounted omni camera as well as a rectilinear camera. Successful detection and tracking of moving vehicles and generation of a surround map are demonstrated for application to intelligent driver support.
---
paper_title: Tracking with Stereo-vision System for Low Speed Following Applications

paper_content:

Research in adaptative cruise control (ACC) is currently one of the most important topics in the field of intelligent transportation systems. The main challenge is to perceive the environment, especially at low speed. In this paper, we present a novel approach to track the 3D trajectory and speed of the obstacles and the surrounding vehicles through a stereo-vision system. This tracking method extends the classical patch-based Lucas-Kanade algorithm by integrating the geometric constraints of the stereo system into the motion model: the epipolar constraint, which enforces the tracked patches to remain on the epipolar lines, and the magnification constraint, which links the disparity of the tracked patches to the apparent size of these patches. We report experimental results on simulated and real data showing the improvement in accuracy and robustness of our algorithm compared to the classical Lucas-Kanade tracker.
---
paper_title: Real-time dynamic environment perception in driving scenarios using difference fronts

paper_content:

The environment representation is one of the main challenges of autonomous navigation. In the case of complex driving environments such as crowded city traffic scenarios, achieving satisfactory results becomes even more difficult. In this paper we propose a real-time solution for two main issues of advanced driver assistance systems: unstructured environment representation and extraction of dynamic properties of traffic participants. For the real-time environment representation we propose a solution to extract object delimiters from the traffic scenes and represent them as polygonal models. In order to track dynamic entities, an intermediate evidence map named “Stereo Temporal Difference Map” is proposed. This difference map is computed by comparing the occupancy of a cell between two consecutive frames. Based on the Stereo Temporal Difference Map information, difference fronts are extracted and are subjected to a particle based filtering mechanism. Finally, the provided dynamic features are associated to the extracted polygonal models. The result is a more compact representation of the dynamic environment.
---
paper_title: Real-time vehicle detection and tracking using stereo vision and multi-view AdaBoost

paper_content:

We propose a multi-layer, real-time vehicle detection and tracking system using stereo vision, multi-view AdaBoost detectors, and optical flow. By adopting a ground plane estimate extracted from stereo information, we generate a sparse set of hypotheses and apply trained AdaBoost classifiers in addition to fast disparity histogramming, for Hypothesis Verification (HV) purposes. Our tracking system employs one Kalman filter per detected vehicle and motion vectors from optical flow, as a means to increase its robustness. An acceptable detection rate with few false positives is obtained at 25 fps with generic hardware.
---
paper_title: Tracking multiple objects in urban traffic environments using dense stereo and optical flow

paper_content:

Stereo vision based sensors provide large amounts of data, a fact which is advantageous when trying to extract semantic information about the imaged scene. However, this data is corrupted by errors, caused especially by the uncertainties in the stereo reconstruction process. Temporal information can be used in order to minimize these errors. This paper presents an advanced object model, a novel association mechanism and the design of a Kalman filter based tracking algorithm, for tracking multiple objects, in complex, urban traffic scenarios.
---
paper_title: Tomographical scene reconstruction in the active safety car project

paper_content:

Tomographical reconstruction algorithms can be applied to camera based mesaurements and thus reconstruct the scenery without knowledge about included objects. The latter is interesting in the domain of driver assistance systems that have to monitor the driveway independently from a priori knowledge about possibly appearing objects. The paper presents tomographical background information, the transfer from radiographing to visual light rays and its negative impacts, and some first result obtained by applying the presented algorithm to images of a fromn view camera.
---
paper_title: On-board robust vehicle detection and tracking using adaptive quality evaluation

paper_content:

This paper presents a robust method for real-time vehicle detection and tracking in dynamic traffic environments. The proposed strategy aims to find a trade-off between the robustness shown by time-uncorrelated detection techniques and the speed-up obtained with tracking algorithms. It combines both advantages by continuously evaluating the quality of the tracking results along time and triggering new detections to restart the tracking process when quality falls behind a certain quality requirement. Robustness is also ensured within the tracking algorithm with an outlier rejection stage and the use of stochastic filtering. Several sequences from real traffic situations have been tested, obtaining highly accurate multiple vehicle detections.
---
paper_title: Vehicle Detection and Tracking in Car Video Based on Motion Model

paper_content:

This paper aims at real-time in-car video analysis to detect and track vehicles ahead for safety, autodriving, and target tracing. This paper describes a comprehensive approach to localizing target vehicles in video under various environmental conditions. The extracted geometry features from the video are continuously projected onto a 1-D profile and are constantly tracked. We rely on temporal information of features and their motion behaviors for vehicle identification, which compensates for the complexity in recognizing vehicle shapes, colors, and types. We probabilistically model the motion in the field of view according to the scene characteristic and the vehicle motion model. The hidden Markov model (HMM) is used to separate target vehicles from the background and track them probabilistically. We have investigated videos of day and night on different types of roads, showing that our approach is robust and effective in dealing with changes in environment and illumination and that real-time processing becomes possible for vehicle-borne cameras.
---
paper_title: Moving Obstacle Detection using Monocular Vision

paper_content:

This paper proposes a method for detecting moving obstacles on roads, by using a vehicle mounted monocular camera. To detect various moving obstacles, such as vehicles and pedestrians, the ego-motion of the vehicle is initially estimated from images captured by the camera. There are two problems in ego-motion estimation. Firstly, a typical road scene contains moving obstacles. This causes false estimation of the ego-motion. Secondly, roads possess fewer features, when compared to the number associated with background structures. This reduces the accuracy of ego-motion estimation. In our approach, the ego-motion is estimated from the correspondences of dispersed feature points extracted from various regions other than those that contain moving obstacles. After estimating the ego-motion, any moving obstacles are detected by tracking the feature points over consecutive frames. In our experiments, it has been shown that the proposed method is able to detect moving obstacles.
---
paper_title: Modeling and Tracking the Driving Environment With a Particle-Based Occupancy Grid

paper_content:

Modeling and tracking the driving environment is a complex problem due to the heterogeneous nature of the real world. In many situations, modeling the obstacles and the driving surfaces can be achieved by the use of geometrical objects, and tracking becomes the problem of estimating the parameters of these objects. In the more complex cases, the scene can be modeled and tracked as an occupancy grid. This paper presents a novel occupancy grid tracking solution based on particles for tracking the dynamic driving environment. The particles will have a dual nature-they will denote hypotheses, as in the particle filtering algorithms, but they will also be the building blocks of our modeled world. The particles have position and speed, and they can migrate in the grid from cell to cell, depending on their motion model and motion parameters, but they will be also created and destroyed using a weighting-resampling mechanism that is specific to particle filtering algorithms. The tracking algorithm will be centered on particles, instead of cells. An obstacle grid derived from processing a stereovision-generated elevation map is used as measurement information, and the measurement model takes into account the uncertainties of the stereo reconstruction. The resulting system is a flexible real-time tracking solution for dynamic unstructured driving environments.
---
paper_title: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography

paper_content:

A new paradigm, Random Sample Consensus (RANSAC), for fitting a model to experimental data is introduced. RANSAC is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors. A major portion of this paper describes the application of RANSAC to the Location Determination Problem (LDP): Given an image depicting a set of landmarks with known locations, determine that point in space from which the image was obtained. In response to a RANSAC requirement, new results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form. These results provide the basis for an automatic system that can solve the LDP under difficult viewing
---
paper_title: Moving vehicle detection by optimal segmentation of the Dynamic Stixel World

paper_content:

The reliable detection of moving objects from a moving observer is one of the most challenging and important tasks for driver assistance and safety systems. Modern sensors such as Lidar, Imaging Radar or Stereo Vision deliver range data plus longitudinal motion (Radar) or even full 3D-motion (space-time vision). Based on this data, moving objects have to be separated from the static background to be able to determine their pose and motion state. Usually, heuristics are applied to cluster the data. In order to find the most probable segmentation, we formulate the task as a hypotheses testing problem that allows taking into account various constraints and assumptions simultaneously. We show that the optimal segmentation can be efficiently found by means of dynamic programming, for an arbitrary number of objects in the scene. In this paper we concentrate on the segmentation of space-time data obtained from stereo image sequences. The vision-based depth and motion information is transferred into so called Stixels, a very compact representation of 3D scenes that can also be applied to Lidar or Radar data. It turns out that our optimal segmentation is more robust w.r.t. noisy and erroneous data.
---
paper_title: Overtaking Vehicle Detection Using Dynamic and Quasi-Static Background Modeling

paper_content:

Robust and reliable detection of overtaking vehicles is an important component of any on-board driver assistance system. Optical flow, with the abundant motion information present in image sequences, has been studied extensively for vehicle detection. However, using dense optical ?ow for vehicle detection is sensitive to shocks and vibrations of the mobile camera; image outliers caused by illumination changes; and high computational complexity. To improve vehicle detection performance and reduce computational complexity, we propose an efficient and robust methodology for overtaking vehicle detection based on homogeneous sparse optical flow and eigenspace modeling. Specifically, our method models the background into dynamic and quasi-static regions. Instead of using dense optical flow to model the dynamic parts of the background, we employ homogeneous sparse optical flow, which makes detection more robust to camera shocks and vibrations. Moreover, to make detection robust to illumination changes, we employ a blockbased eigenspace approach to represent quasi-static regions in the background. A region-based hysteresis-thresholding approach, augmented by a localized spatial segmentation procedure, attains a good tradeoff between true detections and false positives. The proposed methodology has been evaluated using challenging traf?c scenes illustrating good performance.
---
paper_title: Detection of close cut-in and overtaking vehicles for driver assistance based on planar parallax

paper_content:

Image processing is widely considered an essential part of future driver assistance systems. This paper presents a motion-based vision approach to initial detection of static and moving objects observed by a monocular camera attached to a moving observer. The underlying principle is based on parallax flow induced by all non-planar static or moving object of a 3D scene that is determined from optical flow measurements. Initial object hypotheses are created in regions containing significant parallax flow. The significance is determined from planar parallax decomposition automatically. Furthermore, we propose a separation of detected image motion into three hypotheses classes, namely coplanar, static and moving regions. To achieve a high degree of robustness and accuracy in real traffic situations some key processing steps are supported by the data of inertial sensors rigidly attached to our vehicle. The proposed method serves as a visual short-range surveillance module providing instantaneous object candidates to a driver assistance system. Our experiments and simulations confirm the feasibility and robustness of the detection method even in complex urban environment.
---
paper_title: Lateral Driving Assistance Using Optical Flow and Scene Analysis

paper_content:

This paper presents a novel algorithm of image-based motion detection for lateral driving assistance. We propose a feature-based motion detection algorithm, which combines optical-flow estimation and road plane segmentation using FOE concept. The developed algorithm can segment moving objects from a changing background efficiently. Both cars and motorcycles on the lateral side of a driving vehicle can be detected. We have realized the system in a stand-alone image processing platform, which can operate independently on a vehicle using a CMOS image sensor and embedded processor. Experimental results show that the proposed methods are effective to segment moving vehicles from a moving background for lateral safety assistance.
---
paper_title: Real-time modeling of dynamic environments in traffic scenarios using a stereo-vision system

paper_content:

Real-time modeling of dynamic environments is one of the most demanding research problems in the field of driving assistance systems. The representation module may be affected by several factors such as occlusions, unpredictable nature of the traffic participants, wrong associations or noisy measurements. In this paper we propose two different methods for real-time modeling of dynamic environments. Both motion estimation techniques are vision-based and rely on information provided by a Digital Elevation Map. The first approach consists in determining the differences between the previous and current frames. These differences are then used for computing the speeds of the traffic participants. The second motion estimation technique consists in using a fast pairwise alignment of object delimiters that are extracted by radial scanning of the Elevation Map. The final result is a more compact polygonal map with associated static and dynamic features.
---
paper_title: Lateral vehicles detection using monocular high resolution cameras on TerraMax™

paper_content:

Autonomous driving in complex urban environments, including traffic merge, four-ways stop, overtaking, etc., requires a very wide range sensorial capabilities, both in angle and distance. This paper presents a vision system, designed to help merging into traffic on two-ways intersections, and able to provide a long detection distance (over 100 m) for incoming vehicles. The system is made of two high resolution wide angle cameras, each one looking laterally (70 degrees) with respect of the moving direction, performing a specific background subtraction based technique, along with tracking and speed estimation. The system works when the vehicle is stopped at intersections, and is triggered by the high-level vehicle manager. The system has been developed and tested on the Oshkosh Teampsilas vehicle TerraMaxtrade, one of the 11 robots admitted to the DARPA Urban Challenge 2007 Final Event.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: Critical Motion Detection of Nearby Moving Vehicles in a Vision-Based Driver-Assistance System

paper_content:

Driving always involves risk. Various means have been proposed to reduce the risk. Critical motion detection of nearby moving vehicles is one of the important means of preventing accidents. In this paper, a computational model, which is referred to as the dynamic visual model (DVM), is proposed to detect critical motions of nearby vehicles while driving on a highway. The DVM is motivated by the human visual system and consists of three analyzers: 1) sensory analyzers, 2) perceptual analyzers, and 3) conceptual analyzers. In addition, a memory, which is called the episodic memory, is incorporated, through which a number of features of the system, including hierarchical processing, configurability, adaptive response, and selective attention, are realized. A series of experimental results with both single and multiple critical motions are demonstrated and show the feasibility of the proposed system.
---
paper_title: Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection

paper_content:

A new approach for the detection of the road surface and obstacles is presented. The high accuracy of the method allows the detection of traffic isles as a distinct class. The 3-D data inferred from dense stereo are transformed into a rectangular digital elevation map (DEM). Two classifiers are proposed, namely, density based and road surface based. The density-based obstacle classifier marks DEM cells as road or obstacles, using the density of 3-D points as a criterion. A quadratic road surface model is initially fitted by a random sample consensus (RANSAC) approach to the region in front of the ego vehicle. A region growing-like process refines this primary solution, driven by the 3-D uncertainty model of the stereo sensor. A robust global solution for the road surface is obtained. The road surface is used for discrimination between road, traffic isle, and obstacle points. Fusion and error filtering is performed on the results of the two classifiers. The proposed real-time algorithm was evaluated in an urban scenario and can be used in complex applications from collision avoidance to path planning.
---
paper_title: Object flow: A descriptor for classifying traffic motion

paper_content:

We present and evaluate a novel scene descriptor for classifying urban traffic by object motion. Atomic 3D flow vectors are extracted and compensated for the vehicle's egomotion, using stereo video sequences. Votes cast by each flow vector are accumulated in a bird's eye view histogram grid. Since we are directly using low-level object flow, no prior object detection or tracking is needed. We demonstrate the effectiveness of the proposed descriptor by comparing it to two simpler baselines on the task of classifying more than 100 challenging video sequences into intersection and non-intersection scenarios. Our experiments reveal good classification performance in busy traffic situations, making our method a valuable complement to traditional approaches based on lane markings.
---
paper_title: Vehicle detection and tracking using homography-based plane rectification and particle filtering

paper_content:

This paper presents a full system for vehicle detection and tracking in non-stationary settings based on computer vision. The method proposed for vehicle detection exploits the geometrical relations between the elements in the scene so that moving objects (i.e., vehicles) can be detected by analyzing motion parallax. Namely, the homography of the road plane between successive images is computed. Most remarkably, a novel probabilistic framework based on Kalman filtering is presented for reliable and accurate homography estimation. The estimated homography is used for image alignment, which in turn allows to detect the moving vehicles in the image. Tracking of vehicles is performed on the basis of a multidimensional particle filter, which also manages the exit and entries of objects. The filter involves a mixture likelihood model that allows a better adaptation of the particles to the observed measurements. The system is specially designed for highway environments, where it has been proven to yield excellent results.
---
paper_title: Driving assistance system based on the detection of head-on collisions

paper_content:

An artificial vision system for vehicles is proposed in this article to alert drivers of potential head on collisions. It is capable of detecting any type of frontal collision from any type of obstacle that may present itself in a vehiclepsilas path. The system operates based on a sequence of algorithms whose images are recorded on a camera located in the moving vehicle, resulting in the calculation of Time-to-Contact taken from an analysis of the optical flow, which allows the vehiclepsilas movement to be studied from a sequence of images.
---
paper_title: Monocular road mosaicing for urban environments

paper_content:

Marking-based lane recognition requires an unobstructed view onto the road. In practice however, heavy traffic often constrains the visual field, especially in urban scenarios such as urban crossroads. In this paper we present a novel approach to road mosaicing for dynamic environments. Our method is based on a multistage registration procedure and uses blending techniques. We show that under modest assumptions accurate registration is possible from monocular image sequences. We further demonstrate that fusing visual information from previous frames into the current view can greatly extend the camera's field of view.
---
paper_title: Mapping in dynamic environments using stereo vision

paper_content:

Next generation driver assistance systems demand a precise perception of the vicinity of the vehicle. Sensor readings are usually harnessed to gain knowledge of all moving and stationary obstacles. Commonly two paradigms are followed. Stationary environments are well modeled by non-parametric occupancy grids. Contrary, moving objects require a tracking and are not well suited for grid-based techniques. However, tracking objects requires some prior knowledge and parameterization which is inferior for unstructured cluttered obstacles. Herein we augment a grid-based mapping method designed for static environments with object tracking hence complementing both approaches. To this end we classify regions of stereo images into moving and stationary parts. The stationary part is fused in our static grid whereas moving parts are tracked yielding reliable motion estimates. The classifier used to distinguish moving from stationary parts is based on a Sequential Probability Ratio Test (SPRT), a model selection method which blends well into the tracking architecture. Thereby we achieve real-time operability on modest computing hardware.
---
paper_title: An iterative image registration technique with an application to stereo vision

paper_content:

Image registration finds a variety of applications in computer vision. Unfortunately, traditional image registration techniques tend to be costly. We present a new image registration technique that makes use of the spatial intensity gradient of the images to find a good match using a type of Newton-Raphson iteration. Our technique is taster because it examines far fewer potential matches between the images than existing techniques Furthermore, this registration technique can be generalized to handle rotation, scaling and shearing. We show how our technique can be adapted tor use in a stereo vision system.
---
paper_title: Probabilistic representation of the uncertainty of stereo-vision and application to obstacle detection

paper_content:

Stereo-vision is extensively used for intelligent vehicles, mainly for obstacle detection, as it provides a large amount of data. Many authors use it as a classical 3D sensor which provides a large tri-dimensional cloud of metric measurements, and apply methods usually designed for other sensors, such as clustering based on a distance. For stereo-vision, the measurement uncertainty is related to the range. For medium to long range, often necessary in the field of intelligent vehicles, this uncertainty has a significant impact, limiting the use of this kind of approaches. On the other hand, some authors consider stereo-vision more like a vision sensor and choose to directly work in the disparity space. This provides the ability to exploit the connectivity of the measurements, but roughly takes into consideration the actual size of the objects. In this paper, we propose a probabilistic representation of the specific uncertainty for stereo-vision, which takes advantage of both aspects - distance and disparity. The model is presented and then applied to obstacle detection, using the occupancy grid framework. For this purpose, a computationally-efficient implementation based on the u-disparity approach is given.
---
paper_title: Detection of independently moving objects through stereo vision and ego-motion extraction

paper_content:

Vision-based autonomous vehicles must face numerous challenges in order to be effective in practical areas. Among these lies the detection and localization of independent-moving objects, so as to track or avoid them. In this paper a method that address this particular issue is presented. Information from stereo and motion is used to extract the ego-motion of the vehicle. Known defects of this estimation are exploited to detect independent-moving obstacles. This method allows an early and reliable detection, even for objects partially occluded. Besides, it highlights the errors in the disparity map, which can be used, in future works, to correct depth-estimation, through motion-estimation.
---
paper_title: Real-time obstacle detection in complex scenarios using dense stereo vision and optical flow

paper_content:

Mobile robots as well as tomorrows intelligent vehicles acting in complex dynamic environments must be able to detect both static and moving obstacles. In intersections or crowded urban areas this task proves to be highly demanding. Stereo vision has been extensively used for this task, as it provides a large amount of data. Since it does not reveal any motion information, static and dynamic objects immediately next to each other, or closely positioned obstacles moving in different directions are often merged into a single obstacle. In this paper we address these problems through a powerful fusion between 3D position information delivered by the stereo sensor and 3D motion information, derived from optical flow, in a depth-adaptive occupancy grid. The proposed model is presented and then applied for determining obstacle localization, orientation and speed.
---
paper_title: A Visibility-Based Approach for Occupancy Grid Computation in Disparity Space

paper_content:

Occupancy grids are a very convenient tool for environment representation in robotics. This paper will detail a novel approach for computing occupancy grids from stereo vision and show its application to intelligent vehicles. In the proposed approach, occupancy is initially computed directly in the stereoscopic sensor's disparity space. The calculation formally accounts for the detection of obstacles and road pixels in disparity space, as well as partial occlusions in the scene. In the second stage, this disparity-space occupancy grid is transformed into a Cartesian space occupancy grid to be used by subsequent applications. This transformation includes spatial and temporal filtering. The proposed method is designed to easily be processed in parallel. Consequently, we chose to implement it on a graphics processing unit, which allows real-time processing for demanding applications. In this paper, we present this method, and we propose an application to the problem of perception in a road environment. Results are presented with real road data, qualitatively comparing this approach with other methods.
---
paper_title: Detection of road obstacles using dynamic programming for remapped stereo images to a top-view

paper_content:

This paper proposes an artificial vision algorithm capable of extracting road vehicles in front of a stereo system as well as determining the distance between the stereo system and the detected vehicles. The algorithm transforms stereo perspective images into a virtual top-view. Using edge information of the transformed stereo images, polar accumulation functions (PAFs) are formed. Two processes called spatial and temporal peak joining are introduced to search for corresponding peaks between PAFs based on dynamic programming, which solves the typical stereo correspondence problem without a disparity computation. The proposed algorithm has been proven to be successful through a lot of experiments in various environments.
---
paper_title: Using obstacles and road pixels in the disparity-space computation of stereo-vision based occupancy grids

paper_content:

Occupancy grids have been used for a variety of applications in the field of robotics. These grids have typically been created based on data provided by range sensors such as laser or ultrasound. Current practice is to create the grids based on a probabilistic sensor model such as [1]. The use of stereovision to create occupancy grids is less common. This paper will detail a novel approach to compute occupancy grids, as applied to intelligent vehicles. Occupancy is initially computed directly in the stereoscopic sensor's disparity space, allowing the handling of occlusions in the observed area. It is also computationally efficient, since it uses the u-disparity approach to avoid processing a large point cloud. The occupancy calculation formally accounts for the detection of obstacles and the road in disparity space, as well as partial occlusions in the scene. In a second stage, this disparity-space occupancy grid is transformed into a Cartesian space occupancy grid to be used by subsequent applications. This transformation includes a filtering step to reduce discretization effects and explicitly account for the relation between range and uncertainty in stereoscopic data. In this paper, we present the method and show the results obtained with real road data.
---
paper_title: Reliable Detection of Overtaking Vehicles Using Robust Information Fusion

paper_content:

Early detection of overtaking vehicles is an important task for vision-based driver assistance systems. Techniques utilizing image motion are likely to suffer from spurious image structures caused by shadows and illumination changes, let alone the aperture problem. To achieve reliable detection of overtaking vehicles, the authors have developed a robust detection method, which integrates dynamic scene modeling, hypothesis testing, and robust information fusion. A robust fusion algorithm, based on variable bandwidth density fusion and multiscale mean shift, is introduced to obtain reliable motion estimation against various image noise. To further reduce detection error, the authors model the dynamics of road scenes and exploit useful constraints induced by the temporal coherence in vehicle overtaking. The proposed solution is integrated into a monocular vision system onboard for obstacle detection. Test results have shown superior performance achieved by the new method
---
paper_title: Crossing obstacle detection with a vehicle-mounted camera

paper_content:

We propose a computer vision algorithm that detects obstacles crossing a vehicle's path with a monocular camera mounted on the vehicle. False positives are strongly suppressed even for low-resolution images by imposing constraints on feature-based optical flows. The constraints are derived from a model of crossing obstacle motion under perspective projection. A key concept in this model is “Relative Incoming Angle”, which is an angle between the camera's translational direction and relative velocity of a crossing obstacle with respect to the camera. We show a ROC curve that has been obtained by varying the Relative Incoming Angle using our dataset consisting of 18 scenes, 1456 frames. A representative point on the curve yields the detection rate of 59.7% and false positive rate of 2.6% (per-image).
---
paper_title: Detection and tracking of independently moving objects in urban environments

paper_content:

In this paper we propose an approach for dynamic scene perception from a moving vehicle equipped with a stereo camera rig. The approach is solely based on visual information, hence it is applicable to a large class of autonomous robots working in indoor as well as in outdoor environments. The proposed approach consists of an egomotion estimation based on disparity and optical flow using the Longuet-Higgins-Equations combined with an implicit extended Kalman-Filter. Based on this egomotion estimation a moving object detection and tracking is performed. Each tracked object is labeled with a unique ID while visible in the images. The proposed algorithm was evaluated on numerous challenging real world image sequences.
---
paper_title: Sparse scene flow segmentation for moving object detection in urban environments

paper_content:

Modern driver assistance systems such as collision avoidance or intersection assistance need reliable information on the current environment. Extracting such information from camera-based systems is a complex and challenging task for inner city traffic scenarios. This paper presents an approach for object detection utilizing sparse scene flow. For consecutive stereo images taken from a moving vehicle, corresponding interest points are extracted. Thus, for every interest point, disparity and optical flow values are known and consequently, scene flow can be calculated. Adjacent interest points describing a similar scene flow are considered to belong to one rigid object. The proposed method does not rely on object classes and allows for a robust detection of dynamic objects in traffic scenes. Leading vehicles are continuously detected for several frames. Oncoming objects are detected within five frames after their appearance.
---
paper_title: Real-time vehicle tracking for driving assistance

paper_content:

Detecting car taillights at night is a task which can nowadays be accomplished very fast on cheap hardware. We rely on such detections to build a vision-based system that, coupling them in a rule-based fashion, is able to detect and track vehicles. This allows the generation of an interface that informs a driver of the relative distance and velocity of other vehicles in real time and triggers a warning when a potentially dangerous situation arises. We demonstrate the system using sequences shot using a camera mounted behind a car’s windshield.
---
paper_title: Multiple target tracking for intelligent headlights control

paper_content:

Intelligent vehicle lighting systems aim at automatically regulating the headlights' beam to illuminate as much of the road ahead as possible while avoiding dazzling other drivers. A key component of such a system is computer vision software that is able to distinguish blobs due to vehicles' headlights and rear lights from those due to road lamps and reflective elements such as poles and traffic signs. In a previous work, we have devised a set of specialized supervised classifiers to make such decisions based on blob features related to its intensity and shape. Despite the overall good performance, there remain challenging that have yet to be solved: notably, faint and tiny blobs corresponding to quite distant vehicles. In fact, for such distant blobs, classification decisions can be taken after observing them during a few frames. Hence, incorporating tracking could improve the overall lighting system performance by enforcing the temporal consistency of the classifier decision. Accordingly, this paper focuses on the problem of constructing blob tracks, which is actually one of multiple-target tracking (MTT), but under two special conditions: We have to deal with frequent occlusions, as well as blob splits and merges. We approach it in a novel way by formulating the problem as a maximum a posteriori inference on a Markov random field. The qualitative (in video form) and quantitative evaluation of our new MTT method shows good tracking results. In addition, we will also see that the classification performance of the problematic blobs improves due to the proposed MTT algorithm.
---
paper_title: Vehicle detection under varying poses using Conditional Random Fields

paper_content:

Traditional vision based vehicle detection methods are more successful in detecting front and rear vehicles. However, the problem of detecting vehicles under various poses still presents a great deal of difficulty. Pose variation leads to limit the use of vision based driver assistance systems. In this paper, we present a Conditional Random Fields (CRFs) based algorithm that can detect vehicles under various poses. We treat this problem in a different way. We extract textural properties from small image patches as well as colors. Then CRFs model is employed to incorporate the contextual information. Firstly, we classify these patches into vehicular surfaces or background surfaces. Then we use clustering algorithm to eliminate the false alarms and detect multiple vehicles. From the quantitative evaluation of the proposed methods, our algorithm can be used in many practical applications that do not need accurate segmentation of vehicles.
---
paper_title: Estimating surrounding vehicles' pose using computer vision

paper_content:

This paper presents a computer vision-based approach to tracking surrounding vehicles and estimating their trajectories, in order to detect potentially dangerous situations. Images are acquired using a camera mounted in the egovehicle. Estimations of the distance, velocity and orientation of other vehicles on the road are obtained by detecting their lights and shadow. Because 3D information is not readily available in a mono-camera system, several sets of constraints and assumptions on the geometry of both road and vehicles are proposed and tested in this paper. Kalman filters are used to track the detected vehicles. We also study the advantages of tracking the vehicles in road space (world coordinates), or tracking the position of the lights and shadows on the image. The performance of the approaches is evaluated on video recorded in urban environment.
---
paper_title: Vehicle recognition and TTC estimation at night based on spotlight pairing

paper_content:

Forward Collision Warning (FCW) together with Lane Departure Warning (LDW) and Electronic Stability Control (ESC) can highly contribute to a safer traffic environment. Therefore those three systems will be supported by the US NCAP (New Car Assessment Program) until the end of 2009 and rewarded in the new active safety rating. LDW and ESC are already on the market whereas development on the first vision-based FCW systems is still ongoing and enforced. This paper will give a short overview on common approaches in vehicle recognition and focus on nighttime conditions. Based on existing head- and taillight detection algorithms for Adaptive Headlamp Control (AHC) a novel vehicle detection algorithm is presented. The main challenge hereby is the accurate pairing of detected vehicle spots. The developed system also contains rough distance estimation and a robust Time-To-Collision (TTC) calculation. Sequences in various traffic conditions with hand-labeled vehicles serve as an evaluation set for the vehicle detection performance. Ground truth for TTC is available by radar sensor data. The system proves to be robust in various near-collision situations.
---
paper_title: Vision-based nighttime vehicle detection and range estimation for driver assistance

paper_content:

This paper presents a real-time vision system for assisting driver during nighttime driving. The proposed system provides the following features: 1) effectively detection and tracking of oncoming and preceding vehicles based on image segmentation and pattern analysis techniques. 2) Robust and adaptive vehicle detection under various illuminated conditions at nighttime urban environments benefited by a novel automatic object segmentation scheme. 3) Providing beneficial information for assisting the driver to perceive surrounding traffic conditions outside the car during nighttime driving. 4) Providing a versatile control strategy for in-vehicle facilities of the autonomous vehicles. 5) Offering real-time traffic event-driven video surveillance machinery for recording evidences of possible traffic accidents. Experimental results demonstrate the feasibility and effectiveness of the proposed system on nighttime driver assistance issues.
---
paper_title: Monocular precrash vehicle detection: features and classifiers

paper_content:

Robust and reliable vehicle detection from images acquired by a moving vehicle (i.e., on-road vehicle detection) is an important problem with applications to driver assistance systems and autonomous, self-guided vehicles. The focus of this work is on the issues of feature extraction and classification for rear-view vehicle detection. Specifically, by treating the problem of vehicle detection as a two-class classification problem, we have investigated several different feature extraction methods such as principal component analysis, wavelets, and Gabor filters. To evaluate the extracted features, we have experimented with two popular classifiers, neural networks and support vector machines (SVMs). Based on our evaluation results, we have developed an on-board real-time monocular vehicle detection system that is capable of acquiring grey-scale images, using Ford's proprietary low-light camera, achieving an average detection rate of 10 Hz. Our vehicle detection algorithm consists of two main steps: a multiscale driven hypothesis generation step and an appearance-based hypothesis verification step. During the hypothesis generation step, image locations where vehicles might be present are extracted. This step uses multiscale techniques not only to speed up detection, but also to improve system robustness. The appearance-based hypothesis verification step verifies the hypotheses using Gabor features and SVMs. The system has been tested in Ford's concept vehicle under different traffic conditions (e.g., structured highway, complex urban streets, and varying weather conditions), illustrating good performance.
---
paper_title: Learning a Family of Detectors via Multiplicative Kernels

paper_content:

Object detection is challenging when the object class exhibits large within-class variations. In this work, we show that foreground-background classification (detection) and within-class classification of the foreground class (pose estimation) can be jointly learned in a multiplicative form of two kernel functions. Model training is accomplished via standard SVM learning. When the foreground object masks are provided in training, the detectors can also produce object segmentations. A tracking-by-detection framework to recover foreground state in video sequences is also proposed with our model. The advantages of our method are demonstrated on tasks of object detection, view angle estimation, and tracking. Our approach compares favorably to existing methods on hand and vehicle detection tasks. Quantitative tracking results are given on sequences of moving vehicles and human faces.
---
paper_title: Histograms of oriented gradients for human detection

paper_content:

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.
---
paper_title: Rear-Lamp Vehicle Detection and Tracking in Low-Exposure Color Video for Night Conditions

paper_content:

Automated detection of vehicles in front is an integral component of many advanced driver-assistance systems (ADAS), such as collision mitigation, automatic cruise control (ACC), and automatic headlamp dimming. We present a novel image processing system to detect and track vehicle rear-lamp pairs in forward-facing color video. A standard low-cost camera with a complementary metal-oxide semiconductor (CMOS) sensor and Bayer red-green-blue (RGB) color filter is used and could be utilized for full-color image display or other color image processing applications. The appearance of rear lamps in video and imagery can dramatically change, depending on camera hardware; therefore, we suggest a camera-configuration process that optimizes the appearance of rear lamps for segmentation. Rear-facing lamps are segmented from low-exposure forward-facing color video using a red-color threshold. Unlike previous work in the area, which uses subjective color threshold boundaries, our color threshold is directly derived from automotive regulations and adapted for real-world conditions in the hue-saturation-value (HSV) color space. Lamps are paired using color cross-correlation symmetry analysis and tracked using Kalman filtering. A tracking-based detection stage is introduced to improve robustness and to deal with distortions caused by other light sources and perspective distortion, which are common in automotive environments. Results that demonstrate the system's high detection rates, operating distance, and robustness to different lighting conditions and road environments are presented.
---
paper_title: Realtime Vision Based Multi-Target-Tracking with Particle Filters in Automotive Applications

paper_content:

In the following we present a multi-instance-based multi-target-tracking method using particle filters. We developed a robust and flexible system capable of tracking an unknown and changing number of objects (vehicles) using early-vision image features. The system works in real time.
---
paper_title: Visual classification of coarse vehicle orientation using Histogram of Oriented Gradients features

paper_content:

For an autonomous vehicle, detecting and tracking other vehicles is a critical task. Determining the orientation of a detected vehicle is necessary for assessing whether the vehicle is a potential hazard. If a detected vehicle is moving, the orientation can be inferred from its trajectory, but if the vehicle is stationary, the orientation must be determined directly. In this paper, we focus on vision-based algorithms for determining vehicle orientation of vehicles in images. We train a set of Histogram of Oriented Gradients (HOG) classifiers to recognize different orientations of vehicles detected in imagery. We find that these orientation-specific classifiers perform well, achieving a 88% classification accuracy on a test database of 284 images. We also investigate how combinations of orientation-specific classifiers can be employed to distinguish subsets of orientations, such as driver's side versus passenger's side views. Finally, we compare a vehicle detector formed from orientation-specific classifiers to an orientation-independent classifier and find that, counter-intuitively, the orientation-independent classifier outperforms the set of orientation-specific classifiers.
---
paper_title: Color-based detection of vehicle lights

paper_content:

In this article, we present a first approach in the conception of a vision system dedicated to the detection of vehicles in reduced visibility conditions. It is based on a self-adaptive stereo vision extractor of 3D edges or obstacle and a color detection of vehicle lights. The detection of vehicle lights uses the L*a*b* color space. The vision system detects three kinds of vehicle lights: rear lights and rear-brake-lights; flashing and warning lights; reverse lights and headlights.
---
paper_title: Vehicle Detection under Various Lighting Conditions by Incorporating Particle Filter

paper_content:

We propose an automatic system to detect preceding vehicles on the highway under various lighting and different weather conditions based on the computer vision technologies. To adapt to different characteristics of vehicle appearance at daytime and nighttime, four cues including underneath, vertical edge, symmetry and taillight are fused for the preceding vehicle detection. By using particle filter with four cues through the processes including initial sampling, propagation, observation and cue fusion and evaluation, particle filter accurately generates the vehicle distribution. Thus, the proposed system can successfully detect and track preceding vehicles and be robust to different lighting conditions. Unlike normal particle filter focuses on a single target distribution in a discrete state space, we detect multiple vehicles with particle filter through a high-level tracking strategy using clustering technique called basic sequential algorithmic scheme (BSAS). Finally, experimental results for several videos from different scenes are provided to demonstrate the effectiveness of our proposed system.
---
paper_title: Unsupervised sub-categorization for object detection: Finding cars from a driving vehicle

paper_content:

We present a novel algorithm for unsupervised subcategorization of an object class, in the context of object detection. Dividing the detection problem into smaller subproblems simplifies the object vs. background classification.
---
paper_title: 6D-Vision: Fusion of Stereo and Motion for Robust Environment Perception

paper_content:

Obstacle avoidance is one of the most important challenges for mobile robots as well as future vision based driver assistance systems. This task requires a precise extraction of depth and the robust and fast detection of moving objects. In order to reach these goals, this paper considers vision as a process in space and time. It presents a powerful fusion of depth and motion information for image sequences taken from a moving observer. 3D-position and 3D-motion for a large number of image points are estimated simultaneously by means of Kalman-Filters. There is no need of prior error-prone segmentation. Thus, one gets a rich 6D representation that allows the detection of moving obstacles even in the presence of partial occlusion of foreground or background.
---
paper_title: 3D pose estimation of vehicles using a stereo camera

paper_content:

This study introduces an approach to three-dimensional vehicle pose estimation using a stereo camera system. After computation of stereo and optical flow on the investigated scene, a four-dimensional clustering approach separates the static from the moving objects in the scene. The iterative closest point algorithm (ICP) estimates the vehicle pose using a cuboid as a weak vehicle model. In contrast to classical ICP optimisation a polar distance metric is used which especially takes into account the error distribution of the stereo measurement process. The tracking approach is based on tracking-by-detection such that no temporal filtering is used. The method is evaluated on seven different real-world sequences, where different stereo algorithms, baseline distances, distance metrics, and optimisation algorithms are examined. The results show that the proposed polar distance metric yields a higher accuracy for yaw angle estimation of vehicles than the common Euclidean distance metric, especially when using pixel-accurate stereo points.
---
paper_title: 3D Vision: Developing an Embedded Stereo-Vision System

paper_content:

The DeepSea G2 stereo vision system that Tyzx developed features an embedded stereo camera consisting of two CMOS imagers, a Tyzx DeepSea 2 stereo application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a DSP/coprocessor, a PowerPC running Linux, and an Ethernet connection. About the size of a hardback book, the G2 delivers real-time - 30 frames per second (fps) - interpretations of visual data even in challenging environments and has been deployed in a variety of applications, including person-tracking and autonomous-vehicle navigation.
---
paper_title: Vehicle tracking and motion prediction in complex urban scenarios

paper_content:

The recognition of potentially hazardous situations on road intersections is an indispensable skill of future driver assistance systems. In this context, this study focuses on the task of vehicle tracking in combination with a long-term motion prediction (1–2 s into the future) in a dynamic scenario. A motion-attributed stereo point cloud obtained using computationally efficient feature-based methods represents the scene, relying on images of a stereo camera system mounted on a vehicle. A two-stage mean-shift algorithm is used for detection and tracking of the traffic participants. A hierarchical setup depending on the history of the tracked object is applied for prediction. This includes prediction by optical flow, a standard kinematic prediction, and a particle filter based motion pattern method relying on learned object trajectories. The evaluation shows that the proposed system is able to track the road users in a stable manner and predict their positions at least one order of magnitude more accurately than a standard kinematic prediction method.
---
paper_title: Disparity space image's features analysis for error prediction of a stereo obstacle detector for heavy duty vehicles

paper_content:

Vision-based perception has been explored as low-cost, flexible technology for industrial applications and ADAS. Its inherent flexibility presents a challenge quantifying performance and often even quantifying increases or decreases in system performance as conditions change. Experience enables designers to employ various ”rules of thumb” while commercially viable products require quantitative performance. This paper explores the correlation between features and characteristics of the Disparity Space Image (DSI) and resulting performance for an object detection application. The specific application is an object detection system suitable for highly chaotic environments often found in earthmoving industry. Features and characteristics with strong correlations can be used to improve system design and predict system performances at run-time. High-quality stereo images are used to characterize baseline system performance. These images are then artificially degraded to simulate fog, darkness, and blurring and subsequent system performance compared to baseline results.
---
paper_title: Stereo based Obstacle Detection with Uncertainty in Rough Terrain

paper_content:

Autonomous robot vehicles that operate in off-road terrain should avoid obstacle hazards. In this paper we present a stereo vision based method that is able to cluster reconstructed terrain points into obstacles by evaluating their relative angles and distances. In our approach, constraints are enforced on these geometric properties by a set of pixel threshold values. Because these values are all computed during an initialisation step, only simple pixel threshold operations remain to be performed during the real-time obstacle detection. An advantage of this novel approach is that the distance uncertainties can be incorporated into the thresholds. Detected obstacle points are clustered into objects on the basis of their pixel connectivity. Objects with insufficient pixels, elevation and slope are rejected. Remaining non-obstacle pixels are regarded as ground surface points. They are used to update the orientation of the stereo camera relative to the ground surface. This prevents orientation errors during stereo reconstruction and the subsequent obstacle detection steps. Our results show the drawbacks of ignoring the uncertainties in the stereo distance estimates for obstacle detection. It leads to over-segmentation and increases the number of falsely detected obstacles. Because our method incorporates these uncertainties, it can detect more of the obstacle surface pixels at larger distances. This leads to significantly less false obstacle detections.
---
paper_title: TerraMax Vision at the Urban Challenge 2007

paper_content:

This paper presents the TerraMax vision systems used during the 2007 DARPA Urban Challenge. First, a description of the different vision systems is provided, focusing on their hardware configuration, calibration method, and tasks. Then, each component is described in detail, focusing on the algorithms and sensor fusion opportunities: obstacle detection, road marking detection, and vehicle detection. The conclusions summarize the lesson learned from the developing of the passive sensing suite and its successful fielding in the Urban Challenge.
---
paper_title: Accurate and efficient stereo processing by semi-global matching and mutual information

paper_content:

This paper considers the objectives of accurate stereo matching, especially at object boundaries, robustness against recording or illumination changes and efficiency of the calculation. These objectives lead to the proposed semi-global matching method that performs pixelwise matching based on mutual information and the approximation of a global smoothness constraint. Occlusions are detected and disparities determined with sub-pixel accuracy. Additionally, an extension for multi-baseline stereo images is presented. There are two novel contributions. Firstly, a hierarchical calculation of mutual information based matching is shown, which is almost as fast as intensity based matching. Secondly, an approximation of a global cost calculation is proposed that can be performed in a time that is linear to the number of pixels and disparities. The implementation requires just 1 second on typical images.
---
paper_title: Real-time dynamic environment perception in driving scenarios using difference fronts

paper_content:

The environment representation is one of the main challenges of autonomous navigation. In the case of complex driving environments such as crowded city traffic scenarios, achieving satisfactory results becomes even more difficult. In this paper we propose a real-time solution for two main issues of advanced driver assistance systems: unstructured environment representation and extraction of dynamic properties of traffic participants. For the real-time environment representation we propose a solution to extract object delimiters from the traffic scenes and represent them as polygonal models. In order to track dynamic entities, an intermediate evidence map named “Stereo Temporal Difference Map” is proposed. This difference map is computed by comparing the occupancy of a cell between two consecutive frames. Based on the Stereo Temporal Difference Map information, difference fronts are extracted and are subjected to a particle based filtering mechanism. Finally, the provided dynamic features are associated to the extracted polygonal models. The result is a more compact representation of the dynamic environment.
---
paper_title: Real-time vehicle detection and tracking using stereo vision and multi-view AdaBoost

paper_content:

We propose a multi-layer, real-time vehicle detection and tracking system using stereo vision, multi-view AdaBoost detectors, and optical flow. By adopting a ground plane estimate extracted from stereo information, we generate a sparse set of hypotheses and apply trained AdaBoost classifiers in addition to fast disparity histogramming, for Hypothesis Verification (HV) purposes. Our tracking system employs one Kalman filter per detected vehicle and motion vectors from optical flow, as a means to increase its robustness. An acceptable detection rate with few false positives is obtained at 25 fps with generic hardware.
---
paper_title: Feature-Based Probabilistic Data Association (FBPDA) for visual multi-target detection and tracking under occlusions and split and merge effects

paper_content:

Uncertainties in the sensor data such as measurement noise, false detections caused by clutter, as well as merged, split, incomplete or missed detections due to a sensor malfunction or occlusions (both due to the limited sensor field of view and objects in the scene) make multi-target tracking a very complicated task. Thus one of the big challenges is track management and correct data association between detections and tracks. In this contribution we present an algorithm for visual detection and tracking of multiple extended targets under occlusions and split and merge effects. Unlike most of the state-of-the-art approaches we utilize low-level information integrating it in a unified approach based on a threshold-free probabilistic conception. The introduced scheme makes it possible to utilize information about composition of the measurements gained through tracking of dedicated feature points in the image and resolves data association ambiguities in a soft decision using a globally optimal probabilistic data association approach. Beside existence evolution consideration we also exploit the spatial and temporal relationship between stably tracked points and tracked objects, which along with observability analysis, allows us for reconstruction of compatible measurements and thus correct track update even in cases of splits, merges and partial occlusions of the tracked targets.
---
paper_title: Real-time semi-global dense stereo solution with improved sub-pixel accuracy

paper_content:

In this work we focus on creating a real-time dense stereo reconstruction system with accurate sub-pixel estimation. We selected the Semi-Global Matching method as the basis of our system due to its high quality and possible real-time implementations. In our solution we use the Census transform as the matching metric because our results show that it can reduce the matching errors for traffic images compared to classical solutions. We also propose several modifications to the original Semi-Global algorithm to improve the sub-pixel accuracy and the execution time. One of these proposals is the reduction in the number of optimization directions without affecting the results. The second modification is a correction of the energy function to reduce the spread of depth values. Besides these improvements, the paper also introduces a new aggregation method used to reduce the spread of sub-pixel values. Finally we propose a new method to generate sub-pixel interpolation functions based on real-world data. The result of these enhancements is a significant improvement in sub-pixel accuracy. The system was implemented and evaluated on a current generation GPU with a running time of 19ms for image having the resolution 512×383.
---
paper_title: A generic obstacle detection method for collision avoidance

paper_content:

Obstacle detection is an important component in driver assistance as it helps systems to locate obstacles and then to prevent collisions. The aim of this study is to develop an obstacle detection module through digital images processing. We present a hybrid stereo vision-based method that combines stereo matching and homographic transformation methods. We use a sparse matching method in order to get a rapid geometric representation of the road scene that allows us to extract the upper and lower parts of obstacles. According to the position of the lower part, our method uses either the dense stereo matching or the homographic transformation methods to extract the candidate obstacles regions. A verification test is performed to verify whether the retained region is an obstacle or not. In order to avoid collisions, we compute the distance to the preceding obstacle to maintain the vehicle carrying the camera at a safety distance. The method presented here was tested on DIPLODOC1 road stereo sequence captured on a highway. The obtained results prove the efficiency of our proposed method.
---
paper_title: Stereo-Based Object Detection, Classi?cation, and Quantitative Evaluation with Automotive Applications

paper_content:

A real-time stereo-based pre-crash object detection and classification system is presented. The system employs a model based stereo object detection algorithm to find candidate objects from the scene, followed by a Bayesian classification framework to assign each candidate to its proper class. Our current system detects and classifies several types of objects commonly seen for automotive applications, namely vehicles, pedestrians/bikes, and poles. We describe both the detection and classification algorithms in detail along with real-time implementation issues. A quantitative analysis of performance on a static data set is also presented.
---
paper_title: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography

paper_content:

A new paradigm, Random Sample Consensus (RANSAC), for fitting a model to experimental data is introduced. RANSAC is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors. A major portion of this paper describes the application of RANSAC to the Location Determination Problem (LDP): Given an image depicting a set of landmarks with known locations, determine that point in space from which the image was obtained. In response to a RANSAC requirement, new results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form. These results provide the basis for an automatic system that can solve the LDP under difficult viewing
---
paper_title: A Global Optimization Algorithm for Real-Time On-Board Stereo Obstacle Detection Systems

paper_content:

A fast and robust stereo algorithm for on-board obstacle detection systems is proposed. The proposed method finds the optimum road-obstacle boundary which provides the most consistent interpretation of the input stereo image pair. Global optimization combined with a robust matching measure enables stable detection of obstacles under various circumstances, such as heavy rain and severe lighting conditions. The processing time for VGA size image pair is about 15 msec on a 3.6 GHz pentium IV processor, which is fast enough for realtime applications.
---
paper_title: Moving vehicle detection by optimal segmentation of the Dynamic Stixel World

paper_content:

The reliable detection of moving objects from a moving observer is one of the most challenging and important tasks for driver assistance and safety systems. Modern sensors such as Lidar, Imaging Radar or Stereo Vision deliver range data plus longitudinal motion (Radar) or even full 3D-motion (space-time vision). Based on this data, moving objects have to be separated from the static background to be able to determine their pose and motion state. Usually, heuristics are applied to cluster the data. In order to find the most probable segmentation, we formulate the task as a hypotheses testing problem that allows taking into account various constraints and assumptions simultaneously. We show that the optimal segmentation can be efficiently found by means of dynamic programming, for an arbitrary number of objects in the scene. In this paper we concentrate on the segmentation of space-time data obtained from stereo image sequences. The vision-based depth and motion information is transferred into so called Stixels, a very compact representation of 3D scenes that can also be applied to Lidar or Radar data. It turns out that our optimal segmentation is more robust w.r.t. noisy and erroneous data.
---
paper_title: Use of the Hough transformation to detect lines and curves in pictures

paper_content:

Hough has proposed an interesting and computationally efficient procedure for detecting lines in pictures. This paper points out that the use of angle-radius rather than slope-intercept parameters simplifies the computation further. It also shows how the method can be used for more general curve fitting, and gives alternative interpretations that explain the source of its efficiency.
---
paper_title: Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection

paper_content:

A new approach for the detection of the road surface and obstacles is presented. The high accuracy of the method allows the detection of traffic isles as a distinct class. The 3-D data inferred from dense stereo are transformed into a rectangular digital elevation map (DEM). Two classifiers are proposed, namely, density based and road surface based. The density-based obstacle classifier marks DEM cells as road or obstacles, using the density of 3-D points as a criterion. A quadratic road surface model is initially fitted by a random sample consensus (RANSAC) approach to the region in front of the ego vehicle. A region growing-like process refines this primary solution, driven by the 3-D uncertainty model of the stereo sensor. A robust global solution for the road surface is obtained. The road surface is used for discrimination between road, traffic isle, and obstacle points. Fusion and error filtering is performed on the results of the two classifiers. The proposed real-time algorithm was evaluated in an urban scenario and can be used in complex applications from collision avoidance to path planning.
---
paper_title: Vehicle detection and tracking using Mean Shift segmentation on semi-dense disparity maps

paper_content:

This paper describes an original joint obstacle detection and tracking method based on a Mean Shift algorithm and semi-dense disparity maps. The semi-dense disparity maps are computed with a local 1D fuzzy scanline stereo matching approach. Each map is associated to a confidence map that is used to remove bad matches. The Mean Shift algorithm is applied to simultaneously extract each vehicle and track the 3D points belonging to the same vehicle along the sequence. We show that several vehicles can be efficiently detected and that a semi-dense disparity map is sufficient to reach an accurate segmentation even when occlusions occur. This paper presents some results on real image sequences acquired in the context of Advanced Driver Assistance Systems.
---
paper_title: Mapping in dynamic environments using stereo vision

paper_content:

Next generation driver assistance systems demand a precise perception of the vicinity of the vehicle. Sensor readings are usually harnessed to gain knowledge of all moving and stationary obstacles. Commonly two paradigms are followed. Stationary environments are well modeled by non-parametric occupancy grids. Contrary, moving objects require a tracking and are not well suited for grid-based techniques. However, tracking objects requires some prior knowledge and parameterization which is inferior for unstructured cluttered obstacles. Herein we augment a grid-based mapping method designed for static environments with object tracking hence complementing both approaches. To this end we classify regions of stereo images into moving and stationary parts. The stationary part is fused in our static grid whereas moving parts are tracked yielding reliable motion estimates. The classifier used to distinguish moving from stationary parts is based on a Sequential Probability Ratio Test (SPRT), a model selection method which blends well into the tracking architecture. Thereby we achieve real-time operability on modest computing hardware.
---
paper_title: Probabilistic representation of the uncertainty of stereo-vision and application to obstacle detection

paper_content:

Stereo-vision is extensively used for intelligent vehicles, mainly for obstacle detection, as it provides a large amount of data. Many authors use it as a classical 3D sensor which provides a large tri-dimensional cloud of metric measurements, and apply methods usually designed for other sensors, such as clustering based on a distance. For stereo-vision, the measurement uncertainty is related to the range. For medium to long range, often necessary in the field of intelligent vehicles, this uncertainty has a significant impact, limiting the use of this kind of approaches. On the other hand, some authors consider stereo-vision more like a vision sensor and choose to directly work in the disparity space. This provides the ability to exploit the connectivity of the measurements, but roughly takes into consideration the actual size of the objects. In this paper, we propose a probabilistic representation of the specific uncertainty for stereo-vision, which takes advantage of both aspects - distance and disparity. The model is presented and then applied to obstacle detection, using the occupancy grid framework. For this purpose, a computationally-efficient implementation based on the u-disparity approach is given.
---
paper_title: Detection of independently moving objects through stereo vision and ego-motion extraction

paper_content:

Vision-based autonomous vehicles must face numerous challenges in order to be effective in practical areas. Among these lies the detection and localization of independent-moving objects, so as to track or avoid them. In this paper a method that address this particular issue is presented. Information from stereo and motion is used to extract the ego-motion of the vehicle. Known defects of this estimation are exploited to detect independent-moving obstacles. This method allows an early and reliable detection, even for objects partially occluded. Besides, it highlights the errors in the disparity map, which can be used, in future works, to correct depth-estimation, through motion-estimation.
---
paper_title: Contrast-invariant Obstacle Detection System using Color Stereo Vision

paper_content:

In this article, we present a color stereo vision system conceived to detect road obstacles. Color matching algorithm based on dynamic programming is presented. This algorithm is invariant in the contrast. With this property, the gain of cameras is configured in automatic mode without decrease the performance of color matching. As a consequence, 3D edges of obstacle are correctly extracted even in difficult cases (for example, entrance and exit of the tunnel). Experimental results are shown in the last section of the paper.
---
paper_title: A Visibility-Based Approach for Occupancy Grid Computation in Disparity Space

paper_content:

Occupancy grids are a very convenient tool for environment representation in robotics. This paper will detail a novel approach for computing occupancy grids from stereo vision and show its application to intelligent vehicles. In the proposed approach, occupancy is initially computed directly in the stereoscopic sensor's disparity space. The calculation formally accounts for the detection of obstacles and road pixels in disparity space, as well as partial occlusions in the scene. In the second stage, this disparity-space occupancy grid is transformed into a Cartesian space occupancy grid to be used by subsequent applications. This transformation includes spatial and temporal filtering. The proposed method is designed to easily be processed in parallel. Consequently, we chose to implement it on a graphics processing unit, which allows real-time processing for demanding applications. In this paper, we present this method, and we propose an application to the problem of perception in a road environment. Results are presented with real road data, qualitatively comparing this approach with other methods.
---
paper_title: Detection and tracking of independently moving objects in urban environments

paper_content:

In this paper we propose an approach for dynamic scene perception from a moving vehicle equipped with a stereo camera rig. The approach is solely based on visual information, hence it is applicable to a large class of autonomous robots working in indoor as well as in outdoor environments. The proposed approach consists of an egomotion estimation based on disparity and optical flow using the Longuet-Higgins-Equations combined with an implicit extended Kalman-Filter. Based on this egomotion estimation a moving object detection and tracking is performed. Each tracked object is labeled with a unique ID while visible in the images. The proposed algorithm was evaluated on numerous challenging real world image sequences.
---
paper_title: On-road vehicle tracking using deformable object model and particle filter with integrated likelihoods

paper_content:

This paper proposes a novel method for vehicle detection and tracking using a vehicle-mounted monocular camera. In this method, features of vehicles are learned as a deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOG). The vehicle detector uses both global and local features as the deformable object model. Detected vehicles are tracked by using a particle filter with integrated likelihoods, such as the probability of vehicles estimated from the deformable object model and the intensity correlation between different picture frames. Tracking likelihoods are iteratively used as the a priori probability for the next frame. The experimental results showed that the proposed method can achieve an average vehicle detection rate of 98% and an average vehicle tracking rate of 87% with a false positive rate of less than 0.3%.
---
paper_title: Estimating surrounding vehicles' pose using computer vision

paper_content:

This paper presents a computer vision-based approach to tracking surrounding vehicles and estimating their trajectories, in order to detect potentially dangerous situations. Images are acquired using a camera mounted in the egovehicle. Estimations of the distance, velocity and orientation of other vehicles on the road are obtained by detecting their lights and shadow. Because 3D information is not readily available in a mono-camera system, several sets of constraints and assumptions on the geometry of both road and vehicles are proposed and tested in this paper. Kalman filters are used to track the detected vehicles. We also study the advantages of tracking the vehicles in road space (world coordinates), or tracking the position of the lights and shadows on the image. The performance of the approaches is evaluated on video recorded in urban environment.
---
paper_title: Fusing multiple 2D visual features for vehicle detection

paper_content:

This contribution presents a multi sensor fusion approach for vehicle detection. Shadow and Symmetry features, abstracted to 3D sensors by including street surface information, are combined in an Interacting Multiple Model filter with two system models, one for constant velocity, one for constant acceleration. Measurements are associated to tracks by means of a Cheap Joint Probabilistic Data Association technique. Virtual prediction steps are used to incorporate multiple sensors and to achieve a versatile fusion architecture that allows easy integration of further sensors.
---
paper_title: On-board robust vehicle detection and tracking using adaptive quality evaluation

paper_content:

This paper presents a robust method for real-time vehicle detection and tracking in dynamic traffic environments. The proposed strategy aims to find a trade-off between the robustness shown by time-uncorrelated detection techniques and the speed-up obtained with tracking algorithms. It combines both advantages by continuously evaluating the quality of the tracking results along time and triggering new detections to restart the tracking process when quality falls behind a certain quality requirement. Robustness is also ensured within the tracking algorithm with an outlier rejection stage and the use of stochastic filtering. Several sequences from real traffic situations have been tested, obtaining highly accurate multiple vehicle detections.
---
paper_title: Moving Obstacle Detection using Monocular Vision

paper_content:

This paper proposes a method for detecting moving obstacles on roads, by using a vehicle mounted monocular camera. To detect various moving obstacles, such as vehicles and pedestrians, the ego-motion of the vehicle is initially estimated from images captured by the camera. There are two problems in ego-motion estimation. Firstly, a typical road scene contains moving obstacles. This causes false estimation of the ego-motion. Secondly, roads possess fewer features, when compared to the number associated with background structures. This reduces the accuracy of ego-motion estimation. In our approach, the ego-motion is estimated from the correspondences of dispersed feature points extracted from various regions other than those that contain moving obstacles. After estimating the ego-motion, any moving obstacles are detected by tracking the feature points over consecutive frames. In our experiments, it has been shown that the proposed method is able to detect moving obstacles.
---
paper_title: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography

paper_content:

A new paradigm, Random Sample Consensus (RANSAC), for fitting a model to experimental data is introduced. RANSAC is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors. A major portion of this paper describes the application of RANSAC to the Location Determination Problem (LDP): Given an image depicting a set of landmarks with known locations, determine that point in space from which the image was obtained. In response to a RANSAC requirement, new results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form. These results provide the basis for an automatic system that can solve the LDP under difficult viewing
---
paper_title: Increasing driving safety with a multiple vehicle detection and tracking system using ongoing vehicle shadow information

paper_content:

This paper proposes a vehicle detection and tracking system based on processing monochrome images captured by a single camera. The work has mainly been focused on detecting and tracking vehicles in daylight conditions, viewed from inside a vehicle. Unlike previous work, this approach uses vehicle shadow clues and vehicle edge information to obtain cost effective and fast estimation. The proposed method includes road area finding which has been implemented by a lane detection algorithm to avoid false detections of vehicles caused by the distraction of background objects. Assuming that lanes are successfully detected, vehicle presence inside the road area is hypothesized by using “shadow” as a cue. Hypothesized vehicle locations are verified using vertical edges. After extracting vehicles, the algorithm effectively tracks them using a Kalman filter based tracking algorithm. A vehicle has been instrumented with various sensors for the experiments. Several sequences from real traffic situations have been tested, obtaining highly accurate multiple vehicle detections. Tracking information is used to estimate time-to-collision (TTC) and warn the driver for a possible collision.
---
paper_title: An integrated framework of vision-based vehicle detection with knowledge fusion

paper_content:

This paper describes an integrated framework of on-road vehicle detection through knowledge fusion. In contrast to appearance-based detectors that make instant decisions, the proposed detection framework fuses appearance, geometry and motion information over multiple image frames. The knowledge of vehicle/non-vehicle appearance, scene geometry and vehicle motion is utilized through prior models obtained by learning, modeling and estimation algorithms. It is shown that knowledge fusion largely improves the robustness and reliability of the detection system.
---
paper_title: Vehicle Ego-Motion Estimation and Moving Object Detection using a Monocular Camera

paper_content:

This paper proposes a method for estimating the ego-motion of the vehicle and for detecting moving objects on roads by using a vehicle mounted monocular camera. There are two problems in ego-motion estimation. Firstly, a typical road scene contains moving objects such as other vehicles. Secondly, roads display fewer feature points compared to the number associated with background structures. In our approach, ego-motion is estimated from the correspondences of feature points extracted from various regions other than those in which objects are moving. After estimating the ego-motion, the three dimensional structure of the scene is reconstructed and any moving objects are detected. In our experiments, it has been shown that the proposed method is able to detect moving objects such as vehicles and pedestrians
---
paper_title: On-Road Multivehicle Tracking Using Deformable Object Model and Particle Filter With Improved Likelihood Estimation

paper_content:

This paper proposes a novel method for multivehicle detection and tracking using a vehicle-mounted monocular camera. In the proposed method, the features of vehicles are learned as a deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOGs). The detection algorithm combines both global and local features of the vehicle as a deformable object model. Detected vehicles are tracked through a particle filter, which estimates the particles' likelihood by using a detection scores map and template compatibility for both root and parts of the vehicle while considering the deformation cost caused by the movement of vehicle parts. Tracking likelihoods are iteratively used as a priori probability to generate vehicle hypothesis regions and update the detection threshold to reduce false negatives of the algorithm presented before. Extensive experiments in urban scenarios showed that the proposed method can achieve an average vehicle detection rate of 97% and an average vehicle-tracking rate of 86% with a false positive rate of less than 0.26%.
---
paper_title: Realtime Vision Based Multi-Target-Tracking with Particle Filters in Automotive Applications

paper_content:

In the following we present a multi-instance-based multi-target-tracking method using particle filters. We developed a robust and flexible system capable of tracking an unknown and changing number of objects (vehicles) using early-vision image features. The system works in real time.
---
paper_title: An evolutionary optimized vehicle tracker in collaboration with a detection system

paper_content:

In this work a learning algorithm for visual object tracking is presented. As object representation a fast computable set of Haar-like features is used and a weighted correlation is applied for the matching process. The object tracker utilizes the same set of features that is already calculated for object detection and thus it is possible to reuse features for detection and tracking. The feature's weight values are optimized for the tracking purpose by means of evolutionary strategies. Different tests of the object tracker on real-world sequences are presented using vehicles as example objects. Additionally, an object detection system and the integration of the object tracker into that system is described. Besides the system is based on a cascade of boosted classifiers, Haar and Triangle features, an adaptive sliding window and finally a Kalman filter.
---
paper_title: Real-time side vehicle tracking using parts-based boosting

paper_content:

This paper presents a real-time vision-based side vehicle detection system employing a parts-based boosting algorithm. Working at the part level, as opposed to the whole object level, enables a more flexible class representation and allows scenes in which the query object is significantly occluded to be classified. Therefore, a parts-based learning approach is proposed in order to better deal with side vehicle variability, illumination conditions, partial occlusions, and rotations. Most existing boosting learning algorithms usually select weak classifiers by minimizing a cost directly associated with the error rate, where the learned strong classifier may be sub-optimal for applications in terms of error rate. Nevertheless, the proposed Adaboost approach selects weak classifiers by minimizing multiple types of error functions. The idea is to define multiple types of error functions based on current strong classifier and each selected weak classifier results to represent different effects of each weak classifier. Therefore, weak classifiers can be selected with different requirements at the same time to avoid a sub-optimal solution. To reduce system computation, window-based tracking is employed. Moreover, Kalman filtering is used to predict the position of each part of vehicles in the image plane to effectively relocate the tracking windows. Compared with existing approaches, the proposed system appears to be capable of improving system efficiency and accuracy under varying lighting conditions, changing vehicle poses, and in the presence of partial occlusions. Our approach has been successfully validated in real traffic environments by performing experiments with a CCD camera mounted onboard a highway vehicle.
---
paper_title: A General Active-Learning Framework for On-Road Vehicle Recognition and Tracking

paper_content:

This paper introduces a general active-learning framework for robust on-road vehicle recognition and tracking. This framework takes a novel active-learning approach to building vehicle-recognition and tracking systems. A passively trained recognition system is built using conventional supervised learning. Using the query and archiving interface for active learning (QUAIL), the passively trained vehicle-recognition system is evaluated on an independent real-world data set, and informative samples are queried and archived to perform selective sampling. A second round of learning is then performed to build an active-learning-based vehicle recognizer. Particle filter tracking is integrated to build a complete multiple-vehicle tracking system. The active-learning-based vehicle-recognition and tracking (ALVeRT) system has been thoroughly evaluated on static images and roadway video data captured in a variety of traffic, illumination, and weather conditions. Experimental results show that this framework yields a robust efficient on-board vehicle recognition and tracking system with high precision, high recall, and good localization.
---
paper_title: Robust Visual Tracking and Vehicle Classification via Sparse Representation

paper_content:

In this paper, we propose a robust visual tracking method by casting tracking as a sparse approximation problem in a particle filter framework. In this framework, occlusion, noise, and other challenging issues are addressed seamlessly through a set of trivial templates. Specifically, to find the tracking target in a new frame, each target candidate is sparsely represented in the space spanned by target templates and trivial templates. The sparsity is achieved by solving an l1-regularized least-squares problem. Then, the candidate with the smallest projection error is taken as the tracking target. After that, tracking is continued using a Bayesian state inference framework. Two strategies are used to further improve the tracking performance. First, target templates are dynamically updated to capture appearance changes. Second, nonnegativity constraints are enforced to filter out clutter which negatively resembles tracking targets. We test the proposed approach on numerous sequences involving different types of challenges, including occlusion and variations in illumination, scale, and pose. The proposed approach demonstrates excellent performance in comparison with previously proposed trackers. We also extend the method for simultaneous tracking and recognition by introducing a static template set which stores target images from different classes. The recognition result at each frame is propagated to produce the final result for the whole video. The approach is validated on a vehicle tracking and classification task using outdoor infrared video sequences.
---
paper_title: Object tracking: A survey

paper_content:

The goal of this article is to review the state-of-the-art tracking methods, classify them into different categories, and identify new trends. Object tracking, in general, is a challenging problem. Difficulties in tracking objects can arise due to abrupt object motion, changing appearance patterns of both the object and the scene, nonrigid object structures, object-to-object and object-to-scene occlusions, and camera motion. Tracking is usually performed in the context of higher-level applications that require the location and/or shape of the object in every frame. Typically, assumptions are made to constrain the tracking problem in the context of a particular application. In this survey, we categorize the tracking methods on the basis of the object and motion representations used, provide detailed descriptions of representative methods in each category, and examine their pros and cons. Moreover, we discuss the important issues related to tracking including the use of appropriate image features, selection of motion models, and detection of objects.
---
paper_title: Rear Vehicle Detection and Tracking for Lane Change Assist

paper_content:

A monocular vision based rear vehicle detection and tracking system is presented for Lane Change Assist (LCA), which does not need road boundary and lane information. Our algorithm extracts regions of interest (ROI) using the shadow underneath a vehicle, and accurately localizes vehicle regions in ROI by vehicle features such as symmetry, edge and shadow underneath vehicles. The algorithm realizes vehicle verification by combining knowledge-based and learning-based methods. During vehicle tracking, templates are dynamically created on-line, tracking window is adaptively adjusted with motion estimation, and confidence is determined for tracked vehicle. The algorithm was tested under various traffic scenes at different daytime, the result illustrated good performance.
---
paper_title: Visual Perception and Tracking of Vehicles for Driver Assistance Systems

paper_content:

In this paper a driver assistance system for vehicle detection and tracking is presented. The goal of the system is to perceive the surroundings of the vehicle looking for other vehicles. Depending when they have been detected (overtaking, at long range) movement, or a geometric model of the vehicle are used. Later, the vehicle is tracked. As the algorithm receives information from a road detection module some geometric restrictions can be applied. A multi-resolution approach is used to speed up the algorithm and work in real-time. Examples of real images are shown to validate the algorithm
---
paper_title: Vehicle Detection under Various Lighting Conditions by Incorporating Particle Filter

paper_content:

We propose an automatic system to detect preceding vehicles on the highway under various lighting and different weather conditions based on the computer vision technologies. To adapt to different characteristics of vehicle appearance at daytime and nighttime, four cues including underneath, vertical edge, symmetry and taillight are fused for the preceding vehicle detection. By using particle filter with four cues through the processes including initial sampling, propagation, observation and cue fusion and evaluation, particle filter accurately generates the vehicle distribution. Thus, the proposed system can successfully detect and track preceding vehicles and be robust to different lighting conditions. Unlike normal particle filter focuses on a single target distribution in a discrete state space, we detect multiple vehicles with particle filter through a high-level tracking strategy using clustering technique called basic sequential algorithmic scheme (BSAS). Finally, experimental results for several videos from different scenes are provided to demonstrate the effectiveness of our proposed system.
---
paper_title: 6D-Vision: Fusion of Stereo and Motion for Robust Environment Perception

paper_content:

Obstacle avoidance is one of the most important challenges for mobile robots as well as future vision based driver assistance systems. This task requires a precise extraction of depth and the robust and fast detection of moving objects. In order to reach these goals, this paper considers vision as a process in space and time. It presents a powerful fusion of depth and motion information for image sequences taken from a moving observer. 3D-position and 3D-motion for a large number of image points are estimated simultaneously by means of Kalman-Filters. There is no need of prior error-prone segmentation. Thus, one gets a rich 6D representation that allows the detection of moving obstacles even in the presence of partial occlusion of foreground or background.
---
paper_title: A fusion method of data association and virtual detection for minimizing track loss and false track

paper_content:

In this paper, we present a method to track multiple moving vehicles using the global nearest neighborhood (GNN) data association (DA) based on 2D global position and virtual detection based on motion tracking. Unlikely the single target tracking, multiple target tracking needs to associate observation-to-track pairs. DA is a process to determine which measurements are used to update each track. We use the GNN data association not to lost track and not to connect incorrect measurements. GNN is a simple, robust, and optimal technique for intelligent vehicle applications with a stereo vision system that can reliably estimates the position of a vehicle. However, an incomplete detection and recognition technique bring low track maintenance due to missed detections and false alarms. A complementary virtual detection method adds to GNN method. Virtual detection is used to recover the missed detection by motion tracking when the track maintains for some periods. Motion tracking estimates virtual region of interest (ROI) of the missed detection using a pyramidal Lukas-Kanade feature tracker. Next, GNN associates the lost tracks and virtual measurements if the measurement exists in the validation gate. Our experimental results show that our tracking method works well in a stereo vision system with incomplete detection and recognition ability.
---
paper_title: Vehicle tracking and motion prediction in complex urban scenarios

paper_content:

The recognition of potentially hazardous situations on road intersections is an indispensable skill of future driver assistance systems. In this context, this study focuses on the task of vehicle tracking in combination with a long-term motion prediction (1–2 s into the future) in a dynamic scenario. A motion-attributed stereo point cloud obtained using computationally efficient feature-based methods represents the scene, relying on images of a stereo camera system mounted on a vehicle. A two-stage mean-shift algorithm is used for detection and tracking of the traffic participants. A hierarchical setup depending on the history of the tracked object is applied for prediction. This includes prediction by optical flow, a standard kinematic prediction, and a particle filter based motion pattern method relying on learned object trajectories. The evaluation shows that the proposed system is able to track the road users in a stable manner and predict their positions at least one order of magnitude more accurately than a standard kinematic prediction method.
---
paper_title: Fast detection of moving objects in complex scenarios

paper_content:

More than one third of all traffic accidents with injuries occur in urban areas, especially at intersections. A suitable driver assistance system for such complex situations requires the understanding of the scene, in particular a reliable detection of other moving traffic participants. This contribution shows how a robust and fast detection of relevant moving objects is obtained by a smart combination of stereo vision and motion analysis. This approach, called 6D Vision, estimates location and motion of pixels simultaneously which enables the detection of moving objects on a pixel level. Using a Kalman filter attached to each tracked pixel, the algorithm propagates the current interpretation to the next image. In addition, a Kalman filter based ego-motion compensation is described that takes advantage of the 6D information. This precise information enables us to discriminate between static and moving objects exactly and to obtain a better prediction. This speeds up tracking and a real-time implementation is achieved. Examples of critical situations in urban areas exhibit the potential of the 6D Vision concept which can also be extended to robotics applications.
---
paper_title: Disparity space image's features analysis for error prediction of a stereo obstacle detector for heavy duty vehicles

paper_content:

Vision-based perception has been explored as low-cost, flexible technology for industrial applications and ADAS. Its inherent flexibility presents a challenge quantifying performance and often even quantifying increases or decreases in system performance as conditions change. Experience enables designers to employ various ”rules of thumb” while commercially viable products require quantitative performance. This paper explores the correlation between features and characteristics of the Disparity Space Image (DSI) and resulting performance for an object detection application. The specific application is an object detection system suitable for highly chaotic environments often found in earthmoving industry. Features and characteristics with strong correlations can be used to improve system design and predict system performances at run-time. High-quality stereo images are used to characterize baseline system performance. These images are then artificially degraded to simulate fog, darkness, and blurring and subsequent system performance compared to baseline results.
---
paper_title: Object tracking from stereo sequences using particle filter

paper_content:

In this paper we present a vehicle tracking particle filter system based on gray histogram and sparse optical flow detection in stereo images. The proposed approach is based on the fact that for 2D tracked features we can compute their 3D correspondences, which are used for particle filter tracking improvement. The goal of this paper is to show how vision based particle filter tracking, optical flow and stereovision can be integrated to work together in order to achieve a robust object tracking algorithm.
---
paper_title: Real-time vehicle detection and tracking using stereo vision and multi-view AdaBoost

paper_content:

We propose a multi-layer, real-time vehicle detection and tracking system using stereo vision, multi-view AdaBoost detectors, and optical flow. By adopting a ground plane estimate extracted from stereo information, we generate a sparse set of hypotheses and apply trained AdaBoost classifiers in addition to fast disparity histogramming, for Hypothesis Verification (HV) purposes. Our tracking system employs one Kalman filter per detected vehicle and motion vectors from optical flow, as a means to increase its robustness. An acceptable detection rate with few false positives is obtained at 25 fps with generic hardware.
---
paper_title: Feature-Based Probabilistic Data Association (FBPDA) for visual multi-target detection and tracking under occlusions and split and merge effects

paper_content:

Uncertainties in the sensor data such as measurement noise, false detections caused by clutter, as well as merged, split, incomplete or missed detections due to a sensor malfunction or occlusions (both due to the limited sensor field of view and objects in the scene) make multi-target tracking a very complicated task. Thus one of the big challenges is track management and correct data association between detections and tracks. In this contribution we present an algorithm for visual detection and tracking of multiple extended targets under occlusions and split and merge effects. Unlike most of the state-of-the-art approaches we utilize low-level information integrating it in a unified approach based on a threshold-free probabilistic conception. The introduced scheme makes it possible to utilize information about composition of the measurements gained through tracking of dedicated feature points in the image and resolves data association ambiguities in a soft decision using a globally optimal probabilistic data association approach. Beside existence evolution consideration we also exploit the spatial and temporal relationship between stably tracked points and tracked objects, which along with observability analysis, allows us for reconstruction of compatible measurements and thus correct track update even in cases of splits, merges and partial occlusions of the tracked targets.
---
paper_title: Tracking multiple objects in urban traffic environments using dense stereo and optical flow

paper_content:

Stereo vision based sensors provide large amounts of data, a fact which is advantageous when trying to extract semantic information about the imaged scene. However, this data is corrupted by errors, caused especially by the uncertainties in the stereo reconstruction process. Temporal information can be used in order to minimize these errors. This paper presents an advanced object model, a novel association mechanism and the design of a Kalman filter based tracking algorithm, for tracking multiple objects, in complex, urban traffic scenarios.
---
paper_title: A spectral clustering and kalman filtering based objects detection and tracking using stereo vision with linear cameras

paper_content:

3D scene based objects detection and tracking is a central problem in many intelligent transportation applications. Dynamic stereo vision is the known approach to solve this problem. It consists in detecting and tracking objects from their reconstructed features using stereo images. This paper proposes a new method for detecting and tracking objects using stereo vision with linear cameras. Edge points extracted from the stereo linear images are first matched to reconstruct points that represent the objects in the scene. To detect the objects, a clustering process based on a spectral analysis is then applied to the reconstructed points. The obtained clusters are finally tracked throughout their center of gravity using Kalman filtering and a Nearest Neighbour based data association algorithm. Experimental results using real stereo linear images are shown to demonstrate the effectiveness of the proposed methods for obstacle detection and tracking in front of a vehicle.
---
paper_title: Where will the oncoming vehicle be the next second?

paper_content:

A new image based approach for fast and robust tracking of vehicles from a moving platform is presented. Position, orientation, and the full motion state including velocity, acceleration, and yaw rate of a detected vehicle are estimated from a tracked 3D point cloud. This point cloud is computed by analyzing image sequences in both space and time, i.e. by fusion of stereo vision and tracked optical flow vectors. Starting from an automated initial vehicle hypothesis, the tracking is performed by means of Extended Kalman Filter. The filter combines the knowledge of where a point in the rigid point cloud has moved within a given time interval, with the dynamic model of a vehicle. The proposed system is applied to predict the driving path of other traffic participants and runs currently at 25 Hz (VGA images) on our demonstrator vehicle UTA.
---
paper_title: Modeling and Tracking the Driving Environment With a Particle-Based Occupancy Grid

paper_content:

Modeling and tracking the driving environment is a complex problem due to the heterogeneous nature of the real world. In many situations, modeling the obstacles and the driving surfaces can be achieved by the use of geometrical objects, and tracking becomes the problem of estimating the parameters of these objects. In the more complex cases, the scene can be modeled and tracked as an occupancy grid. This paper presents a novel occupancy grid tracking solution based on particles for tracking the dynamic driving environment. The particles will have a dual nature-they will denote hypotheses, as in the particle filtering algorithms, but they will also be the building blocks of our modeled world. The particles have position and speed, and they can migrate in the grid from cell to cell, depending on their motion model and motion parameters, but they will be also created and destroyed using a weighting-resampling mechanism that is specific to particle filtering algorithms. The tracking algorithm will be centered on particles, instead of cells. An obstacle grid derived from processing a stereovision-generated elevation map is used as measurement information, and the measurement model takes into account the uncertainties of the stereo reconstruction. The resulting system is a flexible real-time tracking solution for dynamic unstructured driving environments.
---
paper_title: Moving vehicle detection by optimal segmentation of the Dynamic Stixel World

paper_content:

The reliable detection of moving objects from a moving observer is one of the most challenging and important tasks for driver assistance and safety systems. Modern sensors such as Lidar, Imaging Radar or Stereo Vision deliver range data plus longitudinal motion (Radar) or even full 3D-motion (space-time vision). Based on this data, moving objects have to be separated from the static background to be able to determine their pose and motion state. Usually, heuristics are applied to cluster the data. In order to find the most probable segmentation, we formulate the task as a hypotheses testing problem that allows taking into account various constraints and assumptions simultaneously. We show that the optimal segmentation can be efficiently found by means of dynamic programming, for an arbitrary number of objects in the scene. In this paper we concentrate on the segmentation of space-time data obtained from stereo image sequences. The vision-based depth and motion information is transferred into so called Stixels, a very compact representation of 3D scenes that can also be applied to Lidar or Radar data. It turns out that our optimal segmentation is more robust w.r.t. noisy and erroneous data.
---
paper_title: Event-driven track management method for robust multi-vehicle tracking

paper_content:

In this paper, we present an event-driven track management method to detect reliably and track robustly while minimizing missing and false detections. No state-of-the-art vehicle detection method can detect all the vehicles on the road without error. A multi-vehicle tracking method is essential to minimize the number of missing and false detections. In a multi-vehicle tracking method, there are three types of errors: false negative alarms, false positive alarms, and track identity switches. Our track management method can reduce the number of these errors remarkably while processing in real time for online application. Our track management method has four states: IDLE, PRE-TRACK, CUR-TRACK, and POST-TRACK. Most false positive alarms are removed in the PRE-TRACK state due to their sparseness. A track state transition to other states is determined by a track score. The track score is calculated by obstacle detection, vehicle recognition, detection-by-tracking, and data association. The proposed method is tested and verified with image sequences in real road environments. The experimental results demonstrate that the event-driven track management method minimizes the number of the false positive and false negative alarms remarkably compared with previous methods.
---
paper_title: Estimating the Driving State of Oncoming Vehicles From a Moving Platform Using Stereo Vision

paper_content:

A new image-based approach for fast and robust vehicle tracking from a moving platform is presented. Position, orientation, and full motion state, including velocity, acceleration, and yaw rate of a detected vehicle, are estimated from a tracked rigid 3-D point cloud. This point cloud represents a 3-D object model and is computed by analyzing image sequences in both space and time, i.e., by fusion of stereo vision and tracked image features. Starting from an automated initial vehicle hypothesis, tracking is performed by means of an extended Kalman filter. The filter combines the knowledge about the movement of the rigid point cloud's points in the world with the dynamic model of a vehicle. Radar information is used to improve the image-based object detection at far distances. The proposed system is applied to predict the driving path of other traffic participants and currently runs at 25 Hz (640 times 480 images) on our demonstrator vehicle.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: Combining monocular and stereo-vision for real-time vehicle ranging and tracking on multilane highways

paper_content:

In this paper, we introduce a novel stereo-monocular fusion approach to on-road localization and tracking of vehicles. Utilizing a calibrated stereo-vision rig, the proposed approach combines monocular detection with stereo-vision for on-road vehicle localization and tracking for driver assistance. The system initially acquires synchronized monocular frames and calculates depth maps from the stereo rig. The system then detects vehicles in the image plane using an active learning-based monocular vision approach. Using the image coordinates of detected vehicles, the system then localizes the vehicles in real-world coordinates using the calculated depth map. The vehicles are tracked both in the image plane, and in real-world coordinates, fusing information from both the monocular and stereo modalities. Vehicles' states are estimated and tracked using Kalman filtering. Quantitative analysis of tracks is provided. The full system takes 46ms to process a single frame.
---
paper_title: Mapping in dynamic environments using stereo vision

paper_content:

Next generation driver assistance systems demand a precise perception of the vicinity of the vehicle. Sensor readings are usually harnessed to gain knowledge of all moving and stationary obstacles. Commonly two paradigms are followed. Stationary environments are well modeled by non-parametric occupancy grids. Contrary, moving objects require a tracking and are not well suited for grid-based techniques. However, tracking objects requires some prior knowledge and parameterization which is inferior for unstructured cluttered obstacles. Herein we augment a grid-based mapping method designed for static environments with object tracking hence complementing both approaches. To this end we classify regions of stereo images into moving and stationary parts. The stationary part is fused in our static grid whereas moving parts are tracked yielding reliable motion estimates. The classifier used to distinguish moving from stationary parts is based on a Sequential Probability Ratio Test (SPRT), a model selection method which blends well into the tracking architecture. Thereby we achieve real-time operability on modest computing hardware.
---
paper_title: Detection and tracking of independently moving objects in urban environments

paper_content:

In this paper we propose an approach for dynamic scene perception from a moving vehicle equipped with a stereo camera rig. The approach is solely based on visual information, hence it is applicable to a large class of autonomous robots working in indoor as well as in outdoor environments. The proposed approach consists of an egomotion estimation based on disparity and optical flow using the Longuet-Higgins-Equations combined with an implicit extended Kalman-Filter. Based on this egomotion estimation a moving object detection and tracking is performed. Each tracked object is labeled with a unique ID while visible in the images. The proposed algorithm was evaluated on numerous challenging real world image sequences.
---
paper_title: Stereo-Assist: Top-down stereo for driver assistance systems

paper_content:

This paper presents a top-down approach to stereo for use in driver assistance systems. We introduce an asymmetric configuration where monocular object detection and range estimation is performed in the primary camera and then that image patch is aligned and matched in the secondary camera. The stereo distance measure from the matching assists in target verification and improved distance measurements. This approach, Stereo-Assist, shows significant advantages over the classical bottom-up stereo approach which relies on first computing a dense depth map and then using the depth map for object detection. The new approach can provide increased object detection range, reduced computational load, greater flexibility in camera configurations (we are no longer limited to side-by-side stereo configurations), greater robustness to obstructions in part of the image and mixed camera modalities FIR/VIS can be used. We show results with two novel configurations and illustrate how monocular object detection allows for simple online calibration of the stereo rig.
---
paper_title: Real-time vehicle detection and tracking using stereo vision and multi-view AdaBoost

paper_content:

We propose a multi-layer, real-time vehicle detection and tracking system using stereo vision, multi-view AdaBoost detectors, and optical flow. By adopting a ground plane estimate extracted from stereo information, we generate a sparse set of hypotheses and apply trained AdaBoost classifiers in addition to fast disparity histogramming, for Hypothesis Verification (HV) purposes. Our tracking system employs one Kalman filter per detected vehicle and motion vectors from optical flow, as a means to increase its robustness. An acceptable detection rate with few false positives is obtained at 25 fps with generic hardware.
---
paper_title: Event-driven track management method for robust multi-vehicle tracking

paper_content:

In this paper, we present an event-driven track management method to detect reliably and track robustly while minimizing missing and false detections. No state-of-the-art vehicle detection method can detect all the vehicles on the road without error. A multi-vehicle tracking method is essential to minimize the number of missing and false detections. In a multi-vehicle tracking method, there are three types of errors: false negative alarms, false positive alarms, and track identity switches. Our track management method can reduce the number of these errors remarkably while processing in real time for online application. Our track management method has four states: IDLE, PRE-TRACK, CUR-TRACK, and POST-TRACK. Most false positive alarms are removed in the PRE-TRACK state due to their sparseness. A track state transition to other states is determined by a track score. The track score is calculated by obstacle detection, vehicle recognition, detection-by-tracking, and data association. The proposed method is tested and verified with image sequences in real road environments. The experimental results demonstrate that the event-driven track management method minimizes the number of the false positive and false negative alarms remarkably compared with previous methods.
---
paper_title: Vehicle detection by means of stereo vision-based obstacles features extraction and monocular pattern analysis

paper_content:

This paper presents a stereo vision system for the detection and distance computation of a preceding vehicle. It is divided in two major steps. Initially, a stereo vision-based algorithm is used to extract relevant three-dimensional (3-D) features in the scene, these features are investigated further in order to select the ones that belong to vertical objects only and not to the road or background. These 3-D vertical features are then used as a starting point for preceding vehicle detection; by using a symmetry operator, a match against a simplified model of a rear vehicle's shape is performed using a monocular vision-based approach that allows the identification of a preceding vehicle. In addition, using the 3-D information previously extracted, an accurate distance computation is performed.
---
paper_title: Combining monocular and stereo-vision for real-time vehicle ranging and tracking on multilane highways

paper_content:

In this paper, we introduce a novel stereo-monocular fusion approach to on-road localization and tracking of vehicles. Utilizing a calibrated stereo-vision rig, the proposed approach combines monocular detection with stereo-vision for on-road vehicle localization and tracking for driver assistance. The system initially acquires synchronized monocular frames and calculates depth maps from the stereo rig. The system then detects vehicles in the image plane using an active learning-based monocular vision approach. Using the image coordinates of detected vehicles, the system then localizes the vehicles in real-world coordinates using the calculated depth map. The vehicles are tracked both in the image plane, and in real-world coordinates, fusing information from both the monocular and stereo modalities. Vehicles' states are estimated and tracked using Kalman filtering. Quantitative analysis of tracks is provided. The full system takes 46ms to process a single frame.
---
paper_title: A General Active-Learning Framework for On-Road Vehicle Recognition and Tracking

paper_content:

This paper introduces a general active-learning framework for robust on-road vehicle recognition and tracking. This framework takes a novel active-learning approach to building vehicle-recognition and tracking systems. A passively trained recognition system is built using conventional supervised learning. Using the query and archiving interface for active learning (QUAIL), the passively trained vehicle-recognition system is evaluated on an independent real-world data set, and informative samples are queried and archived to perform selective sampling. A second round of learning is then performed to build an active-learning-based vehicle recognizer. Particle filter tracking is integrated to build a complete multiple-vehicle tracking system. The active-learning-based vehicle-recognition and tracking (ALVeRT) system has been thoroughly evaluated on static images and roadway video data captured in a variety of traffic, illumination, and weather conditions. Experimental results show that this framework yields a robust efficient on-board vehicle recognition and tracking system with high precision, high recall, and good localization.
---
paper_title: Learning multi-lane trajectories using vehicle-based vision

paper_content:

Safe operation of a motor vehicle requires awareness of the current traffic situation as well as the ability to predict future maneuvers. In order to provide an intelligent vehicle the ability to make predictions, this work proposes a framework for understanding the driving situation based on vehicle mounted vision sensors. Vehicles are tracked using Kalman filtering based on a vision-based system that detects and tracks using a combination of monocular and stereo-vision. The vehicles' full trajectories are recorded, and a data-driven learning framework has been applied to automatically learn surround behaviors. By learning based on observations, the ADAS system is being trained by experience. Learned trajectories have been compared between dense and free-flowing traffic conditions. Preliminary experimental results using real-world multi-lane highways show the basic promise of this approach. Future research directions are discussed.
---
paper_title: 3D Vision: Developing an Embedded Stereo-Vision System

paper_content:

The DeepSea G2 stereo vision system that Tyzx developed features an embedded stereo camera consisting of two CMOS imagers, a Tyzx DeepSea 2 stereo application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a DSP/coprocessor, a PowerPC running Linux, and an Ethernet connection. About the size of a hardback book, the G2 delivers real-time - 30 frames per second (fps) - interpretations of visual data even in challenging environments and has been deployed in a variety of applications, including person-tracking and autonomous-vehicle navigation.
---
paper_title: Real-time vehicle and lane detection with embedded hardware

paper_content:

For autonomously acting robots and driver assistance systems powerful optical stereo sensor systems are required. Object positions and environmental conditions have to be acquired in real-time. In this paper an algorithm based on a hardware-software co-design is applied. A depth-map is generated with a hierarchical detection method. A depth-histogram is generated by using the density distribution of the disparity in the depth-map. It is used for object detection. The object clustering can be accomplished without calculation of 3D-points, due to the almost identical mapping of the objects over the whole distance, within the histogram. A lane detection is applied by using a Hough transform. The suitability at night and the detection of small objects like bikers is proven.
---
paper_title: Accurate and efficient stereo processing by semi-global matching and mutual information

paper_content:

This paper considers the objectives of accurate stereo matching, especially at object boundaries, robustness against recording or illumination changes and efficiency of the calculation. These objectives lead to the proposed semi-global matching method that performs pixelwise matching based on mutual information and the approximation of a global smoothness constraint. Occlusions are detected and disparities determined with sub-pixel accuracy. Additionally, an extension for multi-baseline stereo images is presented. There are two novel contributions. Firstly, a hierarchical calculation of mutual information based matching is shown, which is almost as fast as intensity based matching. Secondly, an approximation of a global cost calculation is proposed that can be performed in a time that is linear to the number of pixels and disparities. The implementation requires just 1 second on typical images.
---
paper_title: Efficient occupancy grid computation on the GPU with lidar and radar for road boundary detection

paper_content:

Accurate maps of the static environment are essential for many advanced driver-assistance systems. A new method for the fast computation of occupancy grid maps with laser range-finders and radar sensors is proposed. The approach utilizes the Graphics Processing Unit to overcome the limitations of classical occupancy grid computation in automotive environments. It is possible to generate highly accurate grid maps in just a few milliseconds without the loss of sensor precision. Moreover, in the case of a lower resolution radar sensor it is shown that it is suitable to apply super-resolution algorithms to achieve the accuracy of a higher resolution laser-scanner. Finally, a novel histogram based approach for road boundary detection with lidar and radar sensors is presented.
---
paper_title: Histograms of oriented gradients for human detection

paper_content:

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.
---
paper_title: Real-time semi-global matching disparity estimation on the GPU

paper_content:

This paper presents the design, implementation and evaluation of new parallelization schemes for performing dense disparity estimation based on non-parametric rank transform and semi-global matching on Graphics Processing Units (GPUs). A detailed analysis of the performance limitating factors (memory throughput, instruction throughput, etc.) for each part of the parallel implementation is performed. Thus, a highly optimized mapping for each parallelization scheme onto the resources of the GPU is obtained. The resulting implementation performs disparity estimation at 27 frames per second for 1024×768 pixel images with 128 disparity levels on a Nvidia Tesla C2050 GPU.
---
paper_title: GPU & CPU cooperative accelerated pedestrian and vehicle detection

paper_content:

This paper presents a fast pedestrian and vehicle detection framework that integrates GPU (graphics processing unit) and CPU implementations. We employ the Histograms of Oriented Gradients (HoG) and the Feature Interaction Descriptor (FIND) as object descriptors. FIND describes the high-level properties of an object's appearance by computing pair-wise interactions of adjacent region-level features. We also employ the cascade approach in a sliding-window manner with multi-classifiers specialized for both the direction of a pedestrian and the distance of the pedestrian to a camera installed on a vehicle. Although our detection framework can detect pedestrians and vehicles in images, it wastes computational cost on calculating high-dimensional FIND features. Therefore, to realize real-time processing, we utilize the NVIDIA CUDA framework for pedestrian and vehicle detection. Three parallelization techniques implemented on both a GPU and a CPU are incorporated in our detection framework. As a result, our proposed implementation can perform detection more than 30 times faster than can a conventional implementation running on a CPU. For a 640×480 image, our parallel techniques attain a processing speed of 23.8 fps (42 [ms]) and detects both pedestrians and vehicles.
---
paper_title: Development of an Embedded Vision based Vehicle Detection System using an ARM Video Processor

paper_content:

Current Intelligent Transportation Systems tends to be integrated in smart environments where sensors are provided with processing and communication capabilities. This is the case of the vehicle detection system proposed in this paper. The ARM-based video processor not only deals with the video processing algorithms, but also takes advantage of the networking capabilities using an embedded operating system. Consequently, the final prototype implements the vehicle detection system as the main functionality, but offers additional features like remote detection area configuration, video delivery, remote software updating, etc. Results will show all the system capabilities as well as satisfactory vehicle detection rates.
---
paper_title: Shadow-based vehicle model refinement and tracking in advanced automotive driver assistance systems

paper_content:

Vision-based automotive driver assistance systems have to cope with complex outdoor illumination conditions. Many applications for vehicle detection and tracking consider shadows caused by the sunlight as distracting effects and try to computationally compensate or disregard them. In this paper we suggest a shape-from-shadow approach which uses cast shadows as additional supporting information for vehicle model refinement and tracking. To take the position of the sun into account we only use sensor systems that mid-range vehicles are normally equipped with. Analysing shadows is a suitable method to support rear-view based vehicle tracking. A vehicle's shadow turned out to be a strong feature since it is possible to track a vehicle solely based on its shadow. Another benefit is that we are able to set up and refine three-dimensional shape models of vehicles driving ahead in the same lane. In selected situations it is possible to visually track two vehicles driving on the same lane one after another.
---
paper_title: Night-time vehicle classification with an embedded, vision system

paper_content:

The paper presents night-time vehicle classification using an embedded vision system based on an optical transient sensor. This neuromorphic sensor features an array of 128×128 pixels that respond to relative light intensity changes with low latency and high dynamic range. The proposed algorithm exploits the temporal resolution and sparse representation of the data, delivered by the sensor in the data-driven Address-Event Representation (AER) format, to efficiently implement a robust classification of vehicles into two classes, car-like and truck-like, during night-time operation. The classification is based on the extraction of the positions and distances of the vehicles head lights to estimate vehicle width. We present the algorithm, test data and an evaluation of the classification accuracy by comparison of the test data with ground truth from video annotation and reference results from a state-of-the-art ultrasonic/radar-combination reference detector. The results show that the difference in total truck counts with respect to a reference detector and to manually annotated video during nighttime operation under dry and wet road conditions is typically below 6%.
---
paper_title: A Visibility-Based Approach for Occupancy Grid Computation in Disparity Space

paper_content:

Occupancy grids are a very convenient tool for environment representation in robotics. This paper will detail a novel approach for computing occupancy grids from stereo vision and show its application to intelligent vehicles. In the proposed approach, occupancy is initially computed directly in the stereoscopic sensor's disparity space. The calculation formally accounts for the detection of obstacles and road pixels in disparity space, as well as partial occlusions in the scene. In the second stage, this disparity-space occupancy grid is transformed into a Cartesian space occupancy grid to be used by subsequent applications. This transformation includes spatial and temporal filtering. The proposed method is designed to easily be processed in parallel. Consequently, we chose to implement it on a graphics processing unit, which allows real-time processing for demanding applications. In this paper, we present this method, and we propose an application to the problem of perception in a road environment. Results are presented with real road data, qualitatively comparing this approach with other methods.
---
paper_title: Towards a human-like vision system for Driver Assistance

paper_content:

Several advanced driver assistance systems realizing elementary perception and analysis tasks have been introduced to market in recent years. For example, collision mitigation brake systems detect the distance and relative velocity of vehicles in front to assess the risk of a rear-end collision in a clearly defined following situation. In order to go beyond such elementary analysis tasks, todaypsilas research is focusing more and more on powerful perception systems for driver assistance. We believe computer vision will play a central role for achieving a full understanding of generic traffic situations. Besides individual processing algorithms, general vision architectures enabling integrated and more flexible processing are needed. Here we present the first instantiation of a vision architecture for driver assistance systems inspired by the human visual system that is based on task-dependent perception. Core element of our system is a state of the art attention system integrating bottom-up and top-down visual saliency. Combining this task-dependent tunable visual saliency with object recognition and tracking enables for instance warnings according to the context of the scene. We demonstrate the performance of our approach in a construction site setup, where a traffic jam ending within the site is a dangerous situation that the system has to identify in order to warn the driver.
---
paper_title: On-road vehicle detection fusing radar and vision

paper_content:

This paper presents a cross-verification approach to fuse radar and vision data for vehicle detection. Firstly, a realtime vision approach using specific shadow segmentation is used to detect vehicles in whole image independently. The fusion approach contains two steps: matching and validation. The targets respectively from radar and vision verify each other in matching process. Then the unmatched radar targets are validated by vision data once again. Experiment results with test dataset from real traffic scenes on freeway and urban roads are presented to illustrate the performance of this approach.
---
paper_title: Tightly-coupled LIDAR and computer vision integration for vehicle detection

paper_content:

In many driver assistance systems and autonomous driving applications, both LIDAR and computer vision (CV) sensors are often used to detect vehicles. LIDAR provides excellent range information to different objects. However, it is difficult to recognize these objects as vehicles from range information alone. On the other hand, computer vision imagery allows for better recognition, but does not provide high-resolution range information. In this paper, a tightly-coupled LIDAR/CV integrated system is proposed for vehicle detection. This sensing system is mounted on the front of a test vehicle, facing forward. The LIDAR sensor estimates possible vehicle positions. This information is then transformed into the image coordinates. Different Regions of Interest (ROIs) in the imagery are defined based on the LIDAR object hypotheses. An Adaboost object classifier is then utilized to detect the vehicle in ROIs. A classifier error correction approach is used to choose an optimal position of the detected vehicle. Finally, the vehicle's position and dimensions are derived from both the LIDAR and image data. The main contribution of this paper is that the complementary advantages of two sensors are utilized. The LIDAR scanning data are applied for classifier correction. And the output of the Adaboost classifier is used to provide distance and dimension information. Experimental results are presented to illustrate that this LIDAR/CV system is reliable. It can be used in applications such as traffic surveillance and roadway navigation tasks.
---
paper_title: Incremental Online Object Learning in a Vehicular Radar-Vision Fusion Framework

paper_content:

In this paper, we propose an object learning system that incorporates sensory information from an automotive radar system and a video camera. The radar system provides coarse attention for the focus of visual analysis on relatively small areas within the image plane. The attended visual areas are coded and learned by a three-layer neural network utilizing what is called in-place learning: Each neuron is responsible for the learning of its own processing characteristics within the connected network environment, through inhibitory and excitatory connections with other neurons. The modeled bottom-up, lateral, and top-down connections in the network enable sensory sparse coding, unsupervised learning, and supervised learning to occur concurrently. This paper is applied to learn two types of encountered objects in multiple outdoor driving settings. Cross-validation results show that the overall recognition accuracy is above 95% for the radar-attended window images. In comparison with the uncoded representation and purely unsupervised learning (without top-down connection), the proposed network improves the overall recognition rate by 15.93% and 6.35%, respectively. The proposed system is also compared favorably with other learning algorithms. The result indicates that our learning system is the only one that is fit for incremental and online object learning in a real-time driving environment.
---
paper_title: A Radar Guided Vision System for Vehicle Validation and Vehicle Motion Characterization

paper_content:

This paper describes a radar-guided monocular vision system that detects, validates, and tracks the preceding vehicle and thus predicts its lane-change intentions. A vision-based lane tracking process is developed to create a stable motion model in order to map the radar targets to image coordinates and consequently generate the region of interest (ROI) to search for a potential preceding vehicle. Model-based object classification algorithms are then applied to validate the existence of a vehicle in this ROI. Once the detected primary target vehicle, which is in the same lane as the host vehicle, is validated, it will be continuously tracked until it leaves the host lane. The spatial-temporal tracking history of the primary target vehicle is used to infer its intention of changing lanes. This prediction results from a characterization process for the primary target motion vector. The radar-vision integrated system has been evaluated on real-world data collected using a test vehicle equipped with a radar sensor, vision sensor, and a host processor.
---
paper_title: Obstacle detection and classification fusing radar and vision

paper_content:

This paper presents a system whose aim is to detect and classify road obstacles, like pedestrians and vehicles, by fusing data coming from different sensors: a camera, a radar, and an inertial sensor. The camera is mainly used to refine the vehiclespsila boundaries detected by the radar and to discard those who might be false positives; at the same time, a symmetry based pedestrian detection algorithm is executed, and its results are merged with a set of regions of interest, provided by a Motion Stereo technique.
---
paper_title: Hypothesis based vehicle detection for increased simplicity in multi-sensor ACC

paper_content:

Systems for adaptive cruise control (ACC) become increasingly complex in case multiple sensors are used. The search space, detection error and run-time may increase substantially due to combinatory explosion of methods and data. This paper presents a method that simplifies fusion between range and vision devices using corresponding sets of hypotheses. A system is proposed that combines three modules: one uses output of a 24 GHz radar device, one uses single images from a monocular camera system; and one uses the image sequence data of the same system. The radar detection module uses condensation tracking. The vehicle detection module uses scaled symmetry detection. The three modules are fused by sharing sets of hypotheses for detection of vehicles. Results show 96% error reduction with respect to range sensing only and 63% detection increase due to tracking.
---
paper_title: Stereo vision based full-range object detection and tracking

paper_content:

Active safety applications can be divided into two categories according to the sensor technology the system uses: short range applications and long range applications. Short range applications require reliable sensing of a wider field-of-view but limited range, while long range applications require accurate sensing of a long range but within a limited FOV. In this paper, we present a stereo vision based system that is capable of detecting and tracking objects in full range and wide FOV, which effectively enables both short range and long range applications to happen simultaneously in a single system. The system runs at 20 Hz on embedded hardware, computing 1280×456 range maps, road surface estimation, object detection, segmentation and tracking. For vehicle sized objects, the system has a detection capability of 3 to 160 meters, which is needed for the operation of collision warning/ avoidance systems.
---
paper_title: IMMPDA vehicle tracking system using asynchronous sensor fusion of radar and vision

paper_content:

This paper focuses on recognition and tracking of maneuvering vehicles in dense traffic situations. We present an asynchronous multi obstacle multi sensor tracking method that fuses information from radar and monocular vision. A low level fusion method is integrated into the framework of an IMMPDA Kalman filter. Real world experiments demonstrate that the system combines the complementary strengths of the employed sensors.
---
paper_title: Radar and vision based data fusion - Advanced filtering techniques for a multi object vehicle tracking system

paper_content:

The robust and reliable detection of objects in the path of a vehicle is an important prerequisite for collision avoidance and collision mitigation systems. In this paper, an ego-motion compensated tracking approach is presented which combines radar observations with the results of a contour-based image processing algorithm. The approach is able to handle all uncertainties of the system in a unified way without analytical linearization by using the Unscented transform. By that, the covariances of the system can be estimated more accurately. The paper describes both the image processing and the state estimation algorithms. Furthermore, results of several practical tests are presented.
---
paper_title: Collision sensing by stereo vision and radar sensor fusion

paper_content:

To take the advantages of both stereo cameras and radar, this paper proposes a fusion approach to accurately estimate the location, size, pose and motion information of a threat vehicle with respect to the host from observations obtained by both sensors. To do that, we first fit the contour of a threat vehicle from stereo depth information, and find the closest point on the contour from the vision sensor. Then the fused closest point is obtained by fusing radar observations and the vision closest point. Next by translating the fitted contour to the fused closest point, the fused contour is obtained. Finally the fused contour is tracked by using the rigid body constraints to estimate the location, size, pose and motion of the threat vehicle. Experimental results from both the synthetic data and the real world road test data demonstrate the success of the proposed algorithm.
---
paper_title: Visual confirmation of mobile objects tracked by a multi-layer lidar

paper_content:

Integrity of the information provided by a perception system is crucial for advanced driver assistance systems intended for safety applications, like obstacle avoidance systems. A method to ensure integrity is to use different kinds of perception sources. Lidars are key sensors for multiple objects detection and tracking. Stereo vision systems (SVS) can be used to improve the tracking but, in this paper, we use also SVS to confirm the real existence of potential obstacles thanks to 3D dense reconstruction in focused regions of interest. Synchronization issues between the different sensors are addressed using predictive filtering. The proposed approach is evaluated in real conditions thanks to five use cases relevant to urban situations. Results show that this visual confirmation strategy is efficient.
---
paper_title: Radar-Vision Based Vehicle Recognition with Evolutionary Optimized and Boosted Features

paper_content:

We present a real-time monocular vehicle detection and recognition system for driver assistance based on the fusion of data from a radar and a video sensor. The radar data is used both for narrowing down the size of the search area for vehicle rears on the video image and for the distance measurement of the vehicles in front. Using the passive video sensor a radar object is verified and the width as well as the lateral position of the vehicle are determined. The contribution of this work is threefold: At first, we present and apply a methodology for developing a novel evolutionary optimized symmetry measure. Secondly, we demonstrate a vehicle detection and recognition algorithm consisting of two steps: hypothesis generation using a detector based on a set of Haar-like filters and an AdaBoost learning algorithm and hypothesis verification using an evolutionary optimized and biologically motivated vehicle recognition system. Finally, the performance of both the individual components and the complete vehicle detection and recognition system is evaluated by not only using classical confusion matrices but also giving information on the accuracy of the width and lateral position sensing. Our experimental results demonstrate a robust and real-time system trained and tested on more than 30,000 images.
---
paper_title: An analysis of synergy effects in an omnidirectional modular perception system

paper_content:

The increasing complexity of Advanced Driver Assistance Systems (ADAS) requires new design strategies in order to provide the necessary reliability for safety-related applications. In this paper, the connection of several perception modules to a modular ADAS is proposed. The benefits of such an approach are analyzed on the example of a vision and radar based vehicle & lane detection system. In particular, the effect of algorithmic synergies between the modules is investigated. The performance of the modular perception system is demonstrated using real-world data from the concept vehicle Carai. The results show that the performance and reliability of this approach surpasses the properties of the single modules.
---
paper_title: Data fusion for overtaking vehicle detection based on radar and optical flow

paper_content:

Trustworthiness is a key point when dealing with vehicle safety applications. In this paper an approach to a real application is presented, able to fulfill the requirements of such demanding applications. Most of commercial sensors available nowadays are usually designed to detect front vehicles but lack the ability to detect overtaking vehicles. The work presented here combines the information provided by two sensors, a Stop&Go radar and a camera. Fusion is done by using the unprocessed information from the radar, and computer vision based on optical flow. The basic capabilities of the commercial systems are upgraded giving the possibility to improve the front vehicles detection system, by detecting overtaking vehicles with a high positive rate.
---
paper_title: Sensorfusion Using Spatio-Temporal Aligned Video and Lidar for Improved Vehicle Detection

paper_content:

This contribution introduces a novel approach to cross-calibrate automotive vision and ranging sensors. The resulting sensor alignment allows the incorporation of multiple sensor data into a detection and tracking framework. Exemplarily, we show how a realtime vehicle detection system, intended for emergency breaking or ACC applications, benefits from the low level fusion of multibeam lidar and vision sensor measurements in discrimination performance and computational complexity.
---
paper_title: A Lidar and Vision-based Approach for Pedestrian and Vehicle Detection and Tracking

paper_content:

This paper presents a sensorial-cooperative architecture to detect, track and classify entities in semi-structured outdoor scenarios for intelligent vehicles. In order to accomplish this task, information provided by in-vehicle Lidar and monocular vision is used. The detection and tracking phases are performed in the laser space, and the object classification methods work both in laser space (using a Gaussian Mixture Model classifier) and in vision spaces (AdaBoost classifier). A Bayesian-sum decision rule is used in order to combine the results of both classification techniques, and hence a more reliable object classification is achieved. Experiments confirm the effectiveness of the proposed architecture.
---
paper_title: Vehicle and Guard Rail Detection Using Radar and Vision Data Fusion

paper_content:

This paper describes a vehicle detection system fusing radar and vision data. Radar data are used to locate areas of interest on images. Vehicle search in these areas is mainly based on vertical symmetry. All the vehicles found in different image areas are mixed together, and a series of filters is applied in I order to delete false detections. In order to speed up and improve system performance, guard rail detection and a method to manage overlapping areas are also included. Both methods are explained and justified in this paper. The current algorithm analyzes images on a frame-by-frame basis without any temporal correlation. Two different statistics, namely 1) frame based and 2) event based, are computed to evaluate vehicle detection efficiency, while guard rail detection efficiency is computed in terms of time savings and correct detection rates. Results and problems are discussed, and directions for future enhancements are provided.
---
paper_title: Vehicle environment detection by a combined low and mid level fusion of a laser scanner and stereo vision

paper_content:

A sensor system consisting of a multi layer laser scanner and a stereo camera is used to observe the surrounding environment of a vehicle. By a novel multi level fusion framework the heterogeneous sensor data can be merged on different processing levels. The following paper gives a short introduction to the processing steps of the sensor data and focuses on the lower raw level and middle object level fusion. Here, the low level fusion obtains a more precise object discrimination, whereas the mid level fusion is able to solve conflicting sensor statements with competing object information. A combination of both, obviously results in a more accurate environmental description. The effectiveness of both fusion approaches is tested and compared with real road reference data, obtained from highly precise GPS data.
---
paper_title: Towards fully autonomous driving: Systems and algorithms

paper_content:

In order to achieve autonomous operation of a vehicle in urban situations with unpredictable traffic, several realtime systems must interoperate, including environment perception, localization, planning, and control. In addition, a robust vehicle platform with appropriate sensors, computational hardware, networking, and software infrastructure is essential.
---
paper_title: Monocular Video serves RADAR-based Emergency Braking

paper_content:

Much work was carried out recently for emergency braking based on radar signals. The key step for emergency braking is the reliable detection of obstacles. Moving objects are verified as such by tracking the radar signal. However, discarding so-called phantom objects remains a challenge for stationary objects. This leads to the question of sensor fusion for more reliable verification of obstacles. In this paper we propose a novel method using a monocular camera, such as the night view camera in the Mercedes S class. Our two goals in this paper are the verification of obstacles and the detection of obstacle boundaries. This allows analysis of the situation for carrying out emergency braking. The verification of obstacles is done by analyzing the scaling of obstacles as they get closer to the camera. The perspective image motion of the ground plane serves as a counter hypothesis to detect phantom objects. Obstacle boundaries are found by graph cut segmentation on these two motion fields.
---
paper_title: Vision-based lateral position improvement of RADAR detections

paper_content:

Detecting vehicles ahead by sensors mounted on an ego-vehicle is an essential element of Advanced Driver Assistance Systems (ADAS). Although millimeter-wave radar sensors are very robust at detecting vehicles, the lateral position resolution is very low. However, a more precise lateral position would not only improve the performance of existing ADAS, but it would also enable a wide range of additional applications. In this paper, we propose a method for improving the lateral position of radar sensor detections by fusing them with a vision sensor. Our method is based on a robust symmetry axis detection of vehicles' rear sides. In contrast to the typical approach of detecting symmetry by means of edges, we use a patch-matching-based approach together with a robust cost function. The initial lateral position of the radar detection is improved by conducting a local symmetry search in the image region corresponding to the objects detected by the radar. The experimental results show that the proposed method is able to improve the lateral position accuracy by a factor of seven. Furthermore, additional consistency checks discover miss-detections of the symmetry axis, thus ensuring that the lateral position never drops below the accuracy of the radar detection.
---
paper_title: Efficient Resource Allocation for Attentive Automotive Vision Systems

paper_content:

We describe a novel architecture for automotive vision organized on five levels of abstraction, i.e., sensor, data, semantic, reasoning, and resource allocation levels, respectively. Although we implement and evaluate processes to detect and classify other participants within the immediate environment of a moving vehicle, our main emphasis is on the allocation of computational resource and attentive processing by the sensor suite. To that end, an efficient multiobjective resource allocation method is formalized and implemented. This includes a decision-making process dependent upon the environment, the current goal, the available sensors and computational resource, and the time available to make a decision. We evaluate our approach on road traffic test sequences acquired by a test vehicle provided by Audi. This vehicle includes lidar, video, radar, and sonar sensors, in addition to conventional global positioning system (GPS) navigation, but our evaluation is confined to lidar and video data alone.
---
paper_title: Frontal object perception using radar and mono-vision

paper_content:

In this paper, we detail a complete software architecture of a key task that an intelligent vehicle has to deal with: frontal object perception. This task is solved by processing raw data of a radar and a mono-camera to detect and track moving objects. Data sets obtained from highways, country roads and urban areas were used to test the proposed method. Several experiments were conducted to show that the proposed method obtains a better environment representation, i.e., reduces the false alarms and missed detections from individual sensor evidence.
---
paper_title: Long-term vehicle motion prediction

paper_content:

Future driver assistance systems will have to cope with complex traffic situations, especially in the road crossing scenario. To detect potentially hazardous situations as early as possible, it is therefore desirable to know the position and motion of the ego-vehicle and vehicles around it for several seconds in advance. For this purpose, we propose in this study a long-term prediction approach based on a combined trajectory classification and particle filter framework. As a measure for the similarity between trajectories, we introduce the quaternion-based rotationally invariant longest common subsequence (QRLCS) metric. The trajectories are classified by a radial basis function (RBF) classifier with an architecture that is able to process trajectories of arbitrary non-uniform length. The particle filter framework simultaneously tracks and assesses a large number of motion hypotheses (∼102), where the class-specific probabilities estimated by the RBF classifier are used as a-priori probabilities for the hypotheses of the particle filter. The hypotheses are clustered with a mean-shift technique and are assigned a likelihood value. Motion prediction is performed based on the cluster centre with the highest likelihood. While traditional motion prediction based on curve radius and acceleration is inaccurate especially during turning manoeuvres, we show that our approach achieves a reasonable motion prediction even for long prediction intervals of 3 s for these complex motion patterns.
---
paper_title: Object-oriented Bayesian networks for detection of lane change maneuvers

paper_content:

In this paper we introduce a novel approach towards the recognition of typical driving maneuvers in structured highway scenarios and identify some of the key benefits of traffic scene modeling with object-oriented Bayesian networks (OOBNs). The approach exploits the advantages of an introduced lane-related coordinate system together with individual occupancy grids for all vehicles. This combination allows for an efficient classification of the existing vehiclelane and vehicle-vehicle relations in a traffic scene and thus substantially improves the understanding of complex traffic scenes. We systematically propagate probabilities and variances within our network which results in probabilistic sets of the modeled driving maneuvers. Using this generic approach, we are able to classify a total of 27 driving maneuvers including merging and object following.
---
paper_title: Data fusion for overtaking vehicle detection based on radar and optical flow

paper_content:

Trustworthiness is a key point when dealing with vehicle safety applications. In this paper an approach to a real application is presented, able to fulfill the requirements of such demanding applications. Most of commercial sensors available nowadays are usually designed to detect front vehicles but lack the ability to detect overtaking vehicles. The work presented here combines the information provided by two sensors, a Stop&Go radar and a camera. Fusion is done by using the unprocessed information from the radar, and computer vision based on optical flow. The basic capabilities of the commercial systems are upgraded giving the possibility to improve the front vehicles detection system, by detecting overtaking vehicles with a high positive rate.
---
paper_title: On the Roles of Eye Gaze and Head Dynamics in Predicting Driver's Intent to Change Lanes

paper_content:

Driver behavioral cues may present a rich source of information and feedback for future intelligent advanced driver-assistance systems (ADASs). With the design of a simple and robust ADAS in mind, we are interested in determining the most important driver cues for distinguishing driver intent. Eye gaze may provide a more accurate proxy than head movement for determining driver attention, whereas the measurement of head motion is less cumbersome and more reliable in harsh driving conditions. We use a lane-change intent-prediction system (McCall et al., 2007) to determine the relative usefulness of each cue for determining intent. Various combinations of input data are presented to a discriminative classifier, which is trained to output a prediction of probable lane-change maneuver at a particular point in the future. Quantitative results from a naturalistic driving study are presented and show that head motion, when combined with lane position and vehicle dynamics, is a reliable cue for lane-change intent prediction. The addition of eye gaze does not improve performance as much as simpler head dynamics cues. The advantage of head data over eye data is shown to be statistically significant (p < 0.01) 3 s ahead of lane-change situations, indicating that there may be a biological basis for head motion to begin earlier than eye motion during "lane-change"-related gaze shifts.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: Critical Motion Detection of Nearby Moving Vehicles in a Vision-Based Driver-Assistance System

paper_content:

Driving always involves risk. Various means have been proposed to reduce the risk. Critical motion detection of nearby moving vehicles is one of the important means of preventing accidents. In this paper, a computational model, which is referred to as the dynamic visual model (DVM), is proposed to detect critical motions of nearby vehicles while driving on a highway. The DVM is motivated by the human visual system and consists of three analyzers: 1) sensory analyzers, 2) perceptual analyzers, and 3) conceptual analyzers. In addition, a memory, which is called the episodic memory, is incorporated, through which a number of features of the system, including hierarchical processing, configurability, adaptive response, and selective attention, are realized. A series of experimental results with both single and multiple critical motions are demonstrated and show the feasibility of the proposed system.
---
paper_title: Learning lane change trajectories from on-road driving data

paper_content:

Lane change is one of the most principle driving behaviors on structure roads. It frequently happens in daily driving. A key issue in lane change technique is trajectory planning, where a set of trajectories describing possible vehicle motions are generated by applying a parametric function, and by uniformly sampling the end states in configuration space; the trajectories are then examined to find an optimal one for execution. However, such a trajectory set has poor efficiency due to the large sample number. Many trajectories in this set seldom happen in real human driving behaviors. In this research, lane change trajectories are collected from real driving data of different drivers. Their statistics are analyzed, through which, a simplified trajectory set is generated. Experiment results show that the trajectory set has much less number of samples but can still guarantee to cover usual lane change behaviors of human being.
---
paper_title: Vehicle tracking and motion prediction in complex urban scenarios

paper_content:

The recognition of potentially hazardous situations on road intersections is an indispensable skill of future driver assistance systems. In this context, this study focuses on the task of vehicle tracking in combination with a long-term motion prediction (1–2 s into the future) in a dynamic scenario. A motion-attributed stereo point cloud obtained using computationally efficient feature-based methods represents the scene, relying on images of a stereo camera system mounted on a vehicle. A two-stage mean-shift algorithm is used for detection and tracking of the traffic participants. A hierarchical setup depending on the history of the tracked object is applied for prediction. This includes prediction by optical flow, a standard kinematic prediction, and a particle filter based motion pattern method relying on learned object trajectories. The evaluation shows that the proposed system is able to track the road users in a stable manner and predict their positions at least one order of magnitude more accurately than a standard kinematic prediction method.
---
paper_title: Vehicle Detection and Tracking in Car Video Based on Motion Model

paper_content:

This paper aims at real-time in-car video analysis to detect and track vehicles ahead for safety, autodriving, and target tracing. This paper describes a comprehensive approach to localizing target vehicles in video under various environmental conditions. The extracted geometry features from the video are continuously projected onto a 1-D profile and are constantly tracked. We rely on temporal information of features and their motion behaviors for vehicle identification, which compensates for the complexity in recognizing vehicle shapes, colors, and types. We probabilistically model the motion in the field of view according to the scene characteristic and the vehicle motion model. The hidden Markov model (HMM) is used to separate target vehicles from the background and track them probabilistically. We have investigated videos of day and night on different types of roads, showing that our approach is robust and effective in dealing with changes in environment and illumination and that real-time processing becomes possible for vehicle-borne cameras.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: Critical Motion Detection of Nearby Moving Vehicles in a Vision-Based Driver-Assistance System

paper_content:

Driving always involves risk. Various means have been proposed to reduce the risk. Critical motion detection of nearby moving vehicles is one of the important means of preventing accidents. In this paper, a computational model, which is referred to as the dynamic visual model (DVM), is proposed to detect critical motions of nearby vehicles while driving on a highway. The DVM is motivated by the human visual system and consists of three analyzers: 1) sensory analyzers, 2) perceptual analyzers, and 3) conceptual analyzers. In addition, a memory, which is called the episodic memory, is incorporated, through which a number of features of the system, including hierarchical processing, configurability, adaptive response, and selective attention, are realized. A series of experimental results with both single and multiple critical motions are demonstrated and show the feasibility of the proposed system.
---
paper_title: Object flow: A descriptor for classifying traffic motion

paper_content:

We present and evaluate a novel scene descriptor for classifying urban traffic by object motion. Atomic 3D flow vectors are extracted and compensated for the vehicle's egomotion, using stereo video sequences. Votes cast by each flow vector are accumulated in a bird's eye view histogram grid. Since we are directly using low-level object flow, no prior object detection or tracking is needed. We demonstrate the effectiveness of the proposed descriptor by comparing it to two simpler baselines on the task of classifying more than 100 challenging video sequences into intersection and non-intersection scenarios. Our experiments reveal good classification performance in busy traffic situations, making our method a valuable complement to traditional approaches based on lane markings.
---
paper_title: Learning multi-lane trajectories using vehicle-based vision

paper_content:

Safe operation of a motor vehicle requires awareness of the current traffic situation as well as the ability to predict future maneuvers. In order to provide an intelligent vehicle the ability to make predictions, this work proposes a framework for understanding the driving situation based on vehicle mounted vision sensors. Vehicles are tracked using Kalman filtering based on a vision-based system that detects and tracks using a combination of monocular and stereo-vision. The vehicles' full trajectories are recorded, and a data-driven learning framework has been applied to automatically learn surround behaviors. By learning based on observations, the ADAS system is being trained by experience. Learned trajectories have been compared between dense and free-flowing traffic conditions. Preliminary experimental results using real-world multi-lane highways show the basic promise of this approach. Future research directions are discussed.
---
paper_title: A probabilistic model for estimating driver behaviors and vehicle trajectories in traffic environments

paper_content:

This paper presents a filter that is able to simultaneously estimate the behaviors of traffic participants and anticipate their future trajectories. This is achieved by recognizing the type of situation derived from the local situational context, which subsumes all information relevant for the drivers decision making. By explicitly taking into account the interactions between vehicles, it achieves a comprehensive situational understanding, inevitable for autonomous vehicles and driver assistance systems. This provides the necessary information for safe behavior decision making or motion planning. The filter is modeled as a Dynamic Bayesian Network. The factored state space, modeling the causal dependencies, allows to describe the models in a compact fashion and reduces the computational complexity of the inference process. The filter is evaluated in the context of a highway scenario, showing a good performance even with very noisy measurements. The presented framework is intended to be used in traffic environments but can be easily transferred to other robotic domains.
---
paper_title: 3D pose estimation of vehicles using a stereo camera

paper_content:

This study introduces an approach to three-dimensional vehicle pose estimation using a stereo camera system. After computation of stereo and optical flow on the investigated scene, a four-dimensional clustering approach separates the static from the moving objects in the scene. The iterative closest point algorithm (ICP) estimates the vehicle pose using a cuboid as a weak vehicle model. In contrast to classical ICP optimisation a polar distance metric is used which especially takes into account the error distribution of the stereo measurement process. The tracking approach is based on tracking-by-detection such that no temporal filtering is used. The method is evaluated on seven different real-world sequences, where different stereo algorithms, baseline distances, distance metrics, and optimisation algorithms are examined. The results show that the proposed polar distance metric yields a higher accuracy for yaw angle estimation of vehicles than the common Euclidean distance metric, especially when using pixel-accurate stereo points.
---
paper_title: Object-oriented Bayesian networks for detection of lane change maneuvers

paper_content:

In this paper we introduce a novel approach towards the recognition of typical driving maneuvers in structured highway scenarios and identify some of the key benefits of traffic scene modeling with object-oriented Bayesian networks (OOBNs). The approach exploits the advantages of an introduced lane-related coordinate system together with individual occupancy grids for all vehicles. This combination allows for an efficient classification of the existing vehiclelane and vehicle-vehicle relations in a traffic scene and thus substantially improves the understanding of complex traffic scenes. We systematically propagate probabilities and variances within our network which results in probabilistic sets of the modeled driving maneuvers. Using this generic approach, we are able to classify a total of 27 driving maneuvers including merging and object following.
---
paper_title: Overtaking Vehicle Detection Using Dynamic and Quasi-Static Background Modeling

paper_content:

Robust and reliable detection of overtaking vehicles is an important component of any on-board driver assistance system. Optical flow, with the abundant motion information present in image sequences, has been studied extensively for vehicle detection. However, using dense optical ?ow for vehicle detection is sensitive to shocks and vibrations of the mobile camera; image outliers caused by illumination changes; and high computational complexity. To improve vehicle detection performance and reduce computational complexity, we propose an efficient and robust methodology for overtaking vehicle detection based on homogeneous sparse optical flow and eigenspace modeling. Specifically, our method models the background into dynamic and quasi-static regions. Instead of using dense optical flow to model the dynamic parts of the background, we employ homogeneous sparse optical flow, which makes detection more robust to camera shocks and vibrations. Moreover, to make detection robust to illumination changes, we employ a blockbased eigenspace approach to represent quasi-static regions in the background. A region-based hysteresis-thresholding approach, augmented by a localized spatial segmentation procedure, attains a good tradeoff between true detections and false positives. The proposed methodology has been evaluated using challenging traf?c scenes illustrating good performance.
---
paper_title: Data fusion for overtaking vehicle detection based on radar and optical flow

paper_content:

Trustworthiness is a key point when dealing with vehicle safety applications. In this paper an approach to a real application is presented, able to fulfill the requirements of such demanding applications. Most of commercial sensors available nowadays are usually designed to detect front vehicles but lack the ability to detect overtaking vehicles. The work presented here combines the information provided by two sensors, a Stop&Go radar and a camera. Fusion is done by using the unprocessed information from the radar, and computer vision based on optical flow. The basic capabilities of the commercial systems are upgraded giving the possibility to improve the front vehicles detection system, by detecting overtaking vehicles with a high positive rate.
---
paper_title: Estimating the Driving State of Oncoming Vehicles From a Moving Platform Using Stereo Vision

paper_content:

A new image-based approach for fast and robust vehicle tracking from a moving platform is presented. Position, orientation, and full motion state, including velocity, acceleration, and yaw rate of a detected vehicle, are estimated from a tracked rigid 3-D point cloud. This point cloud represents a 3-D object model and is computed by analyzing image sequences in both space and time, i.e., by fusion of stereo vision and tracked image features. Starting from an automated initial vehicle hypothesis, tracking is performed by means of an extended Kalman filter. The filter combines the knowledge about the movement of the rigid point cloud's points in the world with the dynamic model of a vehicle. Radar information is used to improve the image-based object detection at far distances. The proposed system is applied to predict the driving path of other traffic participants and currently runs at 25 Hz (640 times 480 images) on our demonstrator vehicle.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: Reliable Detection of Overtaking Vehicles Using Robust Information Fusion

paper_content:

Early detection of overtaking vehicles is an important task for vision-based driver assistance systems. Techniques utilizing image motion are likely to suffer from spurious image structures caused by shadows and illumination changes, let alone the aperture problem. To achieve reliable detection of overtaking vehicles, the authors have developed a robust detection method, which integrates dynamic scene modeling, hypothesis testing, and robust information fusion. A robust fusion algorithm, based on variable bandwidth density fusion and multiscale mean shift, is introduced to obtain reliable motion estimation against various image noise. To further reduce detection error, the authors model the dynamics of road scenes and exploit useful constraints induced by the temporal coherence in vehicle overtaking. The proposed solution is integrated into a monocular vision system onboard for obstacle detection. Test results have shown superior performance achieved by the new method
---
paper_title: Vehicle tracking and motion prediction in complex urban scenarios

paper_content:

The recognition of potentially hazardous situations on road intersections is an indispensable skill of future driver assistance systems. In this context, this study focuses on the task of vehicle tracking in combination with a long-term motion prediction (1–2 s into the future) in a dynamic scenario. A motion-attributed stereo point cloud obtained using computationally efficient feature-based methods represents the scene, relying on images of a stereo camera system mounted on a vehicle. A two-stage mean-shift algorithm is used for detection and tracking of the traffic participants. A hierarchical setup depending on the history of the tracked object is applied for prediction. This includes prediction by optical flow, a standard kinematic prediction, and a particle filter based motion pattern method relying on learned object trajectories. The evaluation shows that the proposed system is able to track the road users in a stable manner and predict their positions at least one order of magnitude more accurately than a standard kinematic prediction method.
---
paper_title: Recognition and Prediction of Situations in Urban Traffic Scenarios

paper_content:

The recognition and prediction of intersection situations and an accompanying threat assessment are an indispensable skill of future driver assistance systems. This study focuses on the recognition of situations involving two vehicles at intersections. For each vehicle, a set of possible future motion trajectories is estimated and rated based on a motion database for a time interval of 2-4 s ahead. Possible situations involving two vehicles are generated by a pairwise combination of these individual motion trajectories. An interaction model based on the mutual visibility of the vehicles and the assumption that a driver will attempt to avoid a collision is used to rate possible situations. The correspondingly favoured situations are classified with a probabilistic framework. The proposed method is evaluated on a real-world differential GPS data set acquired during a test drive of about 10 km, including three road intersections. Our method is typically able to recognise the situation correctly about 1.5-3 s before the last vehicle has passed its minimum distance to the centre of the intersection.
---
paper_title: Long-term vehicle motion prediction

paper_content:

Future driver assistance systems will have to cope with complex traffic situations, especially in the road crossing scenario. To detect potentially hazardous situations as early as possible, it is therefore desirable to know the position and motion of the ego-vehicle and vehicles around it for several seconds in advance. For this purpose, we propose in this study a long-term prediction approach based on a combined trajectory classification and particle filter framework. As a measure for the similarity between trajectories, we introduce the quaternion-based rotationally invariant longest common subsequence (QRLCS) metric. The trajectories are classified by a radial basis function (RBF) classifier with an architecture that is able to process trajectories of arbitrary non-uniform length. The particle filter framework simultaneously tracks and assesses a large number of motion hypotheses (∼102), where the class-specific probabilities estimated by the RBF classifier are used as a-priori probabilities for the hypotheses of the particle filter. The hypotheses are clustered with a mean-shift technique and are assigned a likelihood value. Motion prediction is performed based on the cluster centre with the highest likelihood. While traditional motion prediction based on curve radius and acceleration is inaccurate especially during turning manoeuvres, we show that our approach achieves a reasonable motion prediction even for long prediction intervals of 3 s for these complex motion patterns.
---
paper_title: Probabilistic trajectory prediction with Gaussian mixture models

paper_content:

In the context of driver assistance, an accurate and reliable prediction of the vehicle's trajectory is beneficial. This can be useful either to increase the flexibility of comfort systems or, in the more interesting case, to detect potentially dangerous situations as early as possible. In this contribution, a novel approach for trajectory prediction is proposed which has the capability to predict the vehicle's trajectory several seconds in advance, the so called long-term prediction. To achieve this, previously observed motion patterns are used to infer a joint probability distribution as motion model. Using this distribution, a trajectory can be predicted by calculating the probability for the future motion, conditioned on the current observed history motion pattern. The advantage of the probabilistic modeling is that the result is not only a prediction, but rather a whole distribution over the future trajectories and a specific prediction can be made by the evaluation of the statistical properties, e.g. the mean of this conditioned distribution. Additionally, an evaluation of the variance can be used to examine the reliability of the prediction.
---
paper_title: Learning multi-lane trajectories using vehicle-based vision

paper_content:

Safe operation of a motor vehicle requires awareness of the current traffic situation as well as the ability to predict future maneuvers. In order to provide an intelligent vehicle the ability to make predictions, this work proposes a framework for understanding the driving situation based on vehicle mounted vision sensors. Vehicles are tracked using Kalman filtering based on a vision-based system that detects and tracks using a combination of monocular and stereo-vision. The vehicles' full trajectories are recorded, and a data-driven learning framework has been applied to automatically learn surround behaviors. By learning based on observations, the ADAS system is being trained by experience. Learned trajectories have been compared between dense and free-flowing traffic conditions. Preliminary experimental results using real-world multi-lane highways show the basic promise of this approach. Future research directions are discussed.
---
paper_title: A probabilistic model for estimating driver behaviors and vehicle trajectories in traffic environments

paper_content:

This paper presents a filter that is able to simultaneously estimate the behaviors of traffic participants and anticipate their future trajectories. This is achieved by recognizing the type of situation derived from the local situational context, which subsumes all information relevant for the drivers decision making. By explicitly taking into account the interactions between vehicles, it achieves a comprehensive situational understanding, inevitable for autonomous vehicles and driver assistance systems. This provides the necessary information for safe behavior decision making or motion planning. The filter is modeled as a Dynamic Bayesian Network. The factored state space, modeling the causal dependencies, allows to describe the models in a compact fashion and reduces the computational complexity of the inference process. The filter is evaluated in the context of a highway scenario, showing a good performance even with very noisy measurements. The presented framework is intended to be used in traffic environments but can be easily transferred to other robotic domains.
---
paper_title: Object-oriented Bayesian networks for detection of lane change maneuvers

paper_content:

In this paper we introduce a novel approach towards the recognition of typical driving maneuvers in structured highway scenarios and identify some of the key benefits of traffic scene modeling with object-oriented Bayesian networks (OOBNs). The approach exploits the advantages of an introduced lane-related coordinate system together with individual occupancy grids for all vehicles. This combination allows for an efficient classification of the existing vehiclelane and vehicle-vehicle relations in a traffic scene and thus substantially improves the understanding of complex traffic scenes. We systematically propagate probabilities and variances within our network which results in probabilistic sets of the modeled driving maneuvers. Using this generic approach, we are able to classify a total of 27 driving maneuvers including merging and object following.
---
paper_title: Overtaking Vehicle Detection Using Dynamic and Quasi-Static Background Modeling

paper_content:

Robust and reliable detection of overtaking vehicles is an important component of any on-board driver assistance system. Optical flow, with the abundant motion information present in image sequences, has been studied extensively for vehicle detection. However, using dense optical ?ow for vehicle detection is sensitive to shocks and vibrations of the mobile camera; image outliers caused by illumination changes; and high computational complexity. To improve vehicle detection performance and reduce computational complexity, we propose an efficient and robust methodology for overtaking vehicle detection based on homogeneous sparse optical flow and eigenspace modeling. Specifically, our method models the background into dynamic and quasi-static regions. Instead of using dense optical flow to model the dynamic parts of the background, we employ homogeneous sparse optical flow, which makes detection more robust to camera shocks and vibrations. Moreover, to make detection robust to illumination changes, we employ a blockbased eigenspace approach to represent quasi-static regions in the background. A region-based hysteresis-thresholding approach, augmented by a localized spatial segmentation procedure, attains a good tradeoff between true detections and false positives. The proposed methodology has been evaluated using challenging traf?c scenes illustrating good performance.
---
paper_title: Data fusion for overtaking vehicle detection based on radar and optical flow

paper_content:

Trustworthiness is a key point when dealing with vehicle safety applications. In this paper an approach to a real application is presented, able to fulfill the requirements of such demanding applications. Most of commercial sensors available nowadays are usually designed to detect front vehicles but lack the ability to detect overtaking vehicles. The work presented here combines the information provided by two sensors, a Stop&Go radar and a camera. Fusion is done by using the unprocessed information from the radar, and computer vision based on optical flow. The basic capabilities of the commercial systems are upgraded giving the possibility to improve the front vehicles detection system, by detecting overtaking vehicles with a high positive rate.
---
paper_title: Estimating the Driving State of Oncoming Vehicles From a Moving Platform Using Stereo Vision

paper_content:

A new image-based approach for fast and robust vehicle tracking from a moving platform is presented. Position, orientation, and full motion state, including velocity, acceleration, and yaw rate of a detected vehicle, are estimated from a tracked rigid 3-D point cloud. This point cloud represents a 3-D object model and is computed by analyzing image sequences in both space and time, i.e., by fusion of stereo vision and tracked image features. Starting from an automated initial vehicle hypothesis, tracking is performed by means of an extended Kalman filter. The filter combines the knowledge about the movement of the rigid point cloud's points in the world with the dynamic model of a vehicle. Radar information is used to improve the image-based object detection at far distances. The proposed system is applied to predict the driving path of other traffic participants and currently runs at 25 Hz (640 times 480 images) on our demonstrator vehicle.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: Probabilistic trajectory prediction with Gaussian mixture models

paper_content:

In the context of driver assistance, an accurate and reliable prediction of the vehicle's trajectory is beneficial. This can be useful either to increase the flexibility of comfort systems or, in the more interesting case, to detect potentially dangerous situations as early as possible. In this contribution, a novel approach for trajectory prediction is proposed which has the capability to predict the vehicle's trajectory several seconds in advance, the so called long-term prediction. To achieve this, previously observed motion patterns are used to infer a joint probability distribution as motion model. Using this distribution, a trajectory can be predicted by calculating the probability for the future motion, conditioned on the current observed history motion pattern. The advantage of the probabilistic modeling is that the result is not only a prediction, but rather a whole distribution over the future trajectories and a specific prediction can be made by the evaluation of the statistical properties, e.g. the mean of this conditioned distribution. Additionally, an evaluation of the variance can be used to examine the reliability of the prediction.
---
paper_title: Reliable Detection of Overtaking Vehicles Using Robust Information Fusion

paper_content:

Early detection of overtaking vehicles is an important task for vision-based driver assistance systems. Techniques utilizing image motion are likely to suffer from spurious image structures caused by shadows and illumination changes, let alone the aperture problem. To achieve reliable detection of overtaking vehicles, the authors have developed a robust detection method, which integrates dynamic scene modeling, hypothesis testing, and robust information fusion. A robust fusion algorithm, based on variable bandwidth density fusion and multiscale mean shift, is introduced to obtain reliable motion estimation against various image noise. To further reduce detection error, the authors model the dynamics of road scenes and exploit useful constraints induced by the temporal coherence in vehicle overtaking. The proposed solution is integrated into a monocular vision system onboard for obstacle detection. Test results have shown superior performance achieved by the new method
---
paper_title: Learning multi-lane trajectories using vehicle-based vision

paper_content:

Safe operation of a motor vehicle requires awareness of the current traffic situation as well as the ability to predict future maneuvers. In order to provide an intelligent vehicle the ability to make predictions, this work proposes a framework for understanding the driving situation based on vehicle mounted vision sensors. Vehicles are tracked using Kalman filtering based on a vision-based system that detects and tracks using a combination of monocular and stereo-vision. The vehicles' full trajectories are recorded, and a data-driven learning framework has been applied to automatically learn surround behaviors. By learning based on observations, the ADAS system is being trained by experience. Learned trajectories have been compared between dense and free-flowing traffic conditions. Preliminary experimental results using real-world multi-lane highways show the basic promise of this approach. Future research directions are discussed.
---
paper_title: 3D pose estimation of vehicles using a stereo camera

paper_content:

This study introduces an approach to three-dimensional vehicle pose estimation using a stereo camera system. After computation of stereo and optical flow on the investigated scene, a four-dimensional clustering approach separates the static from the moving objects in the scene. The iterative closest point algorithm (ICP) estimates the vehicle pose using a cuboid as a weak vehicle model. In contrast to classical ICP optimisation a polar distance metric is used which especially takes into account the error distribution of the stereo measurement process. The tracking approach is based on tracking-by-detection such that no temporal filtering is used. The method is evaluated on seven different real-world sequences, where different stereo algorithms, baseline distances, distance metrics, and optimisation algorithms are examined. The results show that the proposed polar distance metric yields a higher accuracy for yaw angle estimation of vehicles than the common Euclidean distance metric, especially when using pixel-accurate stereo points.
---
paper_title: Impact of out-of-sequence measurements on the joint integrated probabilistic data association filter for vehicle safety systems

paper_content:

This paper addresses the problem of joint state and existence estimation in the presence of temporally asynchronous measurements. In multi-sensor fusion, the problem can occur that measurements from different sensors can arrive at the processing unit out of sequence, i.e., the original temporal order of the measurements is lost. For the first time, the influence of these out-of-sequence measurements on state estimation as well as on existence estimation is examined. The existence probabilities are estimated via the Joint Integrated Probabilistic Data Association Filter (JIPDA) [17]. Two different methods to deal with out-of-sequence measurements in JIPDA are described and compared. It is shown that the handling of out-of-sequence measurements has a considerable influence not only on state, but also on existence estimation.
---
paper_title: Accurate and efficient stereo processing by semi-global matching and mutual information

paper_content:

This paper considers the objectives of accurate stereo matching, especially at object boundaries, robustness against recording or illumination changes and efficiency of the calculation. These objectives lead to the proposed semi-global matching method that performs pixelwise matching based on mutual information and the approximation of a global smoothness constraint. Occlusions are detected and disparities determined with sub-pixel accuracy. Additionally, an extension for multi-baseline stereo images is presented. There are two novel contributions. Firstly, a hierarchical calculation of mutual information based matching is shown, which is almost as fast as intensity based matching. Secondly, an approximation of a global cost calculation is proposed that can be performed in a time that is linear to the number of pixels and disparities. The implementation requires just 1 second on typical images.
---
paper_title: Real-time vehicle detection and tracking using stereo vision and multi-view AdaBoost

paper_content:

We propose a multi-layer, real-time vehicle detection and tracking system using stereo vision, multi-view AdaBoost detectors, and optical flow. By adopting a ground plane estimate extracted from stereo information, we generate a sparse set of hypotheses and apply trained AdaBoost classifiers in addition to fast disparity histogramming, for Hypothesis Verification (HV) purposes. Our tracking system employs one Kalman filter per detected vehicle and motion vectors from optical flow, as a means to increase its robustness. An acceptable detection rate with few false positives is obtained at 25 fps with generic hardware.
---
paper_title: Tracking multiple objects in urban traffic environments using dense stereo and optical flow

paper_content:

Stereo vision based sensors provide large amounts of data, a fact which is advantageous when trying to extract semantic information about the imaged scene. However, this data is corrupted by errors, caused especially by the uncertainties in the stereo reconstruction process. Temporal information can be used in order to minimize these errors. This paper presents an advanced object model, a novel association mechanism and the design of a Kalman filter based tracking algorithm, for tracking multiple objects, in complex, urban traffic scenarios.
---
paper_title: Learning based symmetric features selection for vehicle detection

paper_content:

This paper describes a symmetric features selection strategy based on statistical learning method for detecting vehicles with a single moving camera for autonomous driving. Symmetry is a good class of feature for vehicle detection, but the areas with high symmetry and threshold for segmentation is hard to be decided. Usually, the additional supposition is added artificially, and this will decrease the robustness of algorithms. In this paper, we focus on the problem of symmetric features selection using learning method for autonomous driving environment. Global symmetry and local symmetry are defined and used to construct a cascaded structure with a one-class classifier followed by a two-class classifier. Especially for local symmetric features, different symmetric areas in the rear view image of vehicles are searched through Adaboost based learning, and most useful symmetric features are extracted. The threshold for classification is also found through learning. The effective features selection strategy shows that the integration of global symmetry and local symmetry helps to improve the robustness of algorithms. Experimental results indicate the robustness and real-time performance of the algorithm.
---
paper_title: Modeling and Tracking the Driving Environment With a Particle-Based Occupancy Grid

paper_content:

Modeling and tracking the driving environment is a complex problem due to the heterogeneous nature of the real world. In many situations, modeling the obstacles and the driving surfaces can be achieved by the use of geometrical objects, and tracking becomes the problem of estimating the parameters of these objects. In the more complex cases, the scene can be modeled and tracked as an occupancy grid. This paper presents a novel occupancy grid tracking solution based on particles for tracking the dynamic driving environment. The particles will have a dual nature-they will denote hypotheses, as in the particle filtering algorithms, but they will also be the building blocks of our modeled world. The particles have position and speed, and they can migrate in the grid from cell to cell, depending on their motion model and motion parameters, but they will be also created and destroyed using a weighting-resampling mechanism that is specific to particle filtering algorithms. The tracking algorithm will be centered on particles, instead of cells. An obstacle grid derived from processing a stereovision-generated elevation map is used as measurement information, and the measurement model takes into account the uncertainties of the stereo reconstruction. The resulting system is a flexible real-time tracking solution for dynamic unstructured driving environments.
---
paper_title: Moving vehicle detection by optimal segmentation of the Dynamic Stixel World

paper_content:

The reliable detection of moving objects from a moving observer is one of the most challenging and important tasks for driver assistance and safety systems. Modern sensors such as Lidar, Imaging Radar or Stereo Vision deliver range data plus longitudinal motion (Radar) or even full 3D-motion (space-time vision). Based on this data, moving objects have to be separated from the static background to be able to determine their pose and motion state. Usually, heuristics are applied to cluster the data. In order to find the most probable segmentation, we formulate the task as a hypotheses testing problem that allows taking into account various constraints and assumptions simultaneously. We show that the optimal segmentation can be efficiently found by means of dynamic programming, for an arbitrary number of objects in the scene. In this paper we concentrate on the segmentation of space-time data obtained from stereo image sequences. The vision-based depth and motion information is transferred into so called Stixels, a very compact representation of 3D scenes that can also be applied to Lidar or Radar data. It turns out that our optimal segmentation is more robust w.r.t. noisy and erroneous data.
---
paper_title: Visual confirmation of mobile objects tracked by a multi-layer lidar

paper_content:

Integrity of the information provided by a perception system is crucial for advanced driver assistance systems intended for safety applications, like obstacle avoidance systems. A method to ensure integrity is to use different kinds of perception sources. Lidars are key sensors for multiple objects detection and tracking. Stereo vision systems (SVS) can be used to improve the tracking but, in this paper, we use also SVS to confirm the real existence of potential obstacles thanks to 3D dense reconstruction in focused regions of interest. Synchronization issues between the different sensors are addressed using predictive filtering. The proposed approach is evaluated in real conditions thanks to five use cases relevant to urban situations. Results show that this visual confirmation strategy is efficient.
---
paper_title: Event-driven track management method for robust multi-vehicle tracking

paper_content:

In this paper, we present an event-driven track management method to detect reliably and track robustly while minimizing missing and false detections. No state-of-the-art vehicle detection method can detect all the vehicles on the road without error. A multi-vehicle tracking method is essential to minimize the number of missing and false detections. In a multi-vehicle tracking method, there are three types of errors: false negative alarms, false positive alarms, and track identity switches. Our track management method can reduce the number of these errors remarkably while processing in real time for online application. Our track management method has four states: IDLE, PRE-TRACK, CUR-TRACK, and POST-TRACK. Most false positive alarms are removed in the PRE-TRACK state due to their sparseness. A track state transition to other states is determined by a track score. The track score is calculated by obstacle detection, vehicle recognition, detection-by-tracking, and data association. The proposed method is tested and verified with image sequences in real road environments. The experimental results demonstrate that the event-driven track management method minimizes the number of the false positive and false negative alarms remarkably compared with previous methods.
---
paper_title: Data fusion for overtaking vehicle detection based on radar and optical flow

paper_content:

Trustworthiness is a key point when dealing with vehicle safety applications. In this paper an approach to a real application is presented, able to fulfill the requirements of such demanding applications. Most of commercial sensors available nowadays are usually designed to detect front vehicles but lack the ability to detect overtaking vehicles. The work presented here combines the information provided by two sensors, a Stop&Go radar and a camera. Fusion is done by using the unprocessed information from the radar, and computer vision based on optical flow. The basic capabilities of the commercial systems are upgraded giving the possibility to improve the front vehicles detection system, by detecting overtaking vehicles with a high positive rate.
---
paper_title: Histograms of oriented gradients for human detection

paper_content:

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.
---
paper_title: Sensorfusion Using Spatio-Temporal Aligned Video and Lidar for Improved Vehicle Detection

paper_content:

This contribution introduces a novel approach to cross-calibrate automotive vision and ranging sensors. The resulting sensor alignment allows the incorporation of multiple sensor data into a detection and tracking framework. Exemplarily, we show how a realtime vehicle detection system, intended for emergency breaking or ACC applications, benefits from the low level fusion of multibeam lidar and vision sensor measurements in discrimination performance and computational complexity.
---
paper_title: Scalable Active Learning for Multiclass Image Classification

paper_content:

Machine learning techniques for computer vision applications like object recognition, scene classification, etc., require a large number of training samples for satisfactory performance. Especially when classification is to be performed over many categories, providing enough training samples for each category is infeasible. This paper describes new ideas in multiclass active learning to deal with the training bottleneck, making it easier to train large multiclass image classification systems. First, we propose a new interaction modality for training which requires only yes-no type binary feedback instead of a precise category label. The modality is especially powerful in the presence of hundreds of categories. For the proposed modality, we develop a Value-of-Information (VOI) algorithm that chooses informative queries while also considering user annotation cost. Second, we propose an active selection measure that works with many categories and is extremely fast to compute. This measure is employed to perform a fast seed search before computing VOI, resulting in an algorithm that scales linearly with dataset size. Third, we use locality sensitive hashing to provide a very fast approximation to active learning, which gives sublinear time scaling, allowing application to very large datasets. The approximation provides up to two orders of magnitude speedups with little loss in accuracy. Thorough empirical evaluation of classification accuracy, noise sensitivity, imbalanced data, and computational performance on a diverse set of image datasets demonstrates the strengths of the proposed algorithms.
---
paper_title: Vision-Based Vehicle Detection System With Consideration of the Detecting Location

paper_content:

In this paper, we propose a vision-based vehicle detection system. We use a method composed of a hypothesis generation (HG) step and a hypothesis verification (HV) step, following the general approach to vision-based vehicle detection systems. In the HG step, the system extracts hypotheses using shadow regions that appear under vehicles. In the HV step, the system classifies feature vectors extracted from hypotheses to determine whether those hypotheses are vehicles. Along with the histogram of oriented gradients (HOG), we propose and implement a new type of feature vector, i.e., HOG symmetry vectors, in this paper. We also propose a new classification method that uses data importance in the HV step. The data importance value is based on the locations of hypotheses to prioritize hypotheses that have greater risks of accident. Experimental results show the strong performance of our proposed system.
---
paper_title: LaRASideCam: a Fast and Robust Vision-Based Blindspot Detection System

paper_content:

While shifting lane on the road, the presence of a car in the blindspot can cause many accidents, since the driver does not always turn his head. Therefore, a blindspot car detection is likely to become an essential part of modern vehicles. We developed a program that detects cars in the particular configuration of blindspot using video data taken from the left or right mirror of a car, using on the one hand edge detection and support vector machine (SVM) learning and on the other hand template matching. This makes this program simple, fast and adaptative thanks to SVM learning. The program only uses basical functions of the Ecole des Mines' Camellia open-source image processing library [1], which is close to Intel's IPL library. Thus the program is easy to adapt to another API; it has already been adapted to an embedded system currently in development at NXP Semiconductors (formerly Philips Semiconductors). The source code was tested using the valgrind code checking tool [3] and was validated on real-world video sequences.
---
paper_title: Rapid object detection using a boosted cascade of simple features

paper_content:

This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the "integral image" which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers. The third contribution is a method for combining increasingly more complex classifiers in a "cascade" which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. The cascade can be viewed as an object specific focus-of-attention mechanism which unlike previous approaches provides statistical guarantees that discarded regions are unlikely to contain the object of interest. In the domain of face detection the system yields detection rates comparable to the best previous systems. Used in real-time applications, the detector runs at 15 frames per second without resorting to image differencing or skin color detection.
---
paper_title: On-Road Multivehicle Tracking Using Deformable Object Model and Particle Filter With Improved Likelihood Estimation

paper_content:

This paper proposes a novel method for multivehicle detection and tracking using a vehicle-mounted monocular camera. In the proposed method, the features of vehicles are learned as a deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOGs). The detection algorithm combines both global and local features of the vehicle as a deformable object model. Detected vehicles are tracked through a particle filter, which estimates the particles' likelihood by using a detection scores map and template compatibility for both root and parts of the vehicle while considering the deformation cost caused by the movement of vehicle parts. Tracking likelihoods are iteratively used as a priori probability to generate vehicle hypothesis regions and update the detection threshold to reduce false negatives of the algorithm presented before. Extensive experiments in urban scenarios showed that the proposed method can achieve an average vehicle detection rate of 97% and an average vehicle-tracking rate of 86% with a false positive rate of less than 0.26%.
---
paper_title: A note on Platt’s probabilistic outputs for support vector machines

paper_content:

Platt's probabilistic outputs for Support Vector Machines (Platt, J. in Smola, A., et al. (eds.) Advances in large margin classifiers. Cambridge, 2000) has been popular for applications that require posterior class probabilities. In this note, we propose an improved algorithm that theoretically converges and avoids numerical difficulties. A simple and ready-to-use pseudo code is included.
---
paper_title: Combining monocular and stereo-vision for real-time vehicle ranging and tracking on multilane highways

paper_content:

In this paper, we introduce a novel stereo-monocular fusion approach to on-road localization and tracking of vehicles. Utilizing a calibrated stereo-vision rig, the proposed approach combines monocular detection with stereo-vision for on-road vehicle localization and tracking for driver assistance. The system initially acquires synchronized monocular frames and calculates depth maps from the stereo rig. The system then detects vehicles in the image plane using an active learning-based monocular vision approach. Using the image coordinates of detected vehicles, the system then localizes the vehicles in real-world coordinates using the calculated depth map. The vehicles are tracked both in the image plane, and in real-world coordinates, fusing information from both the monocular and stereo modalities. Vehicles' states are estimated and tracked using Kalman filtering. Quantitative analysis of tracks is provided. The full system takes 46ms to process a single frame.
---
paper_title: Rear Vehicle Detection and Tracking for Lane Change Assist

paper_content:

A monocular vision based rear vehicle detection and tracking system is presented for Lane Change Assist (LCA), which does not need road boundary and lane information. Our algorithm extracts regions of interest (ROI) using the shadow underneath a vehicle, and accurately localizes vehicle regions in ROI by vehicle features such as symmetry, edge and shadow underneath vehicles. The algorithm realizes vehicle verification by combining knowledge-based and learning-based methods. During vehicle tracking, templates are dynamically created on-line, tracking window is adaptively adjusted with motion estimation, and confidence is determined for tracked vehicle. The algorithm was tested under various traffic scenes at different daytime, the result illustrated good performance.
---
paper_title: Efficient Resource Allocation for Attentive Automotive Vision Systems

paper_content:

We describe a novel architecture for automotive vision organized on five levels of abstraction, i.e., sensor, data, semantic, reasoning, and resource allocation levels, respectively. Although we implement and evaluate processes to detect and classify other participants within the immediate environment of a moving vehicle, our main emphasis is on the allocation of computational resource and attentive processing by the sensor suite. To that end, an efficient multiobjective resource allocation method is formalized and implemented. This includes a decision-making process dependent upon the environment, the current goal, the available sensors and computational resource, and the time available to make a decision. We evaluate our approach on road traffic test sequences acquired by a test vehicle provided by Audi. This vehicle includes lidar, video, radar, and sonar sensors, in addition to conventional global positioning system (GPS) navigation, but our evaluation is confined to lidar and video data alone.
---
paper_title: Real-time vehicle detection using parts at intersections

paper_content:

In this study, we propose a novel, lightweight approach to real-time detection of vehicles using parts at intersections. Intersections feature oncoming, preceding, and cross traffic, which presents challenges for vision-based vehicle detection. Ubiquitous partial occlusions further complicate the vehicle detection task, and occur when vehicles enter and leave the camera's field of view. To confront these issues, we independently detect vehicle parts using strong classifiers trained with active learning. We match part responses using a learned matching classification. The learning process for part configurations leverages user input regarding full vehicle configurations. Part configurations are evaluated using Support Vector Machine classification. We present a comparison of detection results using geometric image features and appearance-based features. The full vehicle detection by parts has been evaluated on real-world data, runs in real time, and shows promise for future work in urban driver assistance.
---
paper_title: Symmetry-based monocular vehicle detection system

paper_content:

In this paper, we describe the development of a symmetry-based vehicle detection system. The system uses a single forward looking camera to capture the road scene. Vehicles are detected based on their edges and symmetrical characteristics. A method to extract the symmetric regions in the image using a multi-sized window and clustering technique is introduced. We hypothesize the vehicle’s locations in the image from the detected symmetric regions and the regions are then further processed to enhance their symmetrical edges. A bounding box of a vehicle is detected from the projection maps of the enhanced vertical and horizontal edges. The hypothesized vehicles are then verified using a two-class classifier, which consists of an edge oriented histogram (EOH) feature extractor and a support vector machine (SVM). Once a vehicle is verified, a tracking process based on a Kalman filter and a reliability point system is used to track the movement of the vehicle in consecutive video frames. The system was successfully implemented and tested on a standard PC. Experimental results on live video feed and pre-recorded video sequences for various road scenes showed that the system is able to detect multiple vehicles in real time.
---
paper_title: Modeling and Tracking the Driving Environment With a Particle-Based Occupancy Grid

paper_content:

Modeling and tracking the driving environment is a complex problem due to the heterogeneous nature of the real world. In many situations, modeling the obstacles and the driving surfaces can be achieved by the use of geometrical objects, and tracking becomes the problem of estimating the parameters of these objects. In the more complex cases, the scene can be modeled and tracked as an occupancy grid. This paper presents a novel occupancy grid tracking solution based on particles for tracking the dynamic driving environment. The particles will have a dual nature-they will denote hypotheses, as in the particle filtering algorithms, but they will also be the building blocks of our modeled world. The particles have position and speed, and they can migrate in the grid from cell to cell, depending on their motion model and motion parameters, but they will be also created and destroyed using a weighting-resampling mechanism that is specific to particle filtering algorithms. The tracking algorithm will be centered on particles, instead of cells. An obstacle grid derived from processing a stereovision-generated elevation map is used as measurement information, and the measurement model takes into account the uncertainties of the stereo reconstruction. The resulting system is a flexible real-time tracking solution for dynamic unstructured driving environments.
---
paper_title: Moving vehicle detection by optimal segmentation of the Dynamic Stixel World

paper_content:

The reliable detection of moving objects from a moving observer is one of the most challenging and important tasks for driver assistance and safety systems. Modern sensors such as Lidar, Imaging Radar or Stereo Vision deliver range data plus longitudinal motion (Radar) or even full 3D-motion (space-time vision). Based on this data, moving objects have to be separated from the static background to be able to determine their pose and motion state. Usually, heuristics are applied to cluster the data. In order to find the most probable segmentation, we formulate the task as a hypotheses testing problem that allows taking into account various constraints and assumptions simultaneously. We show that the optimal segmentation can be efficiently found by means of dynamic programming, for an arbitrary number of objects in the scene. In this paper we concentrate on the segmentation of space-time data obtained from stereo image sequences. The vision-based depth and motion information is transferred into so called Stixels, a very compact representation of 3D scenes that can also be applied to Lidar or Radar data. It turns out that our optimal segmentation is more robust w.r.t. noisy and erroneous data.
---
paper_title: Event-driven track management method for robust multi-vehicle tracking

paper_content:

In this paper, we present an event-driven track management method to detect reliably and track robustly while minimizing missing and false detections. No state-of-the-art vehicle detection method can detect all the vehicles on the road without error. A multi-vehicle tracking method is essential to minimize the number of missing and false detections. In a multi-vehicle tracking method, there are three types of errors: false negative alarms, false positive alarms, and track identity switches. Our track management method can reduce the number of these errors remarkably while processing in real time for online application. Our track management method has four states: IDLE, PRE-TRACK, CUR-TRACK, and POST-TRACK. Most false positive alarms are removed in the PRE-TRACK state due to their sparseness. A track state transition to other states is determined by a track score. The track score is calculated by obstacle detection, vehicle recognition, detection-by-tracking, and data association. The proposed method is tested and verified with image sequences in real road environments. The experimental results demonstrate that the event-driven track management method minimizes the number of the false positive and false negative alarms remarkably compared with previous methods.
---
paper_title: Estimating the Driving State of Oncoming Vehicles From a Moving Platform Using Stereo Vision

paper_content:

A new image-based approach for fast and robust vehicle tracking from a moving platform is presented. Position, orientation, and full motion state, including velocity, acceleration, and yaw rate of a detected vehicle, are estimated from a tracked rigid 3-D point cloud. This point cloud represents a 3-D object model and is computed by analyzing image sequences in both space and time, i.e., by fusion of stereo vision and tracked image features. Starting from an automated initial vehicle hypothesis, tracking is performed by means of an extended Kalman filter. The filter combines the knowledge about the movement of the rigid point cloud's points in the world with the dynamic model of a vehicle. Radar information is used to improve the image-based object detection at far distances. The proposed system is applied to predict the driving path of other traffic participants and currently runs at 25 Hz (640 times 480 images) on our demonstrator vehicle.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: On-Road Multivehicle Tracking Using Deformable Object Model and Particle Filter With Improved Likelihood Estimation

paper_content:

This paper proposes a novel method for multivehicle detection and tracking using a vehicle-mounted monocular camera. In the proposed method, the features of vehicles are learned as a deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOGs). The detection algorithm combines both global and local features of the vehicle as a deformable object model. Detected vehicles are tracked through a particle filter, which estimates the particles' likelihood by using a detection scores map and template compatibility for both root and parts of the vehicle while considering the deformation cost caused by the movement of vehicle parts. Tracking likelihoods are iteratively used as a priori probability to generate vehicle hypothesis regions and update the detection threshold to reduce false negatives of the algorithm presented before. Extensive experiments in urban scenarios showed that the proposed method can achieve an average vehicle detection rate of 97% and an average vehicle-tracking rate of 86% with a false positive rate of less than 0.26%.
---
paper_title: An evolutionary optimized vehicle tracker in collaboration with a detection system

paper_content:

In this work a learning algorithm for visual object tracking is presented. As object representation a fast computable set of Haar-like features is used and a weighted correlation is applied for the matching process. The object tracker utilizes the same set of features that is already calculated for object detection and thus it is possible to reuse features for detection and tracking. The feature's weight values are optimized for the tracking purpose by means of evolutionary strategies. Different tests of the object tracker on real-world sequences are presented using vehicles as example objects. Additionally, an object detection system and the integration of the object tracker into that system is described. Besides the system is based on a cascade of boosted classifiers, Haar and Triangle features, an adaptive sliding window and finally a Kalman filter.
---
paper_title: Rear Vehicle Detection and Tracking for Lane Change Assist

paper_content:

A monocular vision based rear vehicle detection and tracking system is presented for Lane Change Assist (LCA), which does not need road boundary and lane information. Our algorithm extracts regions of interest (ROI) using the shadow underneath a vehicle, and accurately localizes vehicle regions in ROI by vehicle features such as symmetry, edge and shadow underneath vehicles. The algorithm realizes vehicle verification by combining knowledge-based and learning-based methods. During vehicle tracking, templates are dynamically created on-line, tracking window is adaptively adjusted with motion estimation, and confidence is determined for tracked vehicle. The algorithm was tested under various traffic scenes at different daytime, the result illustrated good performance.
---
paper_title: A probabilistic model for estimating driver behaviors and vehicle trajectories in traffic environments

paper_content:

This paper presents a filter that is able to simultaneously estimate the behaviors of traffic participants and anticipate their future trajectories. This is achieved by recognizing the type of situation derived from the local situational context, which subsumes all information relevant for the drivers decision making. By explicitly taking into account the interactions between vehicles, it achieves a comprehensive situational understanding, inevitable for autonomous vehicles and driver assistance systems. This provides the necessary information for safe behavior decision making or motion planning. The filter is modeled as a Dynamic Bayesian Network. The factored state space, modeling the causal dependencies, allows to describe the models in a compact fashion and reduces the computational complexity of the inference process. The filter is evaluated in the context of a highway scenario, showing a good performance even with very noisy measurements. The presented framework is intended to be used in traffic environments but can be easily transferred to other robotic domains.
---
paper_title: Object Detection with Discriminatively Trained Part-Based Models

paper_content:

We describe an object detection system based on mixtures of multiscale deformable part models. Our system is able to represent highly variable object classes and achieves state-of-the-art results in the PASCAL object detection challenges. While deformable part models have become quite popular, their value had not been demonstrated on difficult benchmarks such as the PASCAL data sets. Our system relies on new methods for discriminative training with partially labeled data. We combine a margin-sensitive approach for data-mining hard negative examples with a formalism we call latent SVM. A latent SVM is a reformulation of MI--SVM in terms of latent variables. A latent SVM is semiconvex, and the training problem becomes convex once latent information is specified for the positive examples. This leads to an iterative training algorithm that alternates between fixing latent values for positive examples and optimizing the latent SVM objective function.
---
paper_title: Mixtures of von Mises Distributions for People Trajectory Shape Analysis

paper_content:

People trajectory analysis is a recurrent task in many pattern recognition applications, such as surveillance, behavior analysis, video annotation, and many others. In this paper, we propose a new framework for analyzing trajectory shape, invariant to spatial shifts of the people motion in the scene. In order to cope with the noise and the uncertainty of the trajectory samples, we propose to describe the trajectories as a sequence of angles modeled by distributions of circular statistics, i.e., a mixture of von Mises (MovM) distributions. To deal with MovM, we define a new specific expectation-maximization (EM) algorithm for estimating the parameters and derive a closed form of the Bhattacharyya distance between single von Mises pdfs. Trajectories are then modeled with a sequence of symbols, corresponding to the most suitable distribution in the mixture, and compared each other after a global alignment procedure to cope with trajectories of different lengths. The trajectories in the training set are clustered according to their shape similarity in an off-line phase, and testing trajectories are then classified with a specific on-line EM, based on sufficient statistics. The approach is particularly suitable for classifying people trajectories in video surveillance, searching for abnormal (i.e., infrequent) paths. Tests on synthetic and real data are provided with also a complete comparison with other circular statistical and alignment methods.
---
paper_title: Object-oriented Bayesian networks for detection of lane change maneuvers

paper_content:

In this paper we introduce a novel approach towards the recognition of typical driving maneuvers in structured highway scenarios and identify some of the key benefits of traffic scene modeling with object-oriented Bayesian networks (OOBNs). The approach exploits the advantages of an introduced lane-related coordinate system together with individual occupancy grids for all vehicles. This combination allows for an efficient classification of the existing vehiclelane and vehicle-vehicle relations in a traffic scene and thus substantially improves the understanding of complex traffic scenes. We systematically propagate probabilities and variances within our network which results in probabilistic sets of the modeled driving maneuvers. Using this generic approach, we are able to classify a total of 27 driving maneuvers including merging and object following.
---
paper_title: Data fusion for overtaking vehicle detection based on radar and optical flow

paper_content:

Trustworthiness is a key point when dealing with vehicle safety applications. In this paper an approach to a real application is presented, able to fulfill the requirements of such demanding applications. Most of commercial sensors available nowadays are usually designed to detect front vehicles but lack the ability to detect overtaking vehicles. The work presented here combines the information provided by two sensors, a Stop&Go radar and a camera. Fusion is done by using the unprocessed information from the radar, and computer vision based on optical flow. The basic capabilities of the commercial systems are upgraded giving the possibility to improve the front vehicles detection system, by detecting overtaking vehicles with a high positive rate.
---
paper_title: Estimating the Driving State of Oncoming Vehicles From a Moving Platform Using Stereo Vision

paper_content:

A new image-based approach for fast and robust vehicle tracking from a moving platform is presented. Position, orientation, and full motion state, including velocity, acceleration, and yaw rate of a detected vehicle, are estimated from a tracked rigid 3-D point cloud. This point cloud represents a 3-D object model and is computed by analyzing image sequences in both space and time, i.e., by fusion of stereo vision and tracked image features. Starting from an automated initial vehicle hypothesis, tracking is performed by means of an extended Kalman filter. The filter combines the knowledge about the movement of the rigid point cloud's points in the world with the dynamic model of a vehicle. Radar information is used to improve the image-based object detection at far distances. The proposed system is applied to predict the driving path of other traffic participants and currently runs at 25 Hz (640 times 480 images) on our demonstrator vehicle.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: On-road prediction of driver's intent with multimodal sensory cues

paper_content:

By predicting a driver's maneuvers before they occur, a driver-assistance system can prepare for or avoid dangerous situations. This article describes a real-time, on-road lane-change-intent detector that can enhance driver safety.
---
paper_title: Learning multi-lane trajectories using vehicle-based vision

paper_content:

Safe operation of a motor vehicle requires awareness of the current traffic situation as well as the ability to predict future maneuvers. In order to provide an intelligent vehicle the ability to make predictions, this work proposes a framework for understanding the driving situation based on vehicle mounted vision sensors. Vehicles are tracked using Kalman filtering based on a vision-based system that detects and tracks using a combination of monocular and stereo-vision. The vehicles' full trajectories are recorded, and a data-driven learning framework has been applied to automatically learn surround behaviors. By learning based on observations, the ADAS system is being trained by experience. Learned trajectories have been compared between dense and free-flowing traffic conditions. Preliminary experimental results using real-world multi-lane highways show the basic promise of this approach. Future research directions are discussed.
---
paper_title: Vehicle tracking and motion prediction in complex urban scenarios

paper_content:

The recognition of potentially hazardous situations on road intersections is an indispensable skill of future driver assistance systems. In this context, this study focuses on the task of vehicle tracking in combination with a long-term motion prediction (1–2 s into the future) in a dynamic scenario. A motion-attributed stereo point cloud obtained using computationally efficient feature-based methods represents the scene, relying on images of a stereo camera system mounted on a vehicle. A two-stage mean-shift algorithm is used for detection and tracking of the traffic participants. A hierarchical setup depending on the history of the tracked object is applied for prediction. This includes prediction by optical flow, a standard kinematic prediction, and a particle filter based motion pattern method relying on learned object trajectories. The evaluation shows that the proposed system is able to track the road users in a stable manner and predict their positions at least one order of magnitude more accurately than a standard kinematic prediction method.
---
paper_title: Disparity space image's features analysis for error prediction of a stereo obstacle detector for heavy duty vehicles

paper_content:

Vision-based perception has been explored as low-cost, flexible technology for industrial applications and ADAS. Its inherent flexibility presents a challenge quantifying performance and often even quantifying increases or decreases in system performance as conditions change. Experience enables designers to employ various ”rules of thumb” while commercially viable products require quantitative performance. This paper explores the correlation between features and characteristics of the Disparity Space Image (DSI) and resulting performance for an object detection application. The specific application is an object detection system suitable for highly chaotic environments often found in earthmoving industry. Features and characteristics with strong correlations can be used to improve system design and predict system performances at run-time. High-quality stereo images are used to characterize baseline system performance. These images are then artificially degraded to simulate fog, darkness, and blurring and subsequent system performance compared to baseline results.
---
paper_title: Modeling and Tracking the Driving Environment With a Particle-Based Occupancy Grid

paper_content:

Modeling and tracking the driving environment is a complex problem due to the heterogeneous nature of the real world. In many situations, modeling the obstacles and the driving surfaces can be achieved by the use of geometrical objects, and tracking becomes the problem of estimating the parameters of these objects. In the more complex cases, the scene can be modeled and tracked as an occupancy grid. This paper presents a novel occupancy grid tracking solution based on particles for tracking the dynamic driving environment. The particles will have a dual nature-they will denote hypotheses, as in the particle filtering algorithms, but they will also be the building blocks of our modeled world. The particles have position and speed, and they can migrate in the grid from cell to cell, depending on their motion model and motion parameters, but they will be also created and destroyed using a weighting-resampling mechanism that is specific to particle filtering algorithms. The tracking algorithm will be centered on particles, instead of cells. An obstacle grid derived from processing a stereovision-generated elevation map is used as measurement information, and the measurement model takes into account the uncertainties of the stereo reconstruction. The resulting system is a flexible real-time tracking solution for dynamic unstructured driving environments.
---
paper_title: On-road vehicle detection: a review

paper_content:

Developing on-board automotive driver assistance systems aiming to alert drivers about driving environments, and possible collision with other vehicles has attracted a lot of attention lately. In these systems, robust and reliable vehicle detection is a critical step. This paper presents a review of recent vision-based on-road vehicle detection systems. Our focus is on systems where the camera is mounted on the vehicle rather than being fixed such as in traffic/driveway monitoring systems. First, we discuss the problem of on-road vehicle detection using optical sensors followed by a brief review of intelligent vehicle research worldwide. Then, we discuss active and passive sensors to set the stage for vision-based vehicle detection. Methods aiming to quickly hypothesize the location of vehicles in an image as well as to verify the hypothesized locations are reviewed next. Integrating detection with tracking is also reviewed to illustrate the benefits of exploiting temporal continuity for vehicle detection. Finally, we present a critical overview of the methods discussed, we assess their potential for future deployment, and we present directions for future research.
---
paper_title: Event-driven track management method for robust multi-vehicle tracking

paper_content:

In this paper, we present an event-driven track management method to detect reliably and track robustly while minimizing missing and false detections. No state-of-the-art vehicle detection method can detect all the vehicles on the road without error. A multi-vehicle tracking method is essential to minimize the number of missing and false detections. In a multi-vehicle tracking method, there are three types of errors: false negative alarms, false positive alarms, and track identity switches. Our track management method can reduce the number of these errors remarkably while processing in real time for online application. Our track management method has four states: IDLE, PRE-TRACK, CUR-TRACK, and POST-TRACK. Most false positive alarms are removed in the PRE-TRACK state due to their sparseness. A track state transition to other states is determined by a track score. The track score is calculated by obstacle detection, vehicle recognition, detection-by-tracking, and data association. The proposed method is tested and verified with image sequences in real road environments. The experimental results demonstrate that the event-driven track management method minimizes the number of the false positive and false negative alarms remarkably compared with previous methods.
---
paper_title: Data fusion for overtaking vehicle detection based on radar and optical flow

paper_content:

Trustworthiness is a key point when dealing with vehicle safety applications. In this paper an approach to a real application is presented, able to fulfill the requirements of such demanding applications. Most of commercial sensors available nowadays are usually designed to detect front vehicles but lack the ability to detect overtaking vehicles. The work presented here combines the information provided by two sensors, a Stop&Go radar and a camera. Fusion is done by using the unprocessed information from the radar, and computer vision based on optical flow. The basic capabilities of the commercial systems are upgraded giving the possibility to improve the front vehicles detection system, by detecting overtaking vehicles with a high positive rate.
---
paper_title: Tracking oncoming and turning vehicles at intersections

paper_content:

This article addresses the reliable tracking of oncoming traffic at urban intersections from a moving platform with a stereo vision system. Both motion and depth information is combined to estimate the pose and motion parameters of an oncoming vehicle, including the yaw rate, by means of Kalman filtering. Vehicle tracking at intersections is particularly challenging since vehicles can turn quickly. A single filter approach cannot cover the dynamic range of a vehicle sufficiently. We propose a real-time multi-filter approach for vehicle tracking at intersections. A gauge consistency criteria as well as a robust outlier detection method allow for dealing with sudden accelerations and self-occlusions during turn maneuvers. The system is evaluated both on synthetic and real-world data.
---
paper_title: Critical Motion Detection of Nearby Moving Vehicles in a Vision-Based Driver-Assistance System

paper_content:

Driving always involves risk. Various means have been proposed to reduce the risk. Critical motion detection of nearby moving vehicles is one of the important means of preventing accidents. In this paper, a computational model, which is referred to as the dynamic visual model (DVM), is proposed to detect critical motions of nearby vehicles while driving on a highway. The DVM is motivated by the human visual system and consists of three analyzers: 1) sensory analyzers, 2) perceptual analyzers, and 3) conceptual analyzers. In addition, a memory, which is called the episodic memory, is incorporated, through which a number of features of the system, including hierarchical processing, configurability, adaptive response, and selective attention, are realized. A series of experimental results with both single and multiple critical motions are demonstrated and show the feasibility of the proposed system.
---
paper_title: Active learning for on-road vehicle detection: a comparative study

paper_content:

In recent years, active learning has emerged as a powerful tool in building robust systems for object detection using computer vision. Indeed, active learning approaches to on-road vehicle detection have achieved impressive results. While active learning approaches for object detection have been explored and presented in the literature, few studies have been performed to comparatively assess costs and merits. In this study, we provide a cost-sensitive analysis of three popular active learning methods for on-road vehicle detection. The generality of active learning findings is demonstrated via learning experiments performed with detectors based on histogram of oriented gradient features and SVM classification (HOG---SVM), and Haar-like features and Adaboost classification (Haar---Adaboost). Experimental evaluation has been performed on static images and real-world on-road vehicle datasets. Learning approaches are assessed in terms of the time spent annotating, data required, recall, and precision.
---
paper_title: Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol

paper_content:

Common benchmark data sets, standardized performance metrics, and baseline algorithms have demonstrated considerable impact on research and development in a variety of application domains. These resources provide both consumers and developers of technology with a common framework to objectively compare the performance of different algorithms and algorithmic improvements. In this paper, we present such a framework for evaluating object detection and tracking in video: specifically for face, text, and vehicle objects. This framework includes the source video data, ground-truth annotations (along with guidelines for annotation), performance metrics, evaluation protocols, and tools including scoring software and baseline algorithms. For each detection and tracking task and supported domain, we developed a 50-clip training set and a 50-clip test set. Each data clip is approximately 2.5 minutes long and has been completely spatially/temporally annotated at the I-frame level. Each task/domain, therefore, has an associated annotated corpus of approximately 450,000 frames. The scope of such annotation is unprecedented and was designed to begin to support the necessary quantities of data for robust machine learning approaches, as well as a statistically significant comparison of the performance of algorithms. The goal of this work was to systematically address the challenges of object detection and tracking through a common evaluation framework that permits a meaningful objective comparison of techniques, provides the research community with sufficient data for the exploration of automatic modeling techniques, encourages the incorporation of objective evaluation into the development process, and contributes useful lasting resources of a scale and magnitude that will prove to be extremely useful to the computer vision research community for years to come.
---
paper_title: Learning multi-lane trajectories using vehicle-based vision

paper_content:

Safe operation of a motor vehicle requires awareness of the current traffic situation as well as the ability to predict future maneuvers. In order to provide an intelligent vehicle the ability to make predictions, this work proposes a framework for understanding the driving situation based on vehicle mounted vision sensors. Vehicles are tracked using Kalman filtering based on a vision-based system that detects and tracks using a combination of monocular and stereo-vision. The vehicles' full trajectories are recorded, and a data-driven learning framework has been applied to automatically learn surround behaviors. By learning based on observations, the ADAS system is being trained by experience. Learned trajectories have been compared between dense and free-flowing traffic conditions. Preliminary experimental results using real-world multi-lane highways show the basic promise of this approach. Future research directions are discussed.
---

Title: Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis

Section 1: INTRODUCTION
Description 1: This section introduces the problem of on-road vehicle fatalities and the efforts towards developing intelligent driver assistance systems and autonomous vehicles with a focus on vision-based vehicle detection, tracking, and behavior analysis.

Section 2: ON-ROAD ENVIRONMENTAL PERCEPTION
Description 2: This section discusses general sensor-based vehicle detection to place vision-based vehicle detection within the overall context of on-road environmental perception, including a comparison of radar, lidar, and vision.

Section 3: VISION-BASED VEHICLE DETECTION
Description 3: This section reviews various approaches and techniques for vision-based vehicle detection, including monocular and stereo vision methods, appearance-based and motion-based methods, and specific nighttime vision algorithms.

Section 4: ON-ROAD VEHICLE TRACKING
Description 4: This section details the methods and techniques for tracking vehicles on the road using vision, covering monocular and stereo-vision tracking, filtering methods, and data fusion with other sensors.

Section 5: ON-ROAD BEHAVIOR ANALYSIS
Description 5: This section explores the higher-level interpretation of vehicle behavior using vision, including detection of specific maneuvers, context analysis, trajectory analysis, and behavior classification.

Section 6: DISCUSSION AND FUTURE DIRECTIONS
Description 6: This section provides a discussion, critique, and perspective on the current state of vision-based vehicle detection, tracking, and behavior analysis, and insights into future research directions.

Section 7: CONCLUDING REMARKS
Description 7: This section summarizes the survey's findings and discusses the maturity of vision-based vehicle detection systems, highlighting the ongoing research in understanding and interpreting the on-road environment.