Active Recognition through Next View Planning: A Survey
---
paper_title: Three-dimensional object recognition

paper_content:

A general-purpose computer vision system must be capable of recognizing three-dimensional (3-D) objects. This paper proposes a precise definition of the 3-D object recognition problem, discusses basic concepts associated with this problem, and reviews the relevant literature. Because range images (or depth maps) are often used as sensor input instead of intensity images, techniques for obtaining, processing, and characterizing range data are also surveyed.
---
paper_title: Model-based recognition in robot vision

paper_content:

This paper presents a comparative study and survey of model-based object-recognition algorithms for robot vision. The goal of these algorithms is to recognize the identity, position, and orientation of randomly oriented industrial parts. In one form this is commonly referred to as the "bin-picking" problem, in which the parts to be recognized are presented in a jumbled bin. The paper is organized according to 2-D, 2½-D, and 3-D object representations, which are used as the basis for the recognition algorithms. Three central issues common to each category, namely, feature extraction, modeling, and matching, are examined in detail. An evaluation and comparison of existing industrial part-recognition systems and algorithms is given, providing insights for progress toward future robot vision systems.
---
paper_title: Shape from symmetry: detecting and exploiting symmetry in affine images

paper_content:

We investigate the constraints placed on the image projection of a planar object having local reflectional symmetry. Under the affine approximation to projection, we demonstrate an efficient (low-complexity) algorithm for detecting and verifying symmetries despite the distorting effects of image skewing. The symmetries are utilized for three distinct tasks: first, determining image back-projection up to a similarity transformation ambiguity; second, determining the object plane orientation (slant and tilt); and third, as a test for non-coplanarity amongst a collection of objects. These results are illustrated throughout with examples from images of real scenes.
---
paper_title: Planning multiple observations for object recognition

paper_content:

Most computer vision systems perform object recognition on the basis of the features extracted from a single image of the object. The problem with this approach is that it implicitly assumes that the available features are sufficient to determine the identity and pose of the object uniquely. If this assumption is not met, then the feature set is insufficient, and ambiguity results. Consequently, much research in computer vision has gone toward finding sets of features that are sufficient for specific tasks, with the result that each system has its own associated set of features. A single, general feature set would be desirable. However, research in automatic generation of object recognition programs has demonstrated that predetermined, fixed feature sets are often incapable of providing enough information to unambiguously determine either object identity or pose. One approach to overcoming the inadequacy of any feature set is to utilize multiple sensor observations obtained from different viewpoints, and combine them with knowledge of the 3-D structure of the object to perform unambiguous object recognition. This article presents initial results toward performing object recognition by using multiple observations to resolve ambiguities. Starting from the premise that sensor motions should be planned in advance, the difficulties involved in planning with ambiguous information are discussed. A representation for planning that combines geometric information with viewpoint uncertainty is presented. A sensor planner utilizing the representation was implemented, and the results of pose-determination experiments performed with the planner are discussed.
---
paper_title: 3D Object Recognition using Invariance

paper_content:

Abstract The systems and concepts described in this paper document the evolution of the geometric invariance approach to object recognition over the last five years. Invariance overcomes one of the fundamental difficulties in recognising objects from images: that the appearance of an object depends on viewpoint. This problem is entirely avoided if the geometric description is unaffected by the imaging transformation. Such invariant descriptions can be measured from images without any prior knowledge of the position, orientation and calibration of the camera. These invariant measurements can be used to index a library of object models for recognition and provide a principled basis for the other stages of the recognition process such as feature grouping and hypothesis verification. Object models can be acquired directly from images, allowing efficient construction of model libraries without manual intervention. A significant part of the paper is a summary of recent results on the construction of invariants for 3D objects from a single perspective view. A proposed recognition architecture is described which enables the integration of multiple general object classes and provides a means for enforcing global scene consistency. Various criticisms of the invariant approach are articulated and addressed.
---
paper_title: Viewpoint-invariant representation of generalized cylinders using the symmetry set

paper_content:

We demonstrate that viewpoint-invariant representations can be obtained from images for a useful class of 3D smooth object. The class of surfaces are those generated as the envelope of a sphere of varying radius swept along an axis. This class includes canal surfaces and surfaces of revolution. The representations are computed, using only image information, from the symmetry set of the object's outline. They are viewpoint-invariant under weak-perspective imaging, and quasi-invariant to an excellent approximation under perspective imaging. To this approximation, the planar axis of a canal surface is recovered up to an affine ambiguity from perspective images. Examples are given of the representations obtained from real images, which demonstrate stability and object discrimination, for both canal surfaces and surfaces of revolution. Finally, the representations are used as the basis for a model-based object recognition system
---
paper_title: Reflectance based object recognition

paper_content:

Neighboring points on a smoothly curved surface have similar surface normals and illumination conditions. Therefore, their brightness values can be used to compute the ratio of their reflectance coefficients. Based on this observation, we develop an algorithm that estimates a reflectance ratio for each region in an image with respect to its background. The algorithm is efficient as it computes ratios for all image regions in just two raster scans. The region reflectance ratio represents a physical property that is invariant to illumination and imaging parameters. Several experiments are conducted to demonstrate the accuracy and robustness of ratio invariant. The ratio invariant is used to recognize objects from a single brightness image of a scene. Object models are automatically acquired and represented using a hash table. Recognition and pose estimation algorithms are presented that use ratio estimates of scene regions as well as their geometric properties to index the hash table. The result is a hypothesis for the existence of an object in the image. This hypothesis is verified using the ratios and locations of other regions in the scene. This approach to recognition is effective for objects with printed characters and pictures. Recognition experiments are conducted on images with illumination variations, occlusions, and shadows. The paper is concluded with a discussion on the simultaneous use of reflectance and geometry for visual perception.
---
paper_title: Reconstruction-Based Recognition of Scenes with Translationally Repeated Quadrics

paper_content:

This paper addresses the problem of invariant-based recognition of quadric configurations from a single image. These configurations consist of a pair of rigidly connected translationally repeated quadric surfaces. This problem is approached via a reconstruction framework. A new mathematical framework, using relative affine structure, on the lines of Luong and Vieville (1996), has been proposed. Using this mathematical framework, translationally repeated objects have been projectively reconstructed, from a single image, with four image point correspondences of the distinguished points on the object and its translate. This has been used to obtain a reconstruction of a pair of translationally repeated quadrics. We have proposed joint projective invariants of a pair of proper quadrics. For the purpose of recognition of quadric configurations, we compute these invariants for the pair of reconstructed quadrics. Experimental results on synthetic and real images, establish the discriminatory power and stability of the proposed invariant-based recognition strategy. As a specific example, we have applied this technique for discriminating images of monuments which are characterized by translationally repeated domes modeled as quadrics.
---
paper_title: Three-dimensional object recognition

paper_content:

A general-purpose computer vision system must be capable of recognizing three-dimensional (3-D) objects. This paper proposes a precise definition of the 3-D object recognition problem, discusses basic concepts associated with this problem, and reviews the relevant literature. Because range images (or depth maps) are often used as sensor input instead of intensity images, techniques for obtaining, processing, and characterizing range data are also surveyed.
---
paper_title: Model-based recognition in robot vision

paper_content:

This paper presents a comparative study and survey of model-based object-recognition algorithms for robot vision. The goal of these algorithms is to recognize the identity, position, and orientation of randomly oriented industrial parts. In one form this is commonly referred to as the "bin-picking" problem, in which the parts to be recognized are presented in a jumbled bin. The paper is organized according to 2-D, 2½-D, and 3-D object representations, which are used as the basis for the recognition algorithms. Three central issues common to each category, namely, feature extraction, modeling, and matching, are examined in detail. An evaluation and comparison of existing industrial part-recognition systems and algorithms is given, providing insights for progress toward future robot vision systems.
---
paper_title: Color constant color indexing

paper_content:

Objects can be recognized on the basis of their color alone by color indexing, a technique developed by Swain-Ballard (1991) which involves matching color-space histograms. Color indexing fails, however, when the incident illumination varies either spatially or spectrally. Although this limitation might be overcome by preprocessing with a color constancy algorithm, we instead propose histogramming color ratios. Since the ratios of color RGB triples from neighboring locations are relatively insensitive to changes in the incident illumination, this circumvents the need for color constancy preprocessing. Results of tests with the new color-constant-color-indexing algorithm on synthetic and real images show that it works very well even when the illumination varies spatially in its intensity and color. >
---
paper_title: A Framework for Reconstruction based Recognition of Partially Occluded Repeated Objects

paper_content:

In this paper we propose a reconstruction based recognition scheme for objects with repeated components, using a single image of such a configuration, in which one of the repeated components may be partially occluded. In our strategy we reconstruct each of the components with respect to the same frame and use these to compute invariants.We propose a new mathematical framework for the projective reconstruction of affinely repeated objects. This uses the repetition explicitly and hence is able to handle substantial occlusion of one of the components. We then apply this framework to the reconstruction of a pair of repeated quadrics. The image information required for the reconstruction are the outline conic of one of the quadrics and correspondence between any four points which are images of points in general position on the quadric and its repetition. Projective invariants computed using the reconstructed quadrics have been used for recognition. The recognition strategy has been applied to images of monuments with multi-dome architecture. Experiments have established the discriminatory ability of the invariants.
---
paper_title: Symmetry From Shape and Shape From Symmetry

paper_content:

This article discusses the detection and use of symmetry in planar shapes. The methods are especially useful for indus trial workpieces, where symmetry is omnipresent. "Symmetry" is interpreted in a broad sense as repeated, coplanar shape fragments. In particular, fragments that are "similar" in the mathematical sense are considered symmetric. As a general tool for the extraction and analysis of symmetries, "Arc Length Space" is proposed. In this space symmetries take on a very simple form: they correspond to straight-line segments, as suming an appropriate choice is made for the shapes' contour parameterizations. Reasoning about the possible coexistence of symmetries also becomes easier in this space. Only a restricted number of symmetry patterns can be formed. By making ap propriate choices for the contour parameters, the essential properties of Arc Length Space can be inherited for general viewpoints. Invariance to affine transformations is a key is sue. Specific results include the (informal) deductio...
---
paper_title: Active and exploratory perception

paper_content:

Abstract The main goal of this paper is to show that there is a natural flow from active perception through exploration to perceptual learning. W have attempted to conceptualize the perceptual process of an organism that has the top-level task of surviving in an unknown environment. During this conceptualization process. Four necessary ingredients have emerged for either artificial or biological organisms. First, the sensory apparatus and processing of the organism must be active and flexible. Second, the organism must have exploratory capabilities. Third, the organism must be selective in its data acquisition process. Fourth, the organism must be able to learn. In the section on learning, we have clearly delineated the difference between what must be innate and what must be learned. In order to test our theory, we present the system's architecture that follows from the perceptual task decomposition. The predictions of this theory are that an artificial system can explore and learn about its environment modulo its sensors, manipulators, end effectors, and exploratory procedures/attribute extractors. It can describe its world with respect to the built-in alphabet, that is, the set of perceptual primitives.
---
paper_title: Reactions to peripheral image motion using a head/eye platform

paper_content:

The authors demonstrate four real-time reactive responses to movement in everyday scenes using an active head/eye platform. They first describe the design and realization of a high-bandwidth four-degree-of-freedom head/eye platform and visual feedback loop for the exploration of motion processing within active vision. The vision system divides processing into two scales and two broad functions. At a coarse, quasi-peripheral scale, detection and segmentation of new motion occurs across the whole image, and at fine scale, tracking of already detected motion takes place within a foveal region. Several simple coarse scale motion sensors which run concurrently at 25 Hz with latencies around 100 ms are detailed. The use of these sensors are discussed to drive the following real-time responses: (1) head/eye saccades to moving regions of interest; (2) a panic response to looming motion; (3) an opto-kinetic response to continuous motion across the image and (4) smooth pursuit of a moving target using motion alone. >
---
paper_title: Control of selective perception using bayes nets and decision theory

paper_content:

A selective vision system sequentially collects evidence to support a specified hypothesis about a scene, as long as the additional evidence is worth the effort of obtaining it. Efficiency comes from processing the scene only where necessary, to the level of detail necessary, and with only the necessary operators. Knowledge representation and sequential decision-making are central issues for selective vision, which takes advantage of prior knowledge of a domain''s abstract and geometrical structure and models for the expected performance and cost of visual operators. .pp The TEA-1 selective vision system uses Bayes nets for representation and benefit-cost analysis for control of visual and non-visual actions. It is the high-level control for an active vision system, enabling purposive behavior, the use of qualitative vision modules and a pointable multiresolution sensor. TEA-1 demonstrates that Bayes nets and decision theoretic techniques provide a general, re-usable framework for constructing computer vision systems that are selective perception systems, and that Bayes nets provide a general framework for representing visual tasks. Control, or decision making, is the most important issue in a selective vision system. TEA-1''s decisions about what to do next are based on general hand-crafted ``goodness functions'''' constructed around core decision theoretic elements. Several goodness functions for different decisions are presented and evaluated. .pp The TEA-1 system solves a version of the T-world problem, an abstraction of a large set of domains and tasks. Some key factors that affect the success of selective perception are analyzed by examining how each factor affects the overall performance of TEA-1 when solving ensembles of randomly produced, simulated T-world domains and tasks. TEA-1''s decision making algorithms are also evaluated in this manner. Experiments in the lab for one specific T-world domain, table settings, are also presented.
---
paper_title: Integration and control of reactive visual processes

paper_content:

This paper describes a new approach to the integration and control of continuously operating visual processes. Visual processes are expressed as transformations which map signals from virtual sensors into commands for devices. These transformations define reactive processes which tightly couple perception and action. Such transformations may be used to control robotic devices, including fixation an active binocular head, as well as the to select and control the processes which interpret visual data.
---
paper_title: Uncalibrated Visual Tasks via Linear Interaction

paper_content:

We propose an approach for the design and control of both reflexive and purposive visual tasks with an uncalibrated camera. The approach is based on the bi-dimensional appearance of the objects in the environment, and explicitly takes into account independent object motions. The introduction of a linear model of camera-object interaction dramatically simplifies visual analysis and control by reducing the size of the visual representation. We discuss the implementation of three tasks of increasing complexity, based on active contour analysis and polynomial planning of image contour transformations. Real-time experiments with a robot wrist-mounted camera demonstrate that the approach is conveniently usable for visual navigation, active exploration and perception, and man-robot interaction.
---
paper_title: A survey of sensor planning in computer vision

paper_content:

A survey of research in the area of vision sensor planning is presented. The problem can be summarized as follows: given information about the environment as well as information about the task that the vision system is to accomplish, develop strategies to automatically determine sensor parameter values that achieve this task with a certain degree of satisfaction. With such strategies, sensor parameters values can be selected and can be purposefully changed in order to effectively perform the task at hand. The focus here is on vision sensor planning for the task of robustly detecting object features. For this task, camera and illumination parameters such as position, orientation, and optical settings are determined so that object features are, for example, visible, in focus, within the sensor field of view, magnified as required, and imaged with sufficient contrast. References to, and a brief description of, representative sensing strategies for the tasks of object recognition and scene reconstruction are also presented. For these tasks, sensor configurations are sought that will prove most useful when trying to identify an object or reconstruct a scene. >
---
paper_title: The MVP sensor planning system for robotic vision tasks

paper_content:

The MVP (machine vision planner) model-based sensor planning system for robotic vision is presented. MVP automatically synthesizes desirable camera views of a scene based on geometric models of the environment, optical models of the vision sensors, and models of the task to be achieved. The generic task of feature detectability has been chosen since it is applicable to many robot-controlled vision systems. For such a task, features of interest in the environment are required to simultaneously be visible, inside the field of view, in focus, and magnified as required. In this paper, we present a technique that poses the vision sensor planning problem in an optimization setting and determines viewpoints that satisfy all previous requirements simultaneously and with a margin. In addition, we present experimental results of this technique when applied to a robotic vision system that consists of a camera mounted on a robot manipulator in a hand-eye configuration. >
---
paper_title: Planning for Complete Sensor Coverage in Inspection

paper_content:

Abstract General purpose CAD-based inspection of manufactured objects often involves comparing a model created using intensity or range images of the actual object to a tolerance reference model of the ideal object. Before this comparison is made, a sufficiently complete geometric model of the workpiece must be synthesized from sensor data. In this paper we present planning algorithms for finding a set of sensing operations for completely measuring the exposed surface of an object to be inspected. While these planning algorithms were developed as part of a particular inspection system, the algorithms are applicable to other inspection systems and other applications than inspection.
---
paper_title: Automatic Sensor Placement for Accurate Dimensional Inspection

paper_content:

Deriving accurate 3D object dimensions with a passive vision system demands, in general, the use of multistation sensor configurations. In such configurations, object features appear in images from multiple viewpoints, facilitating their measurement by means of optical triangulation. Previous efforts toward automatic sensor placement have been restricted to single sensor station solutions. In this paper we review photogrammetric expertise in the design of multistation configurations, including the bundle method?a general mathematical model for optical triangulation?and fundamental considerations and constraints influencing the placement of sensor stations. An overview of CONSENS, an expert system-based software tool which exploits these considerations and constraints in automatically designing multistation configurations, is given. Examples of multistation configurations designed by CONSENS demonstrate the tool's capabilities and the potential of our approach for automating sensor placement for inspection tasks.
---
paper_title: A survey of sensor planning in computer vision

paper_content:

A survey of research in the area of vision sensor planning is presented. The problem can be summarized as follows: given information about the environment as well as information about the task that the vision system is to accomplish, develop strategies to automatically determine sensor parameter values that achieve this task with a certain degree of satisfaction. With such strategies, sensor parameters values can be selected and can be purposefully changed in order to effectively perform the task at hand. The focus here is on vision sensor planning for the task of robustly detecting object features. For this task, camera and illumination parameters such as position, orientation, and optical settings are determined so that object features are, for example, visible, in focus, within the sensor field of view, magnified as required, and imaged with sufficient contrast. References to, and a brief description of, representative sensing strategies for the tasks of object recognition and scene reconstruction are also presented. For these tasks, sensor configurations are sought that will prove most useful when trying to identify an object or reconstruct a scene. >
---
paper_title: Viewpoint Selection for Complete Surface Coverage of Three Dimensional Objects

paper_content:

Many machine vision tasks, e.g. object recognition and object inspection, cannot be performed robustly from a single image. For certain tasks (e.g. 3D object recognition and automated inspection) the availability of multiple views of an object is a requirement. This paper presents a novel approach to selecting a minimised number of views that allow each object face to be adequately viewed according to specified constraints on viewpoints and other features. The planner is generic and can be employed for a wide range of multiple view acquisition systems, ranging from camera systems mounted on the end of a robot arm, i.e. an eye-in-hand camera setup, to a turntable and fixed stereo cameras to allow different views of an object to be obtained. The results (both simulated and real) given focus on planning with a fixed camera and turntable.
---
paper_title: Automatic sensor placement from vision task requirements

paper_content:

The problem of automatically generating the possible camera locations for observing an object is defined, and an approach to its solution is presented. The approach, which uses models of the object and the camera, is based on meeting the requirements that: the spatial resolution be above a minimum value, all surface points be in focus, all surfaces lie within the sensor field of view and no surface points be occluded. The approach converts each sensing requirement into a geometric constraint on the sensor location, from which the three-dimensional region of viewpoints that satisfies that constraint is computed. The intersection of these regions is the space where a sensor may be located. The extension of this approach to laser-scanner range sensors is also described. Examples illustrate the resolution, focus, and field-of-view constraints for two vision tasks. >
---
paper_title: Visual learning and recognition of 3-d objects from appearance

paper_content:

The problem of automatically learning object models for recognition and pose estimation is addressed. In contrast to the traditional approach, the recognition problem is formulated as one of matching appearance rather than shape. The appearance of an object in a two-dimensional image depends on its shape, reflectance properties, pose in the scene, and the illumination conditions. While shape and reflectance are intrinsic properties and constant for a rigid object, pose and illumination vary from scene to scene. A compact representation of object appearance is proposed that is parametrized by pose and illumination. For each object of interest, a large set of images is obtained by automatically varying pose and illumination. This image set is compressed to obtain a low-dimensional subspace, called the eigenspace, in which the object is represented as a manifold. Given an unknown input image, the recognition system projects the image to eigenspace. The object is recognized based on the manifold it lies on. The exact position of the projection on the manifold determines the object's pose in the image. A variety of experiments are conducted using objects with complex appearance characteristics. The performance of the recognition and pose estimation algorithms is studied using over a thousand input images of sample objects. Sensitivity of recognition to the number of eigenspace dimensions and the number of learning samples is analyzed. For the objects used, appearance representation in eigenspaces with less than 20 dimensions produces accurate recognition results with an average pose estimation error of about 1.0 degree. A near real-time recognition system with 20 complex objects in the database has been developed. The paper is concluded with a discussion on various issues related to the proposed learning and recognition methodology.
---
paper_title: Active Object Recognition in Parametric Eigenspace

paper_content:

We present an efficient method within an active vision framework for recognizing objects which are ambiguous from certain viewpoints. The system is allowed to reposition the camera to capture additional views and, therefore, to resolve the classification result obtained from a single view. The approach uses an appearance based object representation, namely the parametric eigenspace, and augments it by probability distributions. This captures possible variations in the input images due to errors in the pre-processing chain or the imaging system. Furthermore, the use of probability distributions gives us a gauge to view planning. View planning is shown to be of great use in reducing the number of images to be captured when compared to a random strategy.
---
paper_title: Visual learning and recognition of 3-d objects from appearance

paper_content:

The problem of automatically learning object models for recognition and pose estimation is addressed. In contrast to the traditional approach, the recognition problem is formulated as one of matching appearance rather than shape. The appearance of an object in a two-dimensional image depends on its shape, reflectance properties, pose in the scene, and the illumination conditions. While shape and reflectance are intrinsic properties and constant for a rigid object, pose and illumination vary from scene to scene. A compact representation of object appearance is proposed that is parametrized by pose and illumination. For each object of interest, a large set of images is obtained by automatically varying pose and illumination. This image set is compressed to obtain a low-dimensional subspace, called the eigenspace, in which the object is represented as a manifold. Given an unknown input image, the recognition system projects the image to eigenspace. The object is recognized based on the manifold it lies on. The exact position of the projection on the manifold determines the object's pose in the image. A variety of experiments are conducted using objects with complex appearance characteristics. The performance of the recognition and pose estimation algorithms is studied using over a thousand input images of sample objects. Sensitivity of recognition to the number of eigenspace dimensions and the number of learning samples is analyzed. For the objects used, appearance representation in eigenspaces with less than 20 dimensions produces accurate recognition results with an average pose estimation error of about 1.0 degree. A near real-time recognition system with 20 complex objects in the database has been developed. The paper is concluded with a discussion on various issues related to the proposed learning and recognition methodology.
---
paper_title: Probabilistic object recognition using multidimensional receptive field histograms

paper_content:

This paper describes a probabilistic object recognition technique which does not require correspondence matching of images. This technique is an extension of our earlier work (1996) on object recognition using matching of multi-dimensional receptive field histograms. In the earlier paper we have shown that multi-dimensional receptive field histograms can be matched to provide object recognition which is robust in the face of changes in viewing position and independent of image plane rotation and scale. In this paper we extend this method to compute the probability of the presence of an object in an image. The paper begins with a review of the method and previously presented experimental results. We then extend the method for histogram matching to obtain a genuine probability of the presence of an object. We present experimental results on a database of 100 objects showing that the approach is capable recognizing all objects correctly by using only a small portion of the image. Our results show that receptive field histograms provide a technique for object recognition which is robust, has low computational cost and a computational complexity which is linear with the number of pixels.
---
paper_title: Transinformation for Active Object Recognition

paper_content:

This article develops an analogy between object recognition and the transmission of information through a channel based on the statistical representation of the appearances of 3D objects. This analogy provides a means to quantitatively evaluate the contribution of individual receptive field vectors, and to predict the performance of the object recognition process. Transinformation also provides a quantitative measure of the discrimination provided by each viewpoint, thus permitting the determination of the most discriminant viewpoints. As an application, the article develops an active object recognition algorithm which is able to resolve ambiguities inherent in a single-view recognition algorithm.
---
paper_title: Aspect graph construction with noisy feature detectors

paper_content:

Many three-dimensional (3D) object recognition strategies use aspect graphs to represent objects in the model base. A crucial factor in the success of these object recognition strategies is the accurate construction of the aspect graph, its ease of creation, and the extent to which it can represent all views of the object for a given setup. Factors such as noise and nonadaptive thresholds may introduce errors in the feature detection process. This paper presents a characterization of errors in aspect graphs, as well as an algorithm for estimating aspect graphs, given noisy sensor data. We present extensive results of our strategies applied on a reasonably complex experimental set, and demonstrate applications to a robust 3D object recognition problem.
---
paper_title: Automatic generation of object recognition programs

paper_content:

Issues and techniques are discussed to automatically compile object and sensor models into a visual recognition strategy for recognizing and locating an object in three-dimensional space from visual data. Automatic generation of recognition programs by compilation, in an attempt to automate this process, is described. An object model describes geometric and photometric properties of an object to be recognized. A sensor model specifies the sensor characteristics in predicting object appearances and variations of feature values. It is emphasized that the sensors, as well as objects, must be explicitly modeled to achieve the goal of automatic generation of reliable and efficient recognition programs. Actual creation of interpretation trees for two objects and their execution for recognition from a bin of parts are demonstrated. >
---
paper_title: Appearance-based vision and the automatic generation of object recognition programs

paper_content:

Abstract : The generation of recognition programs by hand is a time-consuming, labor-intensive task that typically results in a special purpose program for the recognition of a single object or a small set of objects. Recent work in automatic code generation has demonstrated the feasibility of automatically generating object recognition programs from CAD-based descriptions of objects. Many of the programs which perform automatic code generation employ a common paradigm of utilizing explicit object and sensor models to predict object appearances; we refer to the paradigm as appearance-based vision, and refer to the programs as vision algorithm compilers (VACs). A CAD-like object model augmented with sensor-specific information like color and reflectance, in conjunction with a sensor model, provides all the information needed to predict the appearance of an object under any specified set of viewing conditions. Appearances, characterized in terms of feature values, can be predicted in two ways: analytically, or synthetically. In relatively simple domains, feature values can be analytically determined from model information. However, in complex domains, the analytic prediction method is impractical. An alternative method for appearance prediction is to use an appearance simulator to generate synthetic im ages of objects which can then be processed to extract feature values. In this paper, we discuss the paradigm of appearance-based vision and present in detail two specific VACs: one that computes feature values analytically, and a second that utilizes an appearance simulator to synthesize sample images.
---
paper_title: Characteristic Views As A Basis For Three-Dimensional Object Recognition

paper_content:

This paper describes a new technique for modeling 3D objects that is applicable to recog-nition tasks in advanced automation. Objects are represented in terms of canonic 2D models which can be used to determine the identity, location and orientation of an unknown object. The reduction in dimensionality is achieved by factoring the space of all possible perspective projections of an object into a set of characteristic views, where each such view defines a characteristic-view domain within which all projections are topologically identical and related by a linear transformation. The characteristic views of an object can then be hierarchically structured for efficient classification. The line-junction labelling constraints are used to match a characteristic view to a given unknown-object projection, and determination of the unknown-object projection-to-characteristic view transformation then provides information about the identity as well as the location and orientation of the object.
---
paper_title: The Scale Space Aspect Graph

paper_content:

Currently the aspect graph is computed from the theoretical standpoint of perfect resolution in object shape, the viewpoint and the projected image. This means that the aspect graph may include details that an observer could never see in practice. Introducing the notion of scale into the aspect graph framework provides a mechanism for selecting a level of detail that is "large enough" to merit explicit representation. This effectively allows control over the number of nodes retained in the aspect graph. This paper introduces the concept of the scale space aspect graph, defines three different interpretations of the scale dimension, and presents a detailed example for a simple class of objects, with scale defined in terms of the spatial extent of features in the image. >
---
paper_title: Planning multiple observations for object recognition

paper_content:

Most computer vision systems perform object recognition on the basis of the features extracted from a single image of the object. The problem with this approach is that it implicitly assumes that the available features are sufficient to determine the identity and pose of the object uniquely. If this assumption is not met, then the feature set is insufficient, and ambiguity results. Consequently, much research in computer vision has gone toward finding sets of features that are sufficient for specific tasks, with the result that each system has its own associated set of features. A single, general feature set would be desirable. However, research in automatic generation of object recognition programs has demonstrated that predetermined, fixed feature sets are often incapable of providing enough information to unambiguously determine either object identity or pose. One approach to overcoming the inadequacy of any feature set is to utilize multiple sensor observations obtained from different viewpoints, and combine them with knowledge of the 3-D structure of the object to perform unambiguous object recognition. This article presents initial results toward performing object recognition by using multiple observations to resolve ambiguities. Starting from the premise that sensor motions should be planned in advance, the difficulties involved in planning with ambiguous information are discussed. A representation for planning that combines geometric information with viewpoint uncertainty is presented. A sensor planner utilizing the representation was implemented, and the results of pose-determination experiments performed with the planner are discussed.
---
paper_title: An Investigation into the Use of Physical Modeling for the Prediction of Various Feature Types Visible from Different Viewpoints

paper_content:

Given that aspect graph and viewsphere-based object recognition systems provide a valid mechanism for 3D object recognition of man-made objects, this paper provides a flexible, automated, and general purpose technique for generating the view information for each viewpoint. An advantage of the work is that the technique is unaffected by object complexity because each step makes no assumptions about object shape. The only limitation is that the object can be described by a boundary representation. A second advantage is that the technique can include other feature types such as specularity. The reason for this is that raytracing techniques are used to simulate the physical process of image generation. Hence it is extendible to visible features resulting from effects due to lighting, surface texture, color, transparency, etc. The work described in this paper shows how occluding and nonoccluding edge-based features can be extracted using image processing techniques and then parametrized and also how regions of specularity can be predicted and described. The use of physical modeling enables situations to be simulated and predicted that are intractable for CAD-based methods (e.g., multiscale feature prediction). An advantage of the method is that the interface between the technique and the raytracing module is a rendered image. Should better physics-based image formation algorithms become available, then they could replace the raytracing module with little modification to the rest of the method.
---
paper_title: The internal representation of solid shape with respect to vision

paper_content:

It is argued that the internal model of any object must take the form of a function, such that for any intended action the resulting reafference is predictable. This function can be derived explicitly for the case of visual perception of rigid bodies by ambulant observers. The function depends on physical causation, not physiology; consequently, one can make a priori statements about possible internal models. A posteriori it seems likely that the orientation sensitive units described by Hubel and Wiesel constitute a physiological substrate subserving the extraction of the invariants of this function. The function is used to define a measure for the visual complexity of solid shape. Relations with Gestalt theories of perception are discussed.
---
paper_title: A relational pyramid approach to view class determination

paper_content:

Given a CAD model of an object, the authors would like to automatically generate a vision model and matching procedure that can be used in robot guidance and inspection tasks. They are building a system that can predict features that will appear in a 2D view of a 3D object, represent each such view with a hierarchical, relational structure, group together similar views into view classes, and match an unknown view to the appropriate view class to find its pose. They describe the relational pyramid structure for describing the features in a particular view or view class of an object, the summary structure that is used to summarize the relational information in the relational pyramid, and an accumulator-based method for rapidly determining the view class(es) that best match an unknown view of an object. >
---
paper_title: Planning sensing strategies in a robot work cell with multi-sensor capabilities

paper_content:

An approach is presented for planning sensing strategies dynamically on the basis of the system's current best information about the world. The approach is for the system to propose a sensing operation automatically and then to determine the maximum ambiguity which might remain in the world description if that sensing operation were applied. The system then applies that sensing operation which minimizes this ambiguity. To do this, the system formulates object hypotheses and assesses its relative belief in those hypotheses to predict what features might be observed by a proposed sensing operation. Furthermore, since the number of sensing operations available to the system can be arbitrarily large, equivalent sensing operations are grouped together using a data structure that is based on the aspect graph. In order to measure the ambiguity in a set of hypotheses, the authors apply the concept of entropy from information theory. This allows them to determine the ambiguity in a hypothesis set in terms of the number of hypotheses and the system's distribution of belief among those hypotheses. >
---
paper_title: Qualitative 3-D shape reconstruction using distributed aspect graph matching

paper_content:

An approach is presented to 3-D primitive reconstruction that is independent of the selection of volumetric primitives used to model objects. The approach first takes an arbitrary set of 3-D volumetric primitives and generates a hierarchical aspect representation based on the projected surfaces of the primitives; conditional probabilities capture the ambiguity of mappings between levels of the hierarchy. The integration of object-centered and viewer-centered representations provides the indexing power of 3-D volumetric primitives, while supporting a 2-D matching paradigm for primitive reconstruction. Formulation of the problem based on grouping the image regions according to aspect is presented. No domain dependent heuristics are used; the authors exploit only the probabilities inherent in the aspect hierarchy. For a given selection of primitives, the success of the heuristic depends on the likelihood of the various aspects; best results are achieved when certain aspects are more likely, and fewer primitives project to a given aspect. >
---
paper_title: Active Object Recognition Integrating Attention and Viewpoint Control

paper_content:

We present an active object recognition strategy which combines the use of an attention mechanism for focusing the search for a 3D object in a 2D image, with a viewpoint control strategy for disambiguating recovered object features. The attention mechanism consists of a probabilistic search through a hierarchy of predicted feature observations, taking objects into a set of regions classified according to the shapes of their bounding contours. We motivate the use of image regions as a focus-feature and compare their uncertainty in inferring objects with the uncertainty of more commonly used features such as lines or corners. If the features recovered during the attention phase do not provide a unique mapping to the 3D object being searched, the probabilistic feature hierarchy can be used to guide the camera to a new viewpoint from where the object can be disambiguated. The power of the underlying representation is its ability to unify these object recognition behaviors within a single framework. We present the approach in detail and evaluate its performance in the context of a project providing robotic aids for the disabled.
---
paper_title: Recognizing large 3-D objects through next view planning using an uncalibrated camera

paper_content:

We present a new on-line scheme for the recognition and pose estimation of a large isolated 3-D object, which may not entirely fit in a camera's field of view. We do not assume any knowledge of the internal parameters of the camera, or their constancy. We use a probabilistic reasoning framework for recognition and next view planning. We show results of successful recognition and pose estimation even in cases of a high degree of interpretation ambiguity associated with the initial view.
---
paper_title: Active Object Recognition Integrating Attention and Viewpoint Control

paper_content:

We present an active object recognition strategy which combines the use of an attention mechanism for focusing the search for a 3D object in a 2D image, with a viewpoint control strategy for disambiguating recovered object features. The attention mechanism consists of a probabilistic search through a hierarchy of predicted feature observations, taking objects into a set of regions classified according to the shapes of their bounding contours. We motivate the use of image regions as a focus-feature and compare their uncertainty in inferring objects with the uncertainty of more commonly used features such as lines or corners. If the features recovered during the attention phase do not provide a unique mapping to the 3D object being searched, the probabilistic feature hierarchy can be used to guide the camera to a new viewpoint from where the object can be disambiguated. The power of the underlying representation is its ability to unify these object recognition behaviors within a single framework. We present the approach in detail and evaluate its performance in the context of a project providing robotic aids for the disabled.
---
paper_title: Generic object recognition: building and matching coarse descriptions from line drawings

paper_content:

Primal access recognition of visual objects (PARVO), a computer vision system that addresses the problem of fast and generic recognition of unexpected 3D objects from single 2D views, is considered. Recently, recognition by components (RBC), which is a new human image understanding theory, based on some psychological results, has been proposed as an explanation of how PARVO works. However, no systematic computational evaluation of its many aspects has yet been reported. The PARVO system discussed is a first step toward this goal, since its design respects and makes explicit the main assumptions of the proposed theory. It analyzes single-view 2D line drawings of 3D objects typical of the ones used in human image understanding studies. It is designed to handle partially occluded objects of different shape and dimension in various spatial orientations and locations in the image plane. The system is shown to successfully compute generic descriptions and then recognize many common man-made objects. >
---
paper_title: Qualitative 3-D shape reconstruction using distributed aspect graph matching

paper_content:

An approach is presented to 3-D primitive reconstruction that is independent of the selection of volumetric primitives used to model objects. The approach first takes an arbitrary set of 3-D volumetric primitives and generates a hierarchical aspect representation based on the projected surfaces of the primitives; conditional probabilities capture the ambiguity of mappings between levels of the hierarchy. The integration of object-centered and viewer-centered representations provides the indexing power of 3-D volumetric primitives, while supporting a 2-D matching paradigm for primitive reconstruction. Formulation of the problem based on grouping the image regions according to aspect is presented. No domain dependent heuristics are used; the authors exploit only the probabilities inherent in the aspect hierarchy. For a given selection of primitives, the success of the heuristic depends on the likelihood of the various aspects; best results are achieved when certain aspects are more likely, and fewer primitives project to a given aspect. >
---
paper_title: Active Object Recognition Integrating Attention and Viewpoint Control

paper_content:

We present an active object recognition strategy which combines the use of an attention mechanism for focusing the search for a 3D object in a 2D image, with a viewpoint control strategy for disambiguating recovered object features. The attention mechanism consists of a probabilistic search through a hierarchy of predicted feature observations, taking objects into a set of regions classified according to the shapes of their bounding contours. We motivate the use of image regions as a focus-feature and compare their uncertainty in inferring objects with the uncertainty of more commonly used features such as lines or corners. If the features recovered during the attention phase do not provide a unique mapping to the 3D object being searched, the probabilistic feature hierarchy can be used to guide the camera to a new viewpoint from where the object can be disambiguated. The power of the underlying representation is its ability to unify these object recognition behaviors within a single framework. We present the approach in detail and evaluate its performance in the context of a project providing robotic aids for the disabled.
---
paper_title: Object recognition using appearance-based parts and relations

paper_content:

The recognition of general three-dimensional objects in cluttered scenes is a challenging problem. In particular, the design of a good representation suitable to model large numbers of generic objects that is also robust to occlusion has been a stumbling block in achieving success. In this paper, we propose a representation using appearance-based parts and relations to overcome these problems. Appearance-based parts and relations are defined in terms of closed regions and the union of these regions, respectively. The regions are segmented using the MDL principle, and their appearance is obtained from collection of images and compactly represented by parametric manifolds in the two eigenspaces spanned by the parts and the relations.
---
paper_title: Recognizing large 3-D objects through next view planning using an uncalibrated camera

paper_content:

We present a new on-line scheme for the recognition and pose estimation of a large isolated 3-D object, which may not entirely fit in a camera's field of view. We do not assume any knowledge of the internal parameters of the camera, or their constancy. We use a probabilistic reasoning framework for recognition and next view planning. We show results of successful recognition and pose estimation even in cases of a high degree of interpretation ambiguity associated with the initial view.
---
paper_title: Hierarchical organization of appearance-based parts and relations for object recognition

paper_content:

Previously a new object representation using appearance-based parts and relations to recognize 3D objects from 2D images, in the presence of occlusion and background clutter, was introduced. Appearance-based parts and relations are defined in terms of closed regions and the union of these regions, respectively. The regions are segmented using the MDL principle, and their appearance is obtained from collection of images and compactly represented by parametric manifolds in the eigenspaces spanned by the parts and the relations. In this paper we introduce the discriminatory power of the proposed features and describe how to use it to organize large databases of objects.
---
paper_title: Fusion, Propagation, and Structuring in Belief Networks

paper_content:

Belief networks are directed acyclic graphs in which the nodes represent propositions (or variables), the arcs signify direct dependencies between the linked propositions, and the strengths of these dependencies are quantified by conditional probabilities. A network of this sort can be used to represent the generic knowledge of a domain expert, and it turns into a computational architecture if the links are used not merely for storing factual knowledge but also for directing and activating the data flow in the computations which manipulate this knowledge. The first part of the paper deals with the task of fusing and propagating the impacts of new information through the networks in such a way that, when equilibrium is reached, each proposition will be assigned a measure of belief consistent with the axioms of probability theory. It is shown that if the network is singly connected (e.g. tree-structured), then probabilities can be updated by local propagation in an isomorphic network of parallel and autonomous processors and that the impact of new information can be imparted to all propositions in time proportional to the longest path in the network. The second part of the paper deals with the problem of finding a tree-structured representation for a collection of probabilistically coupled propositions using auxiliary (dummy) variables, colloquially called "hidden causes." It is shown that if such a tree-structured representation exists, then it is possible to uniquely uncover the topology of the tree by observing pairwise dependencies among the available propositions (i.e., the leaves of the tree). The entire tree structure, including the strengths of all internal relationships, can be reconstructed in time proportional to n log n, where n is the number of leaves.
---
paper_title: A Generalization of Bayesian Inference

paper_content:

Procedures of statistical inference are described which generalize Bayesian inference in specific ways. Probability is used in such a way that in general only bounds may be placed on the probabilities of given events, and probability systems of this kind are suggested both for sample information and for prior information. These systems are then combined using a specified rule. Illustrations are given for inferences about trinomial probabilities, and for inferences about a monotone sequence of binomial pi. Finally, some comments are made on the general class of models which produce upper and lower probabilities, and on the specific models which underlie the suggested inference procedures.
---
paper_title: Control of selective perception using bayes nets and decision theory

paper_content:

A selective vision system sequentially collects evidence to support a specified hypothesis about a scene, as long as the additional evidence is worth the effort of obtaining it. Efficiency comes from processing the scene only where necessary, to the level of detail necessary, and with only the necessary operators. Knowledge representation and sequential decision-making are central issues for selective vision, which takes advantage of prior knowledge of a domain''s abstract and geometrical structure and models for the expected performance and cost of visual operators. .pp The TEA-1 selective vision system uses Bayes nets for representation and benefit-cost analysis for control of visual and non-visual actions. It is the high-level control for an active vision system, enabling purposive behavior, the use of qualitative vision modules and a pointable multiresolution sensor. TEA-1 demonstrates that Bayes nets and decision theoretic techniques provide a general, re-usable framework for constructing computer vision systems that are selective perception systems, and that Bayes nets provide a general framework for representing visual tasks. Control, or decision making, is the most important issue in a selective vision system. TEA-1''s decisions about what to do next are based on general hand-crafted ``goodness functions'''' constructed around core decision theoretic elements. Several goodness functions for different decisions are presented and evaluated. .pp The TEA-1 system solves a version of the T-world problem, an abstraction of a large set of domains and tasks. Some key factors that affect the success of selective perception are analyzed by examining how each factor affects the overall performance of TEA-1 when solving ensembles of randomly produced, simulated T-world domains and tasks. TEA-1''s decision making algorithms are also evaluated in this manner. Experiments in the lab for one specific T-world domain, table settings, are also presented.
---
paper_title: Planning sensing strategies in a robot work cell with multi-sensor capabilities

paper_content:

An approach is presented for planning sensing strategies dynamically on the basis of the system's current best information about the world. The approach is for the system to propose a sensing operation automatically and then to determine the maximum ambiguity which might remain in the world description if that sensing operation were applied. The system then applies that sensing operation which minimizes this ambiguity. To do this, the system formulates object hypotheses and assesses its relative belief in those hypotheses to predict what features might be observed by a proposed sensing operation. Furthermore, since the number of sensing operations available to the system can be arbitrarily large, equivalent sensing operations are grouped together using a data structure that is based on the aspect graph. In order to measure the ambiguity in a set of hypotheses, the authors apply the concept of entropy from information theory. This allows them to determine the ambiguity in a hypothesis set in terms of the number of hypotheses and the system's distribution of belief among those hypotheses. >
---
paper_title: Active Object Recognition Integrating Attention and Viewpoint Control

paper_content:

We present an active object recognition strategy which combines the use of an attention mechanism for focusing the search for a 3D object in a 2D image, with a viewpoint control strategy for disambiguating recovered object features. The attention mechanism consists of a probabilistic search through a hierarchy of predicted feature observations, taking objects into a set of regions classified according to the shapes of their bounding contours. We motivate the use of image regions as a focus-feature and compare their uncertainty in inferring objects with the uncertainty of more commonly used features such as lines or corners. If the features recovered during the attention phase do not provide a unique mapping to the 3D object being searched, the probabilistic feature hierarchy can be used to guide the camera to a new viewpoint from where the object can be disambiguated. The power of the underlying representation is its ability to unify these object recognition behaviors within a single framework. We present the approach in detail and evaluate its performance in the context of a project providing robotic aids for the disabled.
---
paper_title: Recognizing large 3-D objects through next view planning using an uncalibrated camera

paper_content:

We present a new on-line scheme for the recognition and pose estimation of a large isolated 3-D object, which may not entirely fit in a camera's field of view. We do not assume any knowledge of the internal parameters of the camera, or their constancy. We use a probabilistic reasoning framework for recognition and next view planning. We show results of successful recognition and pose estimation even in cases of a high degree of interpretation ambiguity associated with the initial view.
---
paper_title: Planning sensing strategies in a robot work cell with multi-sensor capabilities

paper_content:

An approach is presented for planning sensing strategies dynamically on the basis of the system's current best information about the world. The approach is for the system to propose a sensing operation automatically and then to determine the maximum ambiguity which might remain in the world description if that sensing operation were applied. The system then applies that sensing operation which minimizes this ambiguity. To do this, the system formulates object hypotheses and assesses its relative belief in those hypotheses to predict what features might be observed by a proposed sensing operation. Furthermore, since the number of sensing operations available to the system can be arbitrarily large, equivalent sensing operations are grouped together using a data structure that is based on the aspect graph. In order to measure the ambiguity in a set of hypotheses, the authors apply the concept of entropy from information theory. This allows them to determine the ambiguity in a hypothesis set in terms of the number of hypotheses and the system's distribution of belief among those hypotheses. >
---
paper_title: Active recognition: using uncertainty to reduce ambiguity

paper_content:

Scene ambiguity, due to noisy measurements and uncertain object models, can be quantified and actively used by an autonomous agent to efficiently gather new data and improve its information about the environment. In this work an information-based utility measure is used to derive from a learned classification of shape models an efficient data collection strategy, specifically aimed at increasing classification confidence when recognizing uncertain shapes.
---
paper_title: Transinformation for Active Object Recognition

paper_content:

This article develops an analogy between object recognition and the transmission of information through a channel based on the statistical representation of the appearances of 3D objects. This analogy provides a means to quantitatively evaluate the contribution of individual receptive field vectors, and to predict the performance of the object recognition process. Transinformation also provides a quantitative measure of the discrimination provided by each viewpoint, thus permitting the determination of the most discriminant viewpoints. As an application, the article develops an active object recognition algorithm which is able to resolve ambiguities inherent in a single-view recognition algorithm.
---
paper_title: Active Object Recognition Integrating Attention and Viewpoint Control

paper_content:

We present an active object recognition strategy which combines the use of an attention mechanism for focusing the search for a 3D object in a 2D image, with a viewpoint control strategy for disambiguating recovered object features. The attention mechanism consists of a probabilistic search through a hierarchy of predicted feature observations, taking objects into a set of regions classified according to the shapes of their bounding contours. We motivate the use of image regions as a focus-feature and compare their uncertainty in inferring objects with the uncertainty of more commonly used features such as lines or corners. If the features recovered during the attention phase do not provide a unique mapping to the 3D object being searched, the probabilistic feature hierarchy can be used to guide the camera to a new viewpoint from where the object can be disambiguated. The power of the underlying representation is its ability to unify these object recognition behaviors within a single framework. We present the approach in detail and evaluate its performance in the context of a project providing robotic aids for the disabled.
---
paper_title: Active Object Recognition in Parametric Eigenspace

paper_content:

We present an efficient method within an active vision framework for recognizing objects which are ambiguous from certain viewpoints. The system is allowed to reposition the camera to capture additional views and, therefore, to resolve the classification result obtained from a single view. The approach uses an appearance based object representation, namely the parametric eigenspace, and augments it by probability distributions. This captures possible variations in the input images due to errors in the pre-processing chain or the imaging system. Furthermore, the use of probability distributions gives us a gauge to view planning. View planning is shown to be of great use in reducing the number of images to be captured when compared to a random strategy.
---
paper_title: Visual learning and recognition of 3-d objects from appearance

paper_content:

The problem of automatically learning object models for recognition and pose estimation is addressed. In contrast to the traditional approach, the recognition problem is formulated as one of matching appearance rather than shape. The appearance of an object in a two-dimensional image depends on its shape, reflectance properties, pose in the scene, and the illumination conditions. While shape and reflectance are intrinsic properties and constant for a rigid object, pose and illumination vary from scene to scene. A compact representation of object appearance is proposed that is parametrized by pose and illumination. For each object of interest, a large set of images is obtained by automatically varying pose and illumination. This image set is compressed to obtain a low-dimensional subspace, called the eigenspace, in which the object is represented as a manifold. Given an unknown input image, the recognition system projects the image to eigenspace. The object is recognized based on the manifold it lies on. The exact position of the projection on the manifold determines the object's pose in the image. A variety of experiments are conducted using objects with complex appearance characteristics. The performance of the recognition and pose estimation algorithms is studied using over a thousand input images of sample objects. Sensitivity of recognition to the number of eigenspace dimensions and the number of learning samples is analyzed. For the objects used, appearance representation in eigenspaces with less than 20 dimensions produces accurate recognition results with an average pose estimation error of about 1.0 degree. A near real-time recognition system with 20 complex objects in the database has been developed. The paper is concluded with a discussion on various issues related to the proposed learning and recognition methodology.
---
paper_title: Planning multiple observations for object recognition

paper_content:

Most computer vision systems perform object recognition on the basis of the features extracted from a single image of the object. The problem with this approach is that it implicitly assumes that the available features are sufficient to determine the identity and pose of the object uniquely. If this assumption is not met, then the feature set is insufficient, and ambiguity results. Consequently, much research in computer vision has gone toward finding sets of features that are sufficient for specific tasks, with the result that each system has its own associated set of features. A single, general feature set would be desirable. However, research in automatic generation of object recognition programs has demonstrated that predetermined, fixed feature sets are often incapable of providing enough information to unambiguously determine either object identity or pose. One approach to overcoming the inadequacy of any feature set is to utilize multiple sensor observations obtained from different viewpoints, and combine them with knowledge of the 3-D structure of the object to perform unambiguous object recognition. This article presents initial results toward performing object recognition by using multiple observations to resolve ambiguities. Starting from the premise that sensor motions should be planned in advance, the difficulties involved in planning with ambiguous information are discussed. A representation for planning that combines geometric information with viewpoint uncertainty is presented. A sensor planner utilizing the representation was implemented, and the results of pose-determination experiments performed with the planner are discussed.
---
paper_title: Aspect graph construction with noisy feature detectors

paper_content:

Many three-dimensional (3D) object recognition strategies use aspect graphs to represent objects in the model base. A crucial factor in the success of these object recognition strategies is the accurate construction of the aspect graph, its ease of creation, and the extent to which it can represent all views of the object for a given setup. Factors such as noise and nonadaptive thresholds may introduce errors in the feature detection process. This paper presents a characterization of errors in aspect graphs, as well as an algorithm for estimating aspect graphs, given noisy sensor data. We present extensive results of our strategies applied on a reasonably complex experimental set, and demonstrate applications to a robust 3D object recognition problem.
---
paper_title: Robot localization using uncalibrated camera invariants

paper_content:

We describe a set of image measurements which are invariant to the camera internals but are location variant. We show that using these measurements it is possible to calculate the self-localization of a robot using known landmarks and uncalibrated cameras. We also show that it is possible to compute, using uncalibrated cameras, the Euclidean structure of 3-D world points using multiple views from known positions. We are free to alter the internal parameters of the camera during these operations. Our initial experiments demonstrate the applicability of the method.
---
paper_title: Recognizing large 3-D objects through next view planning using an uncalibrated camera

paper_content:

We present a new on-line scheme for the recognition and pose estimation of a large isolated 3-D object, which may not entirely fit in a camera's field of view. We do not assume any knowledge of the internal parameters of the camera, or their constancy. We use a probabilistic reasoning framework for recognition and next view planning. We show results of successful recognition and pose estimation even in cases of a high degree of interpretation ambiguity associated with the initial view.
---
paper_title: Planning multiple observations for object recognition

paper_content:

Most computer vision systems perform object recognition on the basis of the features extracted from a single image of the object. The problem with this approach is that it implicitly assumes that the available features are sufficient to determine the identity and pose of the object uniquely. If this assumption is not met, then the feature set is insufficient, and ambiguity results. Consequently, much research in computer vision has gone toward finding sets of features that are sufficient for specific tasks, with the result that each system has its own associated set of features. A single, general feature set would be desirable. However, research in automatic generation of object recognition programs has demonstrated that predetermined, fixed feature sets are often incapable of providing enough information to unambiguously determine either object identity or pose. One approach to overcoming the inadequacy of any feature set is to utilize multiple sensor observations obtained from different viewpoints, and combine them with knowledge of the 3-D structure of the object to perform unambiguous object recognition. This article presents initial results toward performing object recognition by using multiple observations to resolve ambiguities. Starting from the premise that sensor motions should be planned in advance, the difficulties involved in planning with ambiguous information are discussed. A representation for planning that combines geometric information with viewpoint uncertainty is presented. A sensor planner utilizing the representation was implemented, and the results of pose-determination experiments performed with the planner are discussed.
---
paper_title: Active Object Recognition in Parametric Eigenspace

paper_content:

We present an efficient method within an active vision framework for recognizing objects which are ambiguous from certain viewpoints. The system is allowed to reposition the camera to capture additional views and, therefore, to resolve the classification result obtained from a single view. The approach uses an appearance based object representation, namely the parametric eigenspace, and augments it by probability distributions. This captures possible variations in the input images due to errors in the pre-processing chain or the imaging system. Furthermore, the use of probability distributions gives us a gauge to view planning. View planning is shown to be of great use in reducing the number of images to be captured when compared to a random strategy.
---
paper_title: Planning sensing strategies in a robot work cell with multi-sensor capabilities

paper_content:

An approach is presented for planning sensing strategies dynamically on the basis of the system's current best information about the world. The approach is for the system to propose a sensing operation automatically and then to determine the maximum ambiguity which might remain in the world description if that sensing operation were applied. The system then applies that sensing operation which minimizes this ambiguity. To do this, the system formulates object hypotheses and assesses its relative belief in those hypotheses to predict what features might be observed by a proposed sensing operation. Furthermore, since the number of sensing operations available to the system can be arbitrarily large, equivalent sensing operations are grouped together using a data structure that is based on the aspect graph. In order to measure the ambiguity in a set of hypotheses, the authors apply the concept of entropy from information theory. This allows them to determine the ambiguity in a hypothesis set in terms of the number of hypotheses and the system's distribution of belief among those hypotheses. >
---
paper_title: Active Object Recognition in Parametric Eigenspace

paper_content:

We present an efficient method within an active vision framework for recognizing objects which are ambiguous from certain viewpoints. The system is allowed to reposition the camera to capture additional views and, therefore, to resolve the classification result obtained from a single view. The approach uses an appearance based object representation, namely the parametric eigenspace, and augments it by probability distributions. This captures possible variations in the input images due to errors in the pre-processing chain or the imaging system. Furthermore, the use of probability distributions gives us a gauge to view planning. View planning is shown to be of great use in reducing the number of images to be captured when compared to a random strategy.
---
paper_title: Planning sensing strategies in a robot work cell with multi-sensor capabilities

paper_content:

An approach is presented for planning sensing strategies dynamically on the basis of the system's current best information about the world. The approach is for the system to propose a sensing operation automatically and then to determine the maximum ambiguity which might remain in the world description if that sensing operation were applied. The system then applies that sensing operation which minimizes this ambiguity. To do this, the system formulates object hypotheses and assesses its relative belief in those hypotheses to predict what features might be observed by a proposed sensing operation. Furthermore, since the number of sensing operations available to the system can be arbitrarily large, equivalent sensing operations are grouped together using a data structure that is based on the aspect graph. In order to measure the ambiguity in a set of hypotheses, the authors apply the concept of entropy from information theory. This allows them to determine the ambiguity in a hypothesis set in terms of the number of hypotheses and the system's distribution of belief among those hypotheses. >
---
paper_title: Active Object Recognition Integrating Attention and Viewpoint Control

paper_content:

We present an active object recognition strategy which combines the use of an attention mechanism for focusing the search for a 3D object in a 2D image, with a viewpoint control strategy for disambiguating recovered object features. The attention mechanism consists of a probabilistic search through a hierarchy of predicted feature observations, taking objects into a set of regions classified according to the shapes of their bounding contours. We motivate the use of image regions as a focus-feature and compare their uncertainty in inferring objects with the uncertainty of more commonly used features such as lines or corners. If the features recovered during the attention phase do not provide a unique mapping to the 3D object being searched, the probabilistic feature hierarchy can be used to guide the camera to a new viewpoint from where the object can be disambiguated. The power of the underlying representation is its ability to unify these object recognition behaviors within a single framework. We present the approach in detail and evaluate its performance in the context of a project providing robotic aids for the disabled.
---
paper_title: Planning sensing strategies in a robot work cell with multi-sensor capabilities

paper_content:

An approach is presented for planning sensing strategies dynamically on the basis of the system's current best information about the world. The approach is for the system to propose a sensing operation automatically and then to determine the maximum ambiguity which might remain in the world description if that sensing operation were applied. The system then applies that sensing operation which minimizes this ambiguity. To do this, the system formulates object hypotheses and assesses its relative belief in those hypotheses to predict what features might be observed by a proposed sensing operation. Furthermore, since the number of sensing operations available to the system can be arbitrarily large, equivalent sensing operations are grouped together using a data structure that is based on the aspect graph. In order to measure the ambiguity in a set of hypotheses, the authors apply the concept of entropy from information theory. This allows them to determine the ambiguity in a hypothesis set in terms of the number of hypotheses and the system's distribution of belief among those hypotheses. >
---
paper_title: Active Object Recognition Integrating Attention and Viewpoint Control

paper_content:

We present an active object recognition strategy which combines the use of an attention mechanism for focusing the search for a 3D object in a 2D image, with a viewpoint control strategy for disambiguating recovered object features. The attention mechanism consists of a probabilistic search through a hierarchy of predicted feature observations, taking objects into a set of regions classified according to the shapes of their bounding contours. We motivate the use of image regions as a focus-feature and compare their uncertainty in inferring objects with the uncertainty of more commonly used features such as lines or corners. If the features recovered during the attention phase do not provide a unique mapping to the 3D object being searched, the probabilistic feature hierarchy can be used to guide the camera to a new viewpoint from where the object can be disambiguated. The power of the underlying representation is its ability to unify these object recognition behaviors within a single framework. We present the approach in detail and evaluate its performance in the context of a project providing robotic aids for the disabled.
---
paper_title: Planning multiple observations for object recognition

paper_content:

Most computer vision systems perform object recognition on the basis of the features extracted from a single image of the object. The problem with this approach is that it implicitly assumes that the available features are sufficient to determine the identity and pose of the object uniquely. If this assumption is not met, then the feature set is insufficient, and ambiguity results. Consequently, much research in computer vision has gone toward finding sets of features that are sufficient for specific tasks, with the result that each system has its own associated set of features. A single, general feature set would be desirable. However, research in automatic generation of object recognition programs has demonstrated that predetermined, fixed feature sets are often incapable of providing enough information to unambiguously determine either object identity or pose. One approach to overcoming the inadequacy of any feature set is to utilize multiple sensor observations obtained from different viewpoints, and combine them with knowledge of the 3-D structure of the object to perform unambiguous object recognition. This article presents initial results toward performing object recognition by using multiple observations to resolve ambiguities. Starting from the premise that sensor motions should be planned in advance, the difficulties involved in planning with ambiguous information are discussed. A representation for planning that combines geometric information with viewpoint uncertainty is presented. A sensor planner utilizing the representation was implemented, and the results of pose-determination experiments performed with the planner are discussed.
---
paper_title: Active Object Recognition in Parametric Eigenspace

paper_content:

We present an efficient method within an active vision framework for recognizing objects which are ambiguous from certain viewpoints. The system is allowed to reposition the camera to capture additional views and, therefore, to resolve the classification result obtained from a single view. The approach uses an appearance based object representation, namely the parametric eigenspace, and augments it by probability distributions. This captures possible variations in the input images due to errors in the pre-processing chain or the imaging system. Furthermore, the use of probability distributions gives us a gauge to view planning. View planning is shown to be of great use in reducing the number of images to be captured when compared to a random strategy.
---
paper_title: Recognizing large 3-D objects through next view planning using an uncalibrated camera

paper_content:

We present a new on-line scheme for the recognition and pose estimation of a large isolated 3-D object, which may not entirely fit in a camera's field of view. We do not assume any knowledge of the internal parameters of the camera, or their constancy. We use a probabilistic reasoning framework for recognition and next view planning. We show results of successful recognition and pose estimation even in cases of a high degree of interpretation ambiguity associated with the initial view.
---
paper_title: Planning multiple observations for object recognition

paper_content:

Most computer vision systems perform object recognition on the basis of the features extracted from a single image of the object. The problem with this approach is that it implicitly assumes that the available features are sufficient to determine the identity and pose of the object uniquely. If this assumption is not met, then the feature set is insufficient, and ambiguity results. Consequently, much research in computer vision has gone toward finding sets of features that are sufficient for specific tasks, with the result that each system has its own associated set of features. A single, general feature set would be desirable. However, research in automatic generation of object recognition programs has demonstrated that predetermined, fixed feature sets are often incapable of providing enough information to unambiguously determine either object identity or pose. One approach to overcoming the inadequacy of any feature set is to utilize multiple sensor observations obtained from different viewpoints, and combine them with knowledge of the 3-D structure of the object to perform unambiguous object recognition. This article presents initial results toward performing object recognition by using multiple observations to resolve ambiguities. Starting from the premise that sensor motions should be planned in advance, the difficulties involved in planning with ambiguous information are discussed. A representation for planning that combines geometric information with viewpoint uncertainty is presented. A sensor planner utilizing the representation was implemented, and the results of pose-determination experiments performed with the planner are discussed.
---
paper_title: Active Object Recognition in Parametric Eigenspace

paper_content:

We present an efficient method within an active vision framework for recognizing objects which are ambiguous from certain viewpoints. The system is allowed to reposition the camera to capture additional views and, therefore, to resolve the classification result obtained from a single view. The approach uses an appearance based object representation, namely the parametric eigenspace, and augments it by probability distributions. This captures possible variations in the input images due to errors in the pre-processing chain or the imaging system. Furthermore, the use of probability distributions gives us a gauge to view planning. View planning is shown to be of great use in reducing the number of images to be captured when compared to a random strategy.
---
paper_title: Planning sensing strategies in a robot work cell with multi-sensor capabilities

paper_content:

An approach is presented for planning sensing strategies dynamically on the basis of the system's current best information about the world. The approach is for the system to propose a sensing operation automatically and then to determine the maximum ambiguity which might remain in the world description if that sensing operation were applied. The system then applies that sensing operation which minimizes this ambiguity. To do this, the system formulates object hypotheses and assesses its relative belief in those hypotheses to predict what features might be observed by a proposed sensing operation. Furthermore, since the number of sensing operations available to the system can be arbitrarily large, equivalent sensing operations are grouped together using a data structure that is based on the aspect graph. In order to measure the ambiguity in a set of hypotheses, the authors apply the concept of entropy from information theory. This allows them to determine the ambiguity in a hypothesis set in terms of the number of hypotheses and the system's distribution of belief among those hypotheses. >
---
paper_title: Recognizing large 3-D objects through next view planning using an uncalibrated camera

paper_content:

We present a new on-line scheme for the recognition and pose estimation of a large isolated 3-D object, which may not entirely fit in a camera's field of view. We do not assume any knowledge of the internal parameters of the camera, or their constancy. We use a probabilistic reasoning framework for recognition and next view planning. We show results of successful recognition and pose estimation even in cases of a high degree of interpretation ambiguity associated with the initial view.
---
paper_title: Transinformation for Active Object Recognition

paper_content:

This article develops an analogy between object recognition and the transmission of information through a channel based on the statistical representation of the appearances of 3D objects. This analogy provides a means to quantitatively evaluate the contribution of individual receptive field vectors, and to predict the performance of the object recognition process. Transinformation also provides a quantitative measure of the discrimination provided by each viewpoint, thus permitting the determination of the most discriminant viewpoints. As an application, the article develops an active object recognition algorithm which is able to resolve ambiguities inherent in a single-view recognition algorithm.
---
paper_title: Active Object Recognition Integrating Attention and Viewpoint Control

paper_content:

We present an active object recognition strategy which combines the use of an attention mechanism for focusing the search for a 3D object in a 2D image, with a viewpoint control strategy for disambiguating recovered object features. The attention mechanism consists of a probabilistic search through a hierarchy of predicted feature observations, taking objects into a set of regions classified according to the shapes of their bounding contours. We motivate the use of image regions as a focus-feature and compare their uncertainty in inferring objects with the uncertainty of more commonly used features such as lines or corners. If the features recovered during the attention phase do not provide a unique mapping to the 3D object being searched, the probabilistic feature hierarchy can be used to guide the camera to a new viewpoint from where the object can be disambiguated. The power of the underlying representation is its ability to unify these object recognition behaviors within a single framework. We present the approach in detail and evaluate its performance in the context of a project providing robotic aids for the disabled.
---
paper_title: Active recognition: using uncertainty to reduce ambiguity

paper_content:

Scene ambiguity, due to noisy measurements and uncertain object models, can be quantified and actively used by an autonomous agent to efficiently gather new data and improve its information about the environment. In this work an information-based utility measure is used to derive from a learned classification of shape models an efficient data collection strategy, specifically aimed at increasing classification confidence when recognizing uncertain shapes.
---
paper_title: Occlusions as a guide for planning the next view

paper_content:

A strategy for acquiring 3-D data of an unknown scene, using range images obtained by a light stripe range finder is addressed. The foci of attention are occluded regions, i.e., only the scene at the borders of the occlusions is modeled to compute the next move. Since the system has knowledge of the sensor geometry, it can resolve the appearance of occlusions by analyzing them. The problem of 3-D data acquisition is divided into two subproblems due to two types of occlusions. An occlusion arises either when the reflected laser light does not reach the camera or when the directed laser light does not reach the scene surface. After taking the range image of a scene, the regions of no data due to the first kind of occlusion are extracted. The missing data are acquired by rotating the sensor system in the scanning plane, which is defined by the first scan. After a complete image of the surface illuminated from the first scanning plane has been built, the regions of missing data due to the second kind of occlusions are located. Then, the directions of the next scanning planes for further 3-D data acquisition are computed. >
---
paper_title: A Best Next View selection algorithm incorporating a quality criterion

paper_content:

This paper presents a method for solving the Best Next View problem. This problem arises while gathering range data for the purpose of building 3D models of objects. The novelty of our solution is the introduction of a quality criterion in addition to the visibility criterion used by previous researchers. This quality criterion aims at obtaining views that improve the overall range data quality of the imaged surfaces. Results demonstrate that this method selects views which generate reasonable volumetric models for convex, concave and curved objects.
---
paper_title: A Two-Stage Algorithm for Planning the Next View From Range Images

paper_content:

A new technique is presented for determining the positions where a range sensor should be located to acquire the surfaces of a complex scene. The algorithm consists of two stages. The first stage applies a voting scheme that considers occlusion edges. Most of the surfaces of the scene are recovered through views computed in that way. Then, the second stage fills up remaining holes through a scheme based on visibility analysis. By leaving the more expensive visibility computations at the end of the exploration process, efficiency is increased.
---
paper_title: Recovering shape by purposive viewpoint adjustment

paper_content:

An approach for recovering surface shape from the occluding contour using an active (i.e., moving) observer is presented. It is based on a relationship between the geometries of a surface in a scene and its occluding contour: If the viewing direction of the observer is along a principal direction for a surface point whose projection is on the contour, surface shape (i.e., curvature) at the surface point can be recovered from the contour. An observer that purposefully changes viewpoint in order to achieve a well-defined geometric relationship with respect to a 3D shape prior to its recognition is used. It is shown that there is a simple and efficient viewing strategy that allows the observer to align the viewing direction with one of the two principal directions for a point on the surface. Experimental results demonstrate that the method can be easily implemented and can provide reliable shape information.<<ETX>>
---
paper_title: An Autonomous Active Vision System for Complete and Accurate 3D Scene Reconstruction

paper_content:

We propose in this paper an active vision approach for performing the 3D reconstruction of static scenes. The perception-action cycles are handled at various levels: from the definition of perception strategies for scene exploration downto the automatic generation of camera motions using visual servoing. To perform the reconstruction, we use a structure from controlled motion method which allows an optimal estimation of geometrical primitive parameters. As this method is based on particular camera motions, perceptual strategies able to appropriately perform a succession of such individual primitive reconstructions are proposed in order to recover the complete spatial structure of the scene. Two algorithms are proposed to ensure the exploration of the scene. The former is an incremental reconstruction algorithm based on the use of a prediction/verification scheme managed using decision theory and Bayes nets. It allows the visual system to get a high level description of the observed part of the scene. The latter, based on the computation of new viewpoints ensures the complete reconstruction of the scene. Experiments carried out on a robotic cell have demonstrated the validity of our approach.
---
paper_title: Using intermediate objects to improve the efficiency of visual search

paper_content:

When using a mobile camera to search for a target object, it is often important to maximize the efficiency of the search. We consider a method for increasing efficiency by searching only those subregions that are especially likely to contain the object. These subregions are identified via spatial relationships. Searches that use this method repeatedly find an “intermediate” object that commonly participates in a spatial relationship with the target object, and then look for the target in the restricted region specified by this relationship. Intuitively, such searches, calledindirect searches, seem likely to provide efficiency increases when the intermediate objects can be recognized at low resolutions and hence can be found with little extra overhead, and when they significantly restrict the area that must be searched for the target. But what is the magnitude of this increase, and upon what other factors does efficiency depend? Although the idea of exploiting spatial relationships has been used in vision systems before, few have quantitatively examined these questions.
---
paper_title: Control of selective perception using bayes nets and decision theory

paper_content:

A selective vision system sequentially collects evidence to support a specified hypothesis about a scene, as long as the additional evidence is worth the effort of obtaining it. Efficiency comes from processing the scene only where necessary, to the level of detail necessary, and with only the necessary operators. Knowledge representation and sequential decision-making are central issues for selective vision, which takes advantage of prior knowledge of a domain''s abstract and geometrical structure and models for the expected performance and cost of visual operators. .pp The TEA-1 selective vision system uses Bayes nets for representation and benefit-cost analysis for control of visual and non-visual actions. It is the high-level control for an active vision system, enabling purposive behavior, the use of qualitative vision modules and a pointable multiresolution sensor. TEA-1 demonstrates that Bayes nets and decision theoretic techniques provide a general, re-usable framework for constructing computer vision systems that are selective perception systems, and that Bayes nets provide a general framework for representing visual tasks. Control, or decision making, is the most important issue in a selective vision system. TEA-1''s decisions about what to do next are based on general hand-crafted ``goodness functions'''' constructed around core decision theoretic elements. Several goodness functions for different decisions are presented and evaluated. .pp The TEA-1 system solves a version of the T-world problem, an abstraction of a large set of domains and tasks. Some key factors that affect the success of selective perception are analyzed by examining how each factor affects the overall performance of TEA-1 when solving ensembles of randomly produced, simulated T-world domains and tasks. TEA-1''s decision making algorithms are also evaluated in this manner. Experiments in the lab for one specific T-world domain, table settings, are also presented.
---
paper_title: Dynamic Relevance: Vision-Based Focus of Attention using Artificial Neural Networks

paper_content:

Abstract This paper presents a method for ascertaining the relevance of inputs in vision-based tasks by exploiting temporal coherence and predictability. In contrast to the tasks explored in many previous relevance experiments, the class of tasks examined in this study is one in which relevance is a time-varying function of the previous and current inputs. The method proposed in this paper dynamically allocates relevance to inputs by using expectations of their future values. As a model of the task is learned, the model is simultaneously extended to create task-specific predictions of the future values of inputs. Inputs that are not relevant, and therefore not accounted for in the model, will not be predicted accurately. These inputs can be de-emphasized, and, in turn, a new, improved, model of the task created. The techniques presented in this paper have been successfully applied to the vision-based autonomous control of a land vehicle, vision-based hand tracking in cluttered scenes, and the detection of faults in the plasma-etch step of semiconductor wafers.
---
paper_title: Map Building for a Mobile Robot from Sensory Data

paper_content:

A method for building a three-dimensional (3-D) world model for a mobile robot from sensory data derived from outdoor scenes is presented. The 3-D world model consists of four kinds of maps: a physical sensor map, a virtual sensor map, a local map, and a global map. First, a range image (physical sensor map) is transformed to a height map (virtual sensor map) relative to the mobile robot. Next, the height map is segmented into unexplored, occluded, traversable and obstacle regions from the height information. Moreover, obstacle regions are classified into artificial objects or natural objects according to their geometrical properties such as slope and curvature. A drawback of the height map (recovery of planes vertical to the ground plane) is overcome by using multiple-height maps that include the maximum and minimum height for each point on the ground plane. Multiple-height maps are useful not only for finding vertical planes but also for mapping obstacle regions into video images for segmentation. Finally, the height maps are integrated into a local map by matching geometrical parameters and by updating region labels. The results obtained using landscape models and the autonomous land vehicle simulator of the University of Maryland are shown, and constructing a global map with local maps is discussed. >
---
paper_title: A comparison of position estimation techniques using occupancy grids

paper_content:

Abstract A mobile robot requires a perception of its local environment for both sensor-based locomotion and for position estimation. Occupancy grids, based on ultrasonic range data, provide a robust description of the local environment for locomotion. Unfortunately, current techniques for position estimation based on occupancy grids are both unreliable and computationally expensive. This paper reports on experiments with four techniques for position estimation using occupancy grids. A world modelling technique based on combining global and local occupancy grids is described. Techniques are described for extracting line segments from an occupancy grid based on a Hough transform. The use of an extended Kalman filter for position estimation is then adapted to this framework. Four matching techniques are presented for obtaining the innovation vector required by the Kalman filter equations. Experimental results show that matching of segments extracted from both the local and global occupancy grids gives results which are superior to a direct matching of grids, or to a mixed matching of segments to grids.
---
paper_title: Three-dimensional computer vision: a geometric viewpoint

paper_content:

Projective geometry modelling and calibrating cameras edge detection representing geometric primitives and their uncertainty stereo vision determining discrete motion from points and lines tracking tokens over time motion fields of curves interpolating and approximating three-dimensional data recognizing and locating objects and places answers to problems. Appendices: constrained optimization some results from algebraic geometry differential geometry.
---
paper_title: A Bayesian Approach to Landmark Discovery and Active Perception in Mobile Robot Navigation

paper_content:

To operate successfully in indoor environments, mobile robots must be able to localize themselves. Over the past few years, localization based on landmarks has become increasingly popular. Virtually all existing approaches to landmark-based navigation, however, rely on the human designer to decide what constitutes appropriate landmarks. This paper presents an approach that enables mobile robots to select their landmarks by themselves. Landmarks are chosen based on their utility for localization. This is done by training neural network landmark detectors so as to minimize the a posteriori localization error that the robot is expected to make after querying its sensors. An empirical study illustrates that self-selected landmarks are superior to landmarks carefully selected by a human. The Bayesian approach is also applied to control the direction of the robot’s camera, and empirical data demonstrates the appropriateness of this approach for active perception. The author is also affiliated with the Computer Science Department III of the University of Bonn, Germany, where part of this research was carried out. This research is sponsored in part by the National Science Foundation under award IRI-9313367, and by the Wright Laboratory, Aeronautical Systems Center, Air Force Materiel Command, USAF, and the Defense Advanced Research Projects Agency (DARPA) under grant number F33615-93-1-1330. The views and conclusions contained in this document are those of the author and should not be interpreted as necessarily representing official policies or endorsements, either expressed or implied, of NSF, Wright Laboratory or the United States Government.
---
paper_title: A Probabilistic Approach to Concurrent Mapping and Localization for Mobile Robots

paper_content:

This paper addresses the problem of building large-scale geometric maps of indoor environments with mobile robots. It poses the map building problem as a constrained, probabilistic maximum-likelihood estimation problem. It then devises a practical algorithm for generating the most likely map from data, along with the most likely path taken by the robot. Experimental results in cyclic environments of size up to 80 by 25 meter illustrate the appropriateness of the approach.
---
paper_title: Indoor scene terrain modeling using multiple range images for autonomous mobile robots

paper_content:

The authors consider the perception subsystem of an autonomous mobile robot which must be able to navigate in 3D terrain. They describe their approach to building a rough geometric model of a 3D terrain accounting for the locomotion capabilities of the vehicle, using a laser range finder. This model may be used as direct input for the robot's path planner. The terrain model relies on two grid-based representations: the local elevation map and the local navigation map. Both are incrementally built at arbitrary resolution using new interpolation and localization methods and other 3D vision techniques. The authors validate the proposed approach by presenting some comprehensive results using real range images of an indoor structured environment. >
---
paper_title: Generation of architectural CAD models using a mobile robot

paper_content:

This paper describes new algorithms for automatically constructing a computer aided design (CAD) model of a structured scene as imaged by a single camera on a mobile robot. The scene to be modeled is assumed to be composed mostly of linear edges with particular orientations in 3-D. This is the case for most indoor scenes as well as some outdoor urban scenes. The orientation data is used by a motion stereo algorithm to estimate the 3-D structure using a sequence of images. The algorithm assumes that the linear edges are the boundaries of opaque planar patches, such as the floor, the ceiling and the walls. The resulting 3-D description is a CAD model of the scene. Applications of this technique include CAD modeling for architecture and computer graphics, and robot navigation. This paper completes earlier publications and therefore concentrates on the latter parts of processing, including automatically tracking segments, and using the resulting models in different applications. >
---
paper_title: Learning metric-topological maps for indoor mobile robot navigation

paper_content:

Abstract Autonomous robots must be able to learn and maintain models of their environments. Research on mobile robot navigation has produced two major paradigms for mapping indoor environments: grid-based and topological. While grid-based methods produce accurate metric maps, their complexity often prohibits efficient planning and problem solving in large-scale indoor environments. Topological maps, on the other hand, can be used much more efficiently, yet accurate and consistent topological maps are often difficult to learn and maintain in large-scale environments, particularly if momentary sensor data is highly ambiguous. This paper describes an approach that integrates both paradigms: grid-based and topological. Grid-based maps are learned using artificial neural networks and naive Bayesian integration. Topological maps are generated on top of the grid-based maps, by partitioning the latter into coherent regions. By combining both paradigms, the approach presented here gains advantages from both worlds: accuracy/consistency and efficiency. The paper gives results for autonomous exploration, mapping and operation of a mobile robot in populated multi-room environments.
---
paper_title: Extraction and interpretation of semantically significant line segments for a mobile robot

paper_content:

The authors describe algorithms for detecting and interpreting linear features of a real scene as images by a single camera on a mobile robot. The low-level processing stages were specifically designed to increase the usefulness and the quality of the extracted features for a semantic interpretation. The detection and interpretation processes provided a 3-D orientation hypothesis for each 2-D segment. This, in turn, was used to estimate the robot's orientation and relative position in the environment and to delimit the free space visible in the image. The orientation data was used by a motion stereo algorithm to fully estimate the 3-D structure when a sequence of images becomes available. From detection to 3-D estimation, an emphasis was placed on real-world applications and very fast processing with conventional hardware. >
---
paper_title: Building visual maps by combining noisy stereo measurements

paper_content:

This paper deals with the problem of coping with noise disturbing data in Stereo, 3-D modelling and navigation. We introduce the idea of a Realistic Uncertain Description of the Environment (RUDE) which is local, i.e attached to a specific reference frame, and incorporates both, information about the geometry and about the parameters measuring this geometry. We also relate this uncertainty to the pixel uncertainty and show how the RUDE corresponding to different frames can be used to relate these frames by a rigid displacement which we describe both by a rotation and translation and a measure of their uncertainty. Finally, we use the relations between frames to update the associated RUDE and decrease their uncertainty.
---
paper_title: Position estimation for a mobile robot using vision and odometry

paper_content:

The authors describe a method for locating a mobile robot moving in a known environment. This technique combines position estimation from odometry with observations of the environment from a mobile camera. Fixed objects in the world provide landmarks which are listed in a database. The system calculates the angle to each landmark and then orients the camera. An extended Kalman filter is used to correct the error between the observed and estimated angle to each landmark. Results from experiments in a real environment are presented. >
---
paper_title: Dynamic Relevance: Vision-Based Focus of Attention using Artificial Neural Networks

paper_content:

Abstract This paper presents a method for ascertaining the relevance of inputs in vision-based tasks by exploiting temporal coherence and predictability. In contrast to the tasks explored in many previous relevance experiments, the class of tasks examined in this study is one in which relevance is a time-varying function of the previous and current inputs. The method proposed in this paper dynamically allocates relevance to inputs by using expectations of their future values. As a model of the task is learned, the model is simultaneously extended to create task-specific predictions of the future values of inputs. Inputs that are not relevant, and therefore not accounted for in the model, will not be predicted accurately. These inputs can be de-emphasized, and, in turn, a new, improved, model of the task created. The techniques presented in this paper have been successfully applied to the vision-based autonomous control of a land vehicle, vision-based hand tracking in cluttered scenes, and the detection of faults in the plasma-etch step of semiconductor wafers.
---
paper_title: Control of selective perception using bayes nets and decision theory

paper_content:

A selective vision system sequentially collects evidence to support a specified hypothesis about a scene, as long as the additional evidence is worth the effort of obtaining it. Efficiency comes from processing the scene only where necessary, to the level of detail necessary, and with only the necessary operators. Knowledge representation and sequential decision-making are central issues for selective vision, which takes advantage of prior knowledge of a domain''s abstract and geometrical structure and models for the expected performance and cost of visual operators. .pp The TEA-1 selective vision system uses Bayes nets for representation and benefit-cost analysis for control of visual and non-visual actions. It is the high-level control for an active vision system, enabling purposive behavior, the use of qualitative vision modules and a pointable multiresolution sensor. TEA-1 demonstrates that Bayes nets and decision theoretic techniques provide a general, re-usable framework for constructing computer vision systems that are selective perception systems, and that Bayes nets provide a general framework for representing visual tasks. Control, or decision making, is the most important issue in a selective vision system. TEA-1''s decisions about what to do next are based on general hand-crafted ``goodness functions'''' constructed around core decision theoretic elements. Several goodness functions for different decisions are presented and evaluated. .pp The TEA-1 system solves a version of the T-world problem, an abstraction of a large set of domains and tasks. Some key factors that affect the success of selective perception are analyzed by examining how each factor affects the overall performance of TEA-1 when solving ensembles of randomly produced, simulated T-world domains and tasks. TEA-1''s decision making algorithms are also evaluated in this manner. Experiments in the lab for one specific T-world domain, table settings, are also presented.
---
paper_title: An Autonomous Active Vision System for Complete and Accurate 3D Scene Reconstruction

paper_content:

We propose in this paper an active vision approach for performing the 3D reconstruction of static scenes. The perception-action cycles are handled at various levels: from the definition of perception strategies for scene exploration downto the automatic generation of camera motions using visual servoing. To perform the reconstruction, we use a structure from controlled motion method which allows an optimal estimation of geometrical primitive parameters. As this method is based on particular camera motions, perceptual strategies able to appropriately perform a succession of such individual primitive reconstructions are proposed in order to recover the complete spatial structure of the scene. Two algorithms are proposed to ensure the exploration of the scene. The former is an incremental reconstruction algorithm based on the use of a prediction/verification scheme managed using decision theory and Bayes nets. It allows the visual system to get a high level description of the observed part of the scene. The latter, based on the computation of new viewpoints ensures the complete reconstruction of the scene. Experiments carried out on a robotic cell have demonstrated the validity of our approach.
---
paper_title: A Bayesian Approach to Landmark Discovery and Active Perception in Mobile Robot Navigation

paper_content:

To operate successfully in indoor environments, mobile robots must be able to localize themselves. Over the past few years, localization based on landmarks has become increasingly popular. Virtually all existing approaches to landmark-based navigation, however, rely on the human designer to decide what constitutes appropriate landmarks. This paper presents an approach that enables mobile robots to select their landmarks by themselves. Landmarks are chosen based on their utility for localization. This is done by training neural network landmark detectors so as to minimize the a posteriori localization error that the robot is expected to make after querying its sensors. An empirical study illustrates that self-selected landmarks are superior to landmarks carefully selected by a human. The Bayesian approach is also applied to control the direction of the robot’s camera, and empirical data demonstrates the appropriateness of this approach for active perception. The author is also affiliated with the Computer Science Department III of the University of Bonn, Germany, where part of this research was carried out. This research is sponsored in part by the National Science Foundation under award IRI-9313367, and by the Wright Laboratory, Aeronautical Systems Center, Air Force Materiel Command, USAF, and the Defense Advanced Research Projects Agency (DARPA) under grant number F33615-93-1-1330. The views and conclusions contained in this document are those of the author and should not be interpreted as necessarily representing official policies or endorsements, either expressed or implied, of NSF, Wright Laboratory or the United States Government.
---
paper_title: Spatial Learning for Navigation in Dynamic Environments

paper_content:

This article describes techniques that have been developed for spatial learning in dynamic environments and a mobile robot system, ELDEN, that integrates these techniques for exploration and navigation. In this research, we introduce the concept of adaptive place networks, incrementally-constructed spatial representations that incorporate variable-confidence links to model uncertainty about topological adjacency. These networks guide the robot's navigation while constantly adapting to any topological changes that are encountered. ELDEN integrates these networks with a reactive controller that is robust to transient changes in the environment and a relocalization system that uses evidence grids to recalibrate dead reckoning.
---
paper_title: Indoor scene terrain modeling using multiple range images for autonomous mobile robots

paper_content:

The authors consider the perception subsystem of an autonomous mobile robot which must be able to navigate in 3D terrain. They describe their approach to building a rough geometric model of a 3D terrain accounting for the locomotion capabilities of the vehicle, using a laser range finder. This model may be used as direct input for the robot's path planner. The terrain model relies on two grid-based representations: the local elevation map and the local navigation map. Both are incrementally built at arbitrary resolution using new interpolation and localization methods and other 3D vision techniques. The authors validate the proposed approach by presenting some comprehensive results using real range images of an indoor structured environment. >
---
paper_title: Generation of architectural CAD models using a mobile robot

paper_content:

This paper describes new algorithms for automatically constructing a computer aided design (CAD) model of a structured scene as imaged by a single camera on a mobile robot. The scene to be modeled is assumed to be composed mostly of linear edges with particular orientations in 3-D. This is the case for most indoor scenes as well as some outdoor urban scenes. The orientation data is used by a motion stereo algorithm to estimate the 3-D structure using a sequence of images. The algorithm assumes that the linear edges are the boundaries of opaque planar patches, such as the floor, the ceiling and the walls. The resulting 3-D description is a CAD model of the scene. Applications of this technique include CAD modeling for architecture and computer graphics, and robot navigation. This paper completes earlier publications and therefore concentrates on the latter parts of processing, including automatically tracking segments, and using the resulting models in different applications. >
---
paper_title: Learning metric-topological maps for indoor mobile robot navigation

paper_content:

Abstract Autonomous robots must be able to learn and maintain models of their environments. Research on mobile robot navigation has produced two major paradigms for mapping indoor environments: grid-based and topological. While grid-based methods produce accurate metric maps, their complexity often prohibits efficient planning and problem solving in large-scale indoor environments. Topological maps, on the other hand, can be used much more efficiently, yet accurate and consistent topological maps are often difficult to learn and maintain in large-scale environments, particularly if momentary sensor data is highly ambiguous. This paper describes an approach that integrates both paradigms: grid-based and topological. Grid-based maps are learned using artificial neural networks and naive Bayesian integration. Topological maps are generated on top of the grid-based maps, by partitioning the latter into coherent regions. By combining both paradigms, the approach presented here gains advantages from both worlds: accuracy/consistency and efficiency. The paper gives results for autonomous exploration, mapping and operation of a mobile robot in populated multi-room environments.
---
paper_title: Extraction and interpretation of semantically significant line segments for a mobile robot

paper_content:

The authors describe algorithms for detecting and interpreting linear features of a real scene as images by a single camera on a mobile robot. The low-level processing stages were specifically designed to increase the usefulness and the quality of the extracted features for a semantic interpretation. The detection and interpretation processes provided a 3-D orientation hypothesis for each 2-D segment. This, in turn, was used to estimate the robot's orientation and relative position in the environment and to delimit the free space visible in the image. The orientation data was used by a motion stereo algorithm to fully estimate the 3-D structure when a sequence of images becomes available. From detection to 3-D estimation, an emphasis was placed on real-world applications and very fast processing with conventional hardware. >
---
paper_title: Localization and Homing using Combinations of Model Views

paper_content:

Abstract Navigation involves recognizing the environment, identifying the current position within the environment, and reaching particular positions. We present a method for localization (the act of recognizing the environment), positioning (the act of computing the exact coordinates of a robot in the environment), and homing (the act of returning to a previously visited position) from visual input. The method is based on representing the scene as a set of 2D views and predicting the appearances of novel views by linear combinations of the model views. The method accurately approximates the appearance of scenes under weak-perspective projection. Analysis of this projection as well as experimental results demonstrate that in many cases this approximation is sufficient to accurately describe the scene. When weak-perspective approximation is invalid, either a larger number of models can be acquired or an iterative solution to account for the perspective distortions can be employed. The method has several advantages over other approaches. It uses relatively rich representations; the representations are 2D rather than 3D; and localization can be done from only a single 2D view without calibration. The same principal method is applied for both the localization and positioning problems, and a simple “qualitative” algorithm for homing is derived from this method.
---
paper_title: Model-directed mobile robot navigation

paper_content:

The authors report on the system and methods used by UMass Mobile Robot Project. Model-based processing of the visual sensory data is the primary mechanism used for controlling movement of an autonomous land vehicle through the environment, measuring progress towards a given goal, and avoiding obstacles. Goal-oriented navigation takes place through a partially modeled, unchanging environment that contains no unmodeled obstacles; this simplified environment provides a foundation for research in more complicated domains. The navigation system integrates perception, planning, and execution of actions. Of particular importance is that the planning processes are reactive and reason about landmarks that should be perceived at various stages of task execution. Correspondence between image features and expected landmark locations are used at several abstraction levels to ensure proper plan execution. The system and some experiments that demonstrate the performance of its components is described. >
---
paper_title: Occlusions as a guide for planning the next view

paper_content:

A strategy for acquiring 3-D data of an unknown scene, using range images obtained by a light stripe range finder is addressed. The foci of attention are occluded regions, i.e., only the scene at the borders of the occlusions is modeled to compute the next move. Since the system has knowledge of the sensor geometry, it can resolve the appearance of occlusions by analyzing them. The problem of 3-D data acquisition is divided into two subproblems due to two types of occlusions. An occlusion arises either when the reflected laser light does not reach the camera or when the directed laser light does not reach the scene surface. After taking the range image of a scene, the regions of no data due to the first kind of occlusion are extracted. The missing data are acquired by rotating the sensor system in the scanning plane, which is defined by the first scan. After a complete image of the surface illuminated from the first scanning plane has been built, the regions of missing data due to the second kind of occlusions are located. Then, the directions of the next scanning planes for further 3-D data acquisition are computed. >
---
paper_title: Control of selective perception using bayes nets and decision theory

paper_content:

A selective vision system sequentially collects evidence to support a specified hypothesis about a scene, as long as the additional evidence is worth the effort of obtaining it. Efficiency comes from processing the scene only where necessary, to the level of detail necessary, and with only the necessary operators. Knowledge representation and sequential decision-making are central issues for selective vision, which takes advantage of prior knowledge of a domain''s abstract and geometrical structure and models for the expected performance and cost of visual operators. .pp The TEA-1 selective vision system uses Bayes nets for representation and benefit-cost analysis for control of visual and non-visual actions. It is the high-level control for an active vision system, enabling purposive behavior, the use of qualitative vision modules and a pointable multiresolution sensor. TEA-1 demonstrates that Bayes nets and decision theoretic techniques provide a general, re-usable framework for constructing computer vision systems that are selective perception systems, and that Bayes nets provide a general framework for representing visual tasks. Control, or decision making, is the most important issue in a selective vision system. TEA-1''s decisions about what to do next are based on general hand-crafted ``goodness functions'''' constructed around core decision theoretic elements. Several goodness functions for different decisions are presented and evaluated. .pp The TEA-1 system solves a version of the T-world problem, an abstraction of a large set of domains and tasks. Some key factors that affect the success of selective perception are analyzed by examining how each factor affects the overall performance of TEA-1 when solving ensembles of randomly produced, simulated T-world domains and tasks. TEA-1''s decision making algorithms are also evaluated in this manner. Experiments in the lab for one specific T-world domain, table settings, are also presented.
---
paper_title: An Autonomous Active Vision System for Complete and Accurate 3D Scene Reconstruction

paper_content:

We propose in this paper an active vision approach for performing the 3D reconstruction of static scenes. The perception-action cycles are handled at various levels: from the definition of perception strategies for scene exploration downto the automatic generation of camera motions using visual servoing. To perform the reconstruction, we use a structure from controlled motion method which allows an optimal estimation of geometrical primitive parameters. As this method is based on particular camera motions, perceptual strategies able to appropriately perform a succession of such individual primitive reconstructions are proposed in order to recover the complete spatial structure of the scene. Two algorithms are proposed to ensure the exploration of the scene. The former is an incremental reconstruction algorithm based on the use of a prediction/verification scheme managed using decision theory and Bayes nets. It allows the visual system to get a high level description of the observed part of the scene. The latter, based on the computation of new viewpoints ensures the complete reconstruction of the scene. Experiments carried out on a robotic cell have demonstrated the validity of our approach.
---
paper_title: A Bayesian Approach to Landmark Discovery and Active Perception in Mobile Robot Navigation

paper_content:

To operate successfully in indoor environments, mobile robots must be able to localize themselves. Over the past few years, localization based on landmarks has become increasingly popular. Virtually all existing approaches to landmark-based navigation, however, rely on the human designer to decide what constitutes appropriate landmarks. This paper presents an approach that enables mobile robots to select their landmarks by themselves. Landmarks are chosen based on their utility for localization. This is done by training neural network landmark detectors so as to minimize the a posteriori localization error that the robot is expected to make after querying its sensors. An empirical study illustrates that self-selected landmarks are superior to landmarks carefully selected by a human. The Bayesian approach is also applied to control the direction of the robot’s camera, and empirical data demonstrates the appropriateness of this approach for active perception. The author is also affiliated with the Computer Science Department III of the University of Bonn, Germany, where part of this research was carried out. This research is sponsored in part by the National Science Foundation under award IRI-9313367, and by the Wright Laboratory, Aeronautical Systems Center, Air Force Materiel Command, USAF, and the Defense Advanced Research Projects Agency (DARPA) under grant number F33615-93-1-1330. The views and conclusions contained in this document are those of the author and should not be interpreted as necessarily representing official policies or endorsements, either expressed or implied, of NSF, Wright Laboratory or the United States Government.
---
paper_title: A Probabilistic Approach to Concurrent Mapping and Localization for Mobile Robots

paper_content:

This paper addresses the problem of building large-scale geometric maps of indoor environments with mobile robots. It poses the map building problem as a constrained, probabilistic maximum-likelihood estimation problem. It then devises a practical algorithm for generating the most likely map from data, along with the most likely path taken by the robot. Experimental results in cyclic environments of size up to 80 by 25 meter illustrate the appropriateness of the approach.
---
paper_title: Recovering shape by purposive viewpoint adjustment

paper_content:

An approach for recovering surface shape from the occluding contour using an active (i.e., moving) observer is presented. It is based on a relationship between the geometries of a surface in a scene and its occluding contour: If the viewing direction of the observer is along a principal direction for a surface point whose projection is on the contour, surface shape (i.e., curvature) at the surface point can be recovered from the contour. An observer that purposefully changes viewpoint in order to achieve a well-defined geometric relationship with respect to a 3D shape prior to its recognition is used. It is shown that there is a simple and efficient viewing strategy that allows the observer to align the viewing direction with one of the two principal directions for a point on the surface. Experimental results demonstrate that the method can be easily implemented and can provide reliable shape information.<<ETX>>
---
paper_title: Using intermediate objects to improve the efficiency of visual search

paper_content:

When using a mobile camera to search for a target object, it is often important to maximize the efficiency of the search. We consider a method for increasing efficiency by searching only those subregions that are especially likely to contain the object. These subregions are identified via spatial relationships. Searches that use this method repeatedly find an “intermediate” object that commonly participates in a spatial relationship with the target object, and then look for the target in the restricted region specified by this relationship. Intuitively, such searches, calledindirect searches, seem likely to provide efficiency increases when the intermediate objects can be recognized at low resolutions and hence can be found with little extra overhead, and when they significantly restrict the area that must be searched for the target. But what is the magnitude of this increase, and upon what other factors does efficiency depend? Although the idea of exploiting spatial relationships has been used in vision systems before, few have quantitatively examined these questions.
---

Title: Active Recognition through Next View Planning: A Survey

Section 1: Introduction
Description 1: Write about the basics of 3-D object recognition, including the challenges in recognizing 3-D objects from 2-D intensity images.

Section 2: The Need for Multiple Views
Description 2: Discuss the necessity of using multiple views for 3-D object recognition, and the limitations of single-view recognition systems.

Section 3: Active Vision
Description 3: Define active sensors and discuss the concepts of active vision, including the ability to control sensor parameters.

Section 4: Object Feature Detection
Description 4: Explain the strategies for object feature detection and the planning for complete sensor coverage of 3-D objects.

Section 5: Object Recognition and Localization, and Scene Reconstruction
Description 5: Describe the fundamental problems in multiple view-based recognition systems and the classification of systems for object recognition and scene analysis.

Section 6: Active Object Recognition Systems
Description 6: Detail the use of multiple views in active object recognition systems and the various representation schemes used for these systems.

Section 7: Methods for Representing Uncertainty
Description 7: Discuss the common methods for representing uncertainty in object recognition tasks, including probability theory, Dempster-Shafer theory, and fuzzy logic.

Section 8: Recognition Strategies
Description 8: Present various recognition strategies for active 3-D object recognition schemes, focusing on next view planning strategies.

Section 9: Active Scene Analysis Systems
Description 9: Review different classes of active scene analysis systems and their information representation and control schemes.

Section 10: Conclusions
Description 10: Summarize the survey and analysis of different active 3-D object recognition and scene analysis systems, highlighting key observations and conclusions.