A Survey of Recent View-based 3D Model Retrieval Methods
---
paper_title: A video summarization tool using two-level redundancy detection for personal video recorders

paper_content:

Storage of video information is experiencing rapid growth as digital broadcasting services have developed. This has rapidly increased video data storage requirements in DVD and hard disc recorders in recent years. We design a video summarization tool for personal video recorders. This tool provides a video summarization function to grasp the original long video content quickly. This tool is with a two- level redundancy detection procedure. After the video parsing step, it removes redundant video content with the hierarchical agglomerative cluster method in the shot level. Impact factors of scenes and key frames are defined, and parts of shots are selected to generate the initial video summary. Finally a repetitive frame segment detection step is used to remove redundant information. We have developed a prototype model and tested this tool on TV series, movies and cartoons data. The experiment results are given to illustrate the proposed tool.
---
paper_title: A powerful relevance feedback mechanism for content-based 3D model retrieval

paper_content:

The technique of relevance feedback has been introduced to content-based 3D model retrieval, however, two essential issues which affect the retrieval performance have not been addressed. In this paper, a novel relevance feedback mechanism is presented, which effectively makes use of strengths of different feature vectors and perfectly solves the problem of small sample and asymmetry. During the retrieval process, the proposed method takes the user's feedback details as the relevant information of query model, and then dynamically updates two important parameters of each feature vector, narrowing the gap between high-level semantic knowledge and low-level object representation. The experiments, based on the publicly available 3D model database Princeton Shape Benchmark (PSB), show that the proposed approach not only precisely captures the user's semantic knowledge, but also significantly improves the retrieval performance of 3D model retrieval. Compared with three state-of-the-art query refinement schemes for 3D model retrieval, it provides superior retrieval effectiveness only with a few rounds of relevance feedback based on several standard measures.
---
paper_title: A new 3D model retrieval approach based on the elevation descriptor

paper_content:

The advances in 3D data acquisition techniques, graphics hardware, and 3D data modeling and visualizing techniques have led to the proliferation of 3D models. This has made the searching for specific 3D models a vital issue. Techniques for effective and efficient content-based retrieval of 3D models have therefore become an essential research topic. In this paper, a novel feature, called elevation descriptor, is proposed for 3D model retrieval. The elevation descriptor is invariant to translation and scaling of 3D models and it is robust for rotation. First, six elevations are obtained to describe the altitude information of a 3D model from six different views. Each elevation is represented by a gray-level image which is decomposed into several concentric circles. The elevation descriptor is obtained by taking the difference between the altitude sums of two successive concentric circles. An efficient similarity matching method is used to find the best match for an input model. Experimental results show that the proposed method is superior to other descriptors, including spherical harmonics, the MPEG-7 3D shape spectrum descriptor, and D2.
---
paper_title: Statistical modeling and many-to-many matching for view-based 3D object retrieval

paper_content:

We address the task of view-based 3D object retrieval, in which each object is represented by a set of views taken from different positions, rather than a geometrical model based on polygonal meshes. As the number of views and the view point setting cannot always be the same for different objects, the retrieval task is more challenging and the existing methods for 3D model retrieval are infeasible. In this paper, the information in the sets of views is exploited from two aspects. On the one hand, the form of histogram is converted from vector to state sequence, and Markov chain (MC) is utilized for modeling the statistical characteristics of all the views representing the same object. On the other hand, the earth mover's distance (EMD) is involved to achieve many-to-many matching between two sets of views. For 3D object retrieval, by combining the above two aspects together, a new distance measure is defined, and a novel approach to automatically determine the edge weights in graph-based semi-supervised learning is proposed. Experimental results on different databases demonstrate the effectiveness of our proposal.
---
paper_title: A 3D Shape Retrieval Framework Supporting Multimodal Queries

paper_content:

This paper presents a unified framework for 3D shape retrieval. The method supports multimodal queries (2D images, sketches, 3D objects) by introducing a novel view-based approach able to handle the different types of multimedia data. More specifically, a set of 2D images (multi-views) are automatically generated from a 3D object, by taking views from uniformly distributed viewpoints. For each image, a set of 2D rotation-invariant shape descriptors is produced. The global shape similarity between two 3D models is achieved by applying a novel matching scheme, which effectively combines the information extracted from the multi-view representation. The experimental results prove that the proposed method demonstrates superior performance over other well-known state-of-the-art approaches.
---
paper_title: Using shape distributions to compare solid models

paper_content:

Our recent work has described how to use feature and topology in-formation to compare 3-D solid models. In this work we describe a new method to compare solid models based on shape distributions. Shape distribution functions are common in the computer graphics and computer vision communities. The typical use of shape dis-tributions is to compare 2-D objects, such as those obtained from imaging devices (cameras and other computer vision equipment). Recent work has applied shape distribution metrics for compari-son of approximate models found in the graphics community, such as polygonal meshes, faceted representation, and Virtual Reality Modeling Language (VRML) models. This paper examines how to adapt these techniques to comparison of 3-D solid models, such as those produced by commercial CAD systems. We provide a brief review of shape matching with distribution functions and present an approach to matching solid models. First, we show how to ex-tend basic distribution-based techniques to handle CAD data that has been exported to VRML format. These extensions address specific geometries that occur in mechanical CAD data. Second, we describe how to use shape distributions to directly interrogate solid models. Lastly, we show how these techniques can be put together to provide a "query by example" interface to a large, het-erogeneous, CAD database: The National Design Repository. One significant contribution of our work is the systematic technique for performing consistent, engineering content-based comparisons of CAD models produced by different CAD systems.
---
paper_title: On Visual Similarity Based 3D Model Retrieval

paper_content:

A large number of 3D models are created and available on the Web, since more and more 3D modelling and digitizing tools are developed for ever increasing applications. The techniques for content-based 3D model retrieval then become necessary. In this paper, a visual similarity-based 3D model retrieval system is proposed. This approach measures the similarity among 3D models by visual similarity, and the main idea is that if two 3D models are similar, they also look similar from all viewing angles. Therefore, one hundred orthogonal projections of an object, excluding symmetry, are encoded both by Zernike moments and Fourier descriptors as features for later retrieval. The visual similarity-based approach is robust against similarity transformation, noise, model degeneracy etc., and provides 42%, 94% and 25% better performance (precision-recall evaluation diagram) than three other competing approaches: (1)the spherical harmonics approach developed by Funkhouser et al., (2)the MPEG-7 Shape 3D descriptors, and (3)the MPEG-7 Multiple View Descriptor. The proposed system is on the Web for practical trial use (http://3d.csie.ntu.edu.tw), and the database contains more than 10,000 publicly available 3D models collected from WWW pages. Furthermore, a user friendly interface is provided to retrieve 3D models by drawing 2D shapes. The retrieval is fast enough on a server with Pentium IV 2.4GHz CPU, and it takes about 2 seconds and 0.1 seconds for querying directly by a 3D model and by hand drawn 2D shapes, respectively.
---
paper_title: 3D object retrieval using the 3D shape impact descriptor

paper_content:

In this paper, the novel 3D shape impact descriptor is introduced, which is based on the resulting gravitational phenomena in the surrounding area of every 3D object. The 3D object is considered as a distributed 3D mass and the descriptor of the 3D object is indirectly computed from the resulting fields. The field is described using both Newton's and general relativity's laws. In the Newtonian approach, histograms of the field values in the surrounding area of the 3D object are computed, while in the relativistic approach the descriptors are histograms of the time-space curvature in the surrounding area of the 3D object. The basic motivation behind the proposed approach is the robustness with respect to object's degeneracies and the native invariance of the resulting descriptors under rotation and translation. Experiments which were performed in various 3D object databases proved that the proposed method can be efficiently used for 3D object retrieval applications.
---
paper_title: Polyhedral model retrieval using weighted point sets

paper_content:

Due to the recent improvements in laser scanning technology, 3D visualization and modeling, there is an increasing need for tools supporting the automatic search for 3D objects in archives. In this paper we describe a new geometric approach to 3D shape comparison and retrieval for arbitrary objects described by 3D polyhedral models that may contain gaps. In contrast with existing approaches, our approach takes the overall relative spatial location into account by representing the 3D shape as a weighted point set. To compare two objects geometrically, we generate for each object a weighted point set, which represents for each non-empty grid cell a salient point. We compare three methods to obtain in each grid cell a salient point and a weight: (1) choose the vertex in the cell with the highest Gaussian curvature, and choose as weight a measure for that curvature, (2) choose the area-weighted mean of the vertices in the cell, and choose as weight a measure denoting the normal variation of the facets in the cell and (3) choose the center of mass of all vertices in the cell, and choose as weight one. Finally, we compute the similarity between two shapes by comparing their weighted point sets using a new shape similarity measure based on weight transportation that is a variation of the Earth Mover's Distance. Unlike the Earth Mover's Distance, the new shape similarity measure satisfies the triangle inequality. This property makes it suitable for use in indexing schemes that depend on the triangle inequality, such as the one we introduce, based on so-called vantage objects. The strength of our approach is proven through experimental results using a database consisting of 133 models such as mugs, cars and boats, and a database consisting of 512 models, mostly air planes, classified into conventional air planes, delta-jets, multi-fuselages, biplanes, helicopters and other models. The results show that the retrieval performance is better than related shape matching methods.
---
paper_title: Using spin images for efficient object recognition in cluttered 3D scenes

paper_content:

We present a 3D shape-based object recognition system for simultaneous recognition of multiple objects in scenes containing clutter and occlusion. Recognition is based on matching surfaces by matching points using the spin image representation. The spin image is a data level shape descriptor that is used to match surfaces represented as surface meshes. We present a compression scheme for spin images that results in efficient multiple object recognition which we verify with results showing the simultaneous recognition of multiple objects from a library of 20 models. Furthermore, we demonstrate the robust performance of recognition in the presence of clutter and occlusion through analysis of recognition trials on 100 scenes.
---
paper_title: A Bayesian 3-D Search Engine Using Adaptive Views Clustering

paper_content:

In this paper, we propose a method for three-dimensional (3D)-model indexing based on two-dimensional (2D) views, which we call adaptive views clustering (AVC). The goal of this method is to provide an "optimal" selection of 2D views from a 3D model, and a probabilistic Bayesian method for 3D-model retrieval from these views. The characteristic view selection algorithm is based on an adaptive clustering algorithm and uses statistical model distribution scores to select the optimal number of views. Starting from the fact that all views do not have equal importance, we also introduce a novel Bayesian approach to improve the retrieval. Finally, we present our results and compare our method to some state-of-the-art 3D retrieval descriptors on the Princeton 3D Shape Benchmark database and a 3D-CAD-models database supplied by the car manufacturer Renault
---
paper_title: Weighted Subspace Distance and Its Applications to Object Recognition and Retrieval With Image Sets

paper_content:

We address the problem of measuring the distance between two subspaces, each of which is spanned by an image set. In the existing methods, only the orthonormal basis is used to represent the subspace. However, the images are usually distributed in a limited area, rather than the whole subspace. Therefore, the characteristics of the distribution should also be considered. In this letter, a weighted subspace distance (WSD) is proposed, in which the principal component values of the data set are adopted to calculate the weights. Experimental results on object recognition and retrieval with image sets demonstrate the effectiveness of our proposal.
---
paper_title: Description of shape information for 2-D and 3-D objects

paper_content:

The description of the spatial characteristics of two- and three-dimensional objects, in the framework of MPEG-7, is considered. The shape of an object is one of its fundamental properties, and this paper describes an efficient way to represent the coarse shape, scale and composition properties of an object. This representation is invariant to resolution, translation and rotation, and may be used for both two-dimensional (2-D) and three-dimensional (3-D) objects. This coarse shape descriptor will be included in the eXperimentation Model (XM) of MPEG-7. Applications of such a description to search object databases, in particular the CAESAR anthropometric database are discussed.
---
paper_title: PANORAMA: A 3D Shape Descriptor Based on Panoramic Views for Unsupervised 3D Object Retrieval

paper_content:

We present a novel 3D shape descriptor that uses a set of panoramic views of a 3D object which describe the position and orientation of the object's surface in 3D space. We obtain a panoramic view of a 3D object by projecting it to the lateral surface of a cylinder parallel to one of its three principal axes and centered at the centroid of the object. The object is projected to three perpendicular cylinders, each one aligned with one of its principal axes in order to capture the global shape of the object. For each projection we compute the corresponding 2D Discrete Fourier Transform as well as 2D Discrete Wavelet Transform. We further increase the retrieval performance by employing a local (unsupervised) relevance feedback technique that shifts the descriptor of an object closer to its cluster centroid in feature space. The effectiveness of the proposed 3D object retrieval methodology is demonstrated via an extensive consistent evaluation in standard benchmarks that clearly shows better performance against state-of-the-art 3D object retrieval methods.
---
paper_title: Visual Word Ambiguity

paper_content:

This paper studies automatic image classification by modeling soft assignment in the popular codebook model. The codebook model describes an image as a bag of discrete visual words selected from a vocabulary, where the frequency distributions of visual words in an image allow classification. One inherent component of the codebook model is the assignment of discrete visual words to continuous image features. Despite the clear mismatch of this hard assignment with the nature of continuous features, the approach has been successfully applied for some years. In this paper, we investigate four types of soft assignment of visual words to image features. We demonstrate that explicitly modeling visual word assignment ambiguity improves classification performance compared to the hard assignment of the traditional codebook model. The traditional codebook model is compared against our method for five well-known data sets: 15 natural scenes, Caltech-101, Caltech-256, and Pascal VOC 2007/2008. We demonstrate that large codebook vocabulary sizes completely deteriorate the performance of the traditional model, whereas the proposed model performs consistently. Moreover, we show that our method profits in high-dimensional feature spaces and reaps higher benefits when increasing the number of image categories.
---
paper_title: Learning with local and global consistency

paper_content:

We consider the general problem of learning from labeled and unlabeled data, which is often called semi-supervised learning or transductive inference. A principled approach to semi-supervised learning is to design a classifying function which is sufficiently smooth with respect to the intrinsic structure collectively revealed by known labeled and unlabeled points. We present a simple algorithm to obtain such a smooth solution. Our method yields encouraging experimental results on a number of classification problems and demonstrates effective use of unlabeled data.
---
paper_title: Adaptive Hypergraph Learning and its Application in Image Classification

paper_content:

Recent years have witnessed a surge of interest in graph-based transductive image classification. Existing simple graph-based transductive learning methods only model the pairwise relationship of images, however, and they are sensitive to the radius parameter used in similarity calculation. Hypergraph learning has been investigated to solve both difficulties. It models the high-order relationship of samples by using a hyperedge to link multiple samples. Nevertheless, the existing hypergraph learning methods face two problems, i.e., how to generate hyperedges and how to handle a large set of hyperedges. This paper proposes an adaptive hypergraph learning method for transductive image classification. In our method, we generate hyperedges by linking images and their nearest neighbors. By varying the size of the neighborhood, we are able to generate a set of hyperedges for each image and its visual neighbors. Our method simultaneously learns the labels of unlabeled images and the weights of hyperedges. In this way, we can automatically modulate the effects of different hyperedges. Thorough empirical studies show the effectiveness of our approach when compared with representative baselines.
---
paper_title: A new 3D model retrieval approach based on the elevation descriptor

paper_content:

The advances in 3D data acquisition techniques, graphics hardware, and 3D data modeling and visualizing techniques have led to the proliferation of 3D models. This has made the searching for specific 3D models a vital issue. Techniques for effective and efficient content-based retrieval of 3D models have therefore become an essential research topic. In this paper, a novel feature, called elevation descriptor, is proposed for 3D model retrieval. The elevation descriptor is invariant to translation and scaling of 3D models and it is robust for rotation. First, six elevations are obtained to describe the altitude information of a 3D model from six different views. Each elevation is represented by a gray-level image which is decomposed into several concentric circles. The elevation descriptor is obtained by taking the difference between the altitude sums of two successive concentric circles. An efficient similarity matching method is used to find the best match for an input model. Experimental results show that the proposed method is superior to other descriptors, including spherical harmonics, the MPEG-7 3D shape spectrum descriptor, and D2.
---
paper_title: 3-D Object Retrieval and Recognition With Hypergraph Analysis

paper_content:

View-based 3-D object retrieval and recognition has become popular in practice, e.g., in computer aided design. It is difficult to precisely estimate the distance between two objects represented by multiple views. Thus, current view-based 3-D object retrieval and recognition methods may not perform well. In this paper, we propose a hypergraph analysis approach to address this problem by avoiding the estimation of the distance between objects. In particular, we construct multiple hypergraphs for a set of 3-D objects based on their 2-D views. In these hypergraphs, each vertex is an object, and each edge is a cluster of views. Therefore, an edge connects multiple vertices. We define the weight of each edge based on the similarities between any two views within the cluster. Retrieval and recognition are performed based on the hypergraphs. Therefore, our method can explore the higher order relationship among objects and does not use the distance between objects. We conduct experiments on the National Taiwan University 3-D model dataset and the ETH 3-D object collection. Experimental results demonstrate the effectiveness of the proposed method by comparing with the state-of-the-art methods.
---
paper_title: Assistive tagging: A survey of multimedia tagging with human-computer joint exploration

paper_content:

Along with the explosive growth of multimedia data, automatic multimedia tagging has attracted great interest of various research communities, such as computer vision, multimedia, and information retrieval. However, despite the great progress achieved in the past two decades, automatic tagging technologies still can hardly achieve satisfactory performance on real-world multimedia data that vary widely in genre, quality, and content. Meanwhile, the power of human intelligence has been fully demonstrated in the Web 2.0 era. If well motivated, Internet users are able to tag a large amount of multimedia data. Therefore, a set of new techniques has been developed by combining humans and computers for more accurate and efficient multimedia tagging, such as batch tagging, active tagging, tag recommendation, and tag refinement. These techniques are able to accomplish multimedia tagging by jointly exploring humans and computers in different ways. This article refers to them collectively as assistive tagging and conducts a comprehensive survey of existing research efforts on this theme. We first introduce the status of automatic tagging and manual tagging and then state why assistive tagging can be a good solution. We categorize existing assistive tagging techniques into three paradigms: (1) tagging with data selection & organization; (2) tag recommendation; and (3) tag processing. We introduce the research efforts on each paradigm and summarize the methodologies. We also provide a discussion on several future trends in this research direction.
---
paper_title: ]Video object segmentation by hypergraph cut

paper_content:

In this paper, we present a new framework of video object segmentation, in which we formulate the task of extracting prominent objects from a scene as the problem of hypergraph cut. We initially over-segment each frame in the sequence, and take the over-segmented image patches as the vertices in the graph. Different from the traditional pairwise graph structure, we build a novel graph structure, hypergraph, to represent the complex spatio-temporal neighborhood relationship among the patches. We assign each patch with several attributes that are computed from the optical flow and the appearance-based motion profile, and the vertices with the same attribute value is connected by a hyperedge. Through all the hyperedges, not only the complex non-pairwise relationships between the patches are described, but also their merits are integrated together organically. The task of video object segmentation is equivalent to the hypergraph partition, which can be solved by the hypergraph cut algorithm. The effectiveness of the proposed method is demonstrated by extensive experiments on nature scenes.
---
paper_title: On Visual Similarity Based 3D Model Retrieval

paper_content:

A large number of 3D models are created and available on the Web, since more and more 3D modelling and digitizing tools are developed for ever increasing applications. The techniques for content-based 3D model retrieval then become necessary. In this paper, a visual similarity-based 3D model retrieval system is proposed. This approach measures the similarity among 3D models by visual similarity, and the main idea is that if two 3D models are similar, they also look similar from all viewing angles. Therefore, one hundred orthogonal projections of an object, excluding symmetry, are encoded both by Zernike moments and Fourier descriptors as features for later retrieval. The visual similarity-based approach is robust against similarity transformation, noise, model degeneracy etc., and provides 42%, 94% and 25% better performance (precision-recall evaluation diagram) than three other competing approaches: (1)the spherical harmonics approach developed by Funkhouser et al., (2)the MPEG-7 Shape 3D descriptors, and (3)the MPEG-7 Multiple View Descriptor. The proposed system is on the Web for practical trial use (http://3d.csie.ntu.edu.tw), and the database contains more than 10,000 publicly available 3D models collected from WWW pages. Furthermore, a user friendly interface is provided to retrieve 3D models by drawing 2D shapes. The retrieval is fast enough on a server with Pentium IV 2.4GHz CPU, and it takes about 2 seconds and 0.1 seconds for querying directly by a 3D model and by hand drawn 2D shapes, respectively.
---
paper_title: Beyond Distance Measurement: Constructing Neighborhood Similarity for Video Annotation

paper_content:

In the past few years, video annotation has benefited a lot from the progress of machine learning techniques. Recently, graph-based semi-supervised learning has gained much attention in this domain. However, as a crucial factor of these algorithms, the estimation of pairwise similarity has not been sufficiently studied. Generally, the similarity of two samples is estimated based on the Euclidean distance between them. But we will show that the similarity between two samples is not merely related to their distance but also related to the distribution of surrounding samples and labels. It is shown that the traditional distance-based similarity measure may lead to high classification error rates even on several simple datasets. To address this issue, we propose a novel neighborhood similarity measure, which explores the local sample and label distributions. We show that the neighborhood similarity between two samples simultaneously takes into account three characteristics: 1) their distance; 2) the distribution difference of the surrounding samples; and 3) the distribution difference of surrounding labels. Extensive experiments have demonstrated the superiority of neighborhood similarity over the existing distance-based similarity.
---
paper_title: Semi-supervised kernel density estimation for video annotation

paper_content:

Insufficiency of labeled training data is a major obstacle for automatic video annotation. Semi-supervised learning is an effective approach to this problem by leveraging a large amount of unlabeled data. However, existing semi-supervised learning algorithms have not demonstrated promising results in large-scale video annotation due to several difficulties, such as large variation of video content and intractable computational cost. In this paper, we propose a novel semi-supervised learning algorithm named semi-supervised kernel density estimation (SSKDE) which is developed based on kernel density estimation (KDE) approach. While only labeled data are utilized in classical KDE, in SSKDE both labeled and unlabeled data are leveraged to estimate class conditional probability densities based on an extended form of KDE. It is a non-parametric method, and it thus naturally avoids the model assumption problem that exists in many parametric semi-supervised methods. Meanwhile, it can be implemented with an efficient iterative solution process. So, this method is appropriate for video annotation. Furthermore, motivated by existing adaptive KDE approach, we propose an improved algorithm named semi-supervised adaptive kernel density estimation (SSAKDE). It employs local adaptive kernels rather than a fixed kernel, such that broader kernels can be applied in the regions with low density. In this way, more accurate density estimates can be obtained. Extensive experiments have demonstrated the effectiveness of the proposed methods.
---
paper_title: Tag-based social image search with visual-text joint hypergraph learning

paper_content:

Tag-based social image search has attracted great interest and how to order the search results based on relevance level is a research problem. Visual content of images and tags have both been investigated. However, existing methods usually employ tags and visual content separately or sequentially to learn the image relevance. This paper proposes a tag-based image search with visual-text joint hypergraph learning. We simultaneously investigate the bag-of-words and bag-of-visual-words representations of images and accomplish the relevance estimation with a hypergraph learning approach. Each textual or visual word generates a hyperedge in the constructed hypergraph. We conduct experiments with a real-world data set and experimental results demonstrate the effectiveness of our approach.
---
paper_title: 3D Object Recognition Using Hyper-Graphs and Ranked Local Invariant Features

paper_content:

Local invariant feature-based methods such as SIFT have been proven highly effective for object recognition. However, they have made either relatively little use or too complex use of geometric constraints and are confounded when the detected features are superabundant. Here we make two contributions aimed at overcoming these problems. First, we rank the SIFT points (R-SIFT) using visual saliency. Second, we use the reduced set of R-SIFT features to construct a class specific hyper graph (CSHG) which comprehensively utilizes local SIFT and global geometric constraints. Moreover, it efficiently captures multiple object appearance instances. We show how the CSHG can be learned from example images for objects of a particular class. Experiments reveal that the method gives excellent recognition performance, with a low false-positive rate.
---
paper_title: Learning with hypergraphs: Clustering, classification, and embedding

paper_content:

We usually endow the investigated objects with pairwise relationships, which can be illustrated as graphs. In many real-world problems, however, relationships among the objects of our interest are more complex than pair-wise. Naively squeezing the complex relationships into pairwise ones will inevitably lead to loss of information which can be expected valuable for our learning tasks however. Therefore we consider using hypergraphs instead to completely represent complex relationships among the objects of our interest, and thus the problem of learning with hypergraphs arises. Our main contribution in this paper is to generalize the powerful methodology of spectral clustering which originally operates on undirected graphs to hypergraphs, and further develop algorithms for hypergraph embedding and transductive classification on the basis of the spectral hypergraph clustering approach. Our experiments on a number of benchmarks showed the advantages of hypergraphs over usual graphs.
---
paper_title: Correlative Linear Neighborhood Propagation for Video Annotation

paper_content:

Recently, graph-based semi-supervised learning methods have been widely applied in multimedia research area. However, for the application of video semantic annotation in multi-label setting, these methods neglect an important characteristic of video data: The semantic concepts appear correlatively and interact naturally with each other rather than exist in isolation. In this paper, we adapt this semantic correlation into graph-based semi-supervised learning and propose a novel method named correlative linear neighborhood propagation to improve annotation performance. Experiments conducted on the Text REtrieval Conference VIDeo retrieval evaluation data set have demonstrated its effectiveness and efficiency.
---
paper_title: The Earth Mover's Distance as a Metric for Image Retrieval

paper_content:

We investigate the properties of a metric between two distributions, the Earth Mover's Distance (EMD), for content-based image retrieval. The EMD is based on the minimal cost that must be paid to transform one distribution into the other, in a precise sense, and was first proposed for certain vision problems by Peleg, Werman, and Rom. For image retrieval, we combine this idea with a representation scheme for distributions that is based on vector quantization. This combination leads to an image comparison framework that often accounts for perceptual similarity better than other previously proposed methods. The EMD is based on a solution to the transportation problem from linear optimization, for which efficient algorithms are available, and also allows naturally for partial matching. It is more robust than histogram matching techniques, in that it can operate on variable-length representations of the distributions that avoid quantization and other binning problems typical of histograms. When used to compare distributions with the same overall mass, the EMD is a true metric. In this paper we focus on applications to color and texture, and we compare the retrieval performance of the EMD with that of other distances.
---
paper_title: 3D model retrieval using weighted bipartite graph matching

paper_content:

In this paper, we propose a view-based 3D model retrieval algorithm, where many-to-many matching method, weighted bipartite graph matching, is employed for comparison between two 3D models. In this work, each 3D model is represented by a set of 2D views. Representative views are first selected from the query model and the corresponding initial weights are provided. These initial weights are further updated based on the relationship among these representative views. The weighted bipartite graph is built with these selected 2D views, and the matching result is used to measure the similarity between two 3D models. Experimental results and comparison with existing methods show the effectiveness of the proposed algorithm.
---
paper_title: Less is More: Efficient 3-D Object Retrieval With Query View Selection

paper_content:

The explosively increasing 3-D objects make their efficient retrieval technology highly desired. Extensive research efforts have been dedicated to view-based 3-D object retrieval for its advantage of using 2-D views to represent 3-D objects. In this paradigm, typically the retrieval is accomplished by matching the views of the query object with the objects in database. However, using all the query views may not only introduce difficulty in rapid retrieval but also degrade retrieval accuracy when there is a mismatch between the query views and the object views in the database. In this work, we propose an interactive 3-D object retrieval scheme. Given a set of query views, we first perform clustering to obtain several candidates. We then incrementally select query views for object matching: in each round of relevance feedback, we only add the query view that is judged to be the most informative one based on the labeling information. In addition, we also propose an efficient approach to learn a distance metric for the newly selected query view and the weights for combining all of the selected query views. We conduct experiments on the National Taiwan University 3D Model database, ETH 3D object collection, and Shape Retrieval Content of Non-Rigid 3D Model, and results demonstrated that our approach not only significantly speeds up the retrieval process but also achieves encouraging retrieval performance.
---
paper_title: 3D models retrieval by using characteristic views

paper_content:

In this work we introduce a new method for indexing 3D models. This method is based on the characterization of 3D objects by a set of 7 characteristic views, including three principals, and four secondaries. The primary, secondary, and tertiary viewing directions are determined by the eigenvector analysis of the covariance matrix related to the 3D object. The secondary views are deduced from the principal views. We propose an index based on "curvature scale space", organized around a tree structure, named M-Tree, which is parameterized by a distance function and allows one to considerably decrease the calculating time by saving the intermediate distances.
---
paper_title: Clip based video summarization and ranking

paper_content:

In this paper, we present a new algorithm for video clip summarization and ranking, which is mainly based on a clip based video similarity measure and the affinity propagation clustering (AP) algorithm. We propose a proportional max-weighted bipartite matching algorithm for clip similarity measure. This method first generates a basic frame set and a corresponding proportion value set from each clip. Then it models two clips as a weighted bipartite graph, where the weight values are determined by both the direct frame similarities and the proportion values. Then the max-weighted bipartite matching is employed to measure the similarity between two clips. This method achieves good retrieval performance when the length of two clips varies greatly. With these clip similarities, clips are clustered using affinity propagation. The clips in one cluster generally describe the same video event. Video ranking is based on the cluster size and the average information entropy of each event. Experimental results are given to illustrate the proposed algorithm.
---
paper_title: Context-Aware Semi-Local Feature Detector

paper_content:

How can interest point detectors benefit from contextual cues? In this articles, we introduce a context-aware semi-local detector (CASL) framework to give a systematic answer with three contributions: (1) We integrate the context of interest points to recurrently refine their detections. (2) This integration boosts interest point detectors from the traditionally local scale to a semi-local scale to discover more discriminative salient regions. (3) Such context-aware structure further enables us to bring forward category learning (usually in the subsequent recognition phase) into interest point detection to locate category-aware, meaningful salient regions. Our CASL detector consists of two phases. The first phase accumulates multiscale spatial correlations of local features into a difference of contextual Gaussians (DoCG) field. DoCG quantizes detector context to highlight contextually salient regions at a semi-local scale, which also reveals visual attentions to a certain extent. The second phase locates contextual peaks by mean shift search over the DoCG field, which subsequently integrates contextual cues into feature description. This phase enables us to integrate category learning into mean shift search kernels. This learning-based CASL mechanism produces more category-aware features, which substantially benefits the subsequent visual categorization process. We conducted experiments in image search, object characterization, and feature detector repeatability evaluations, which reported superior discriminability and comparable repeatability to state-of-the-art works.
---
paper_title: Representative views re-ranking for 3D model retrieval with multi-bipartite graph reinforcement model

paper_content:

In this paper, we propose a multi-bipartite graph reinforcement model for representative views re-ranking in 3D model retrieval. Given the views of one query 3D model, all query views are grouped into clusters to generate representative views and corresponding original weights. In the retrieval procedure, labeled positive retrieval results are employed to refine the query information. Each group of views from positive retrieval results and the group of representative query views are employed to construct a bipartite graph, and a multi-bipartite graph reinforcement algorithm is performed on these bipartite graphs to re-rank all views. Then the weights of all representative query views are updated. Experimental results on two 3D model databases are provided to justify the effectiveness of the proposed method.
---
paper_title: Image retrieval via probabilistic hypergraph ranking

paper_content:

In this paper, we propose a new transductive learning framework for image retrieval, in which images are taken as vertices in a weighted hypergraph and the task of image search is formulated as the problem of hypergraph ranking. Based on the similarity matrix computed from various feature descriptors, we take each image as a ‘centroid’ vertex and form a hyperedge by a centroid and its k-nearest neighbors. To further exploit the correlation information among images, we propose a probabilistic hypergraph, which assigns each vertex v i to a hyperedge e j in a probabilistic way. In the incidence structure of a probabilistic hypergraph, we describe both the higher order grouping information and the affinity relationship between vertices within each hy-peredge. After feedback images are provided, our retrieval system ranks image labels by a transductive inference approach, which tends to assign the same label to vertices that share many incidental hyperedges, with the constraints that predicted labels of feedback images should be similar to their initial labels. We compare the proposed method to several other methods and its effectiveness is demonstrated by extensive experiments on Corel5K, the Scene dataset and Caltech 101.
---
paper_title: View-based 3D object retrieval by bipartite graph matching

paper_content:

Bipartite graph matching has been investigated in multiple view matching for 3D object retrieval. However, existing methods employ one-to-one vertex matching scheme while more than two views may share close semantic meanings in practice. In this work, we propose a bipartite graph matching method to measure the distance between two objects based on multiple views. In the proposed method, representative views are first selected by using view clustering for each object, and the corresponding weights are given based on the cluster results. A bipartite graph is constructed by using the two groups of representative views from two compared objects. To calculate the similarity between two objects, the bipartite graph is first partitioned to several subsets, and the views in the same sub-set are with high possibility to be with similar semantic meanings. The distances between two objects within individual subsets are then assembled through the graph to obtain the final similarity. Experimental results and comparison with the state-of-the-art methods demonstrate the effectiveness of the proposed algorithm.
---
paper_title: Visual-Textual Joint Relevance Learning for Tag-Based Social Image Search

paper_content:

Due to the popularity of social media websites, extensive research efforts have been dedicated to tag-based social image search. Both visual information and tags have been investigated in the research field. However, most existing methods use tags and visual characteristics either separately or sequentially in order to estimate the relevance of images. In this paper, we propose an approach that simultaneously utilizes both visual and textual information to estimate the relevance of user tagged images. The relevance estimation is determined with a hypergraph learning approach. In this method, a social image hypergraph is constructed, where vertices represent images and hyperedges represent visual or textual terms. Learning is achieved with use of a set of pseudo-positive images, where the weights of hyperedges are updated throughout the learning process. In this way, the impact of different tags and visual words can be automatically modulated. Comparative results of the experiments conducted on a dataset including 370+images are presented, which demonstrate the effectiveness of the proposed approach.
---
paper_title: Learning to Distribute Vocabulary Indexing for Scalable Visual Search

paper_content:

In recent years, there is an ever-increasing research focus on Bag-of-Words based near duplicate visual search paradigm with inverted indexing. One fundamental yet unexploited challenge is how to maintain the large indexing structures within a single server subject to its memory constraint, which is extremely hard to scale up to millions or even billions of images. In this paper, we propose to parallelize the near duplicate visual search architecture to index millions of images over multiple servers, including the distribution of both visual vocabulary and the corresponding indexing structure. We optimize the distribution of vocabulary indexing from a machine learning perspective, which provides a “memory light” search paradigm that leverages the computational power across multiple servers to reduce the search latency. Especially, our solution addresses two essential issues: “What to distribute” and “How to distribute”. “What to distribute” is addressed by a “lossy” vocabulary Boosting, which discards both frequent and indiscriminating words prior to distribution. “How to distribute” is addressed by learning an optimal distribution function, which maximizes the uniformity of assigning the words of a given query to multiple servers. We validate the distributed vocabulary indexing scheme in a real world location search system over 10 million landmark images. Comparing to the state-of-the-art alternatives of single-server search [5], [6], [16] and distributed search [23], our scheme has yielded a significant gain of about 200% speedup at comparable precision by distributing only 5% words. We also report excellent robustness even when partial servers crash.
---
paper_title: Recognition and shape synthesis of 3-D objects based on attributed hypergraphs

paper_content:

A computer vision system is presented for shape synthesis and recognition of three-dimensional objects using an attributed hypergraph representation. The vision system is capable of: (1) constructing an attributed hypergraph representation (AHR) based on the information extracted from an image with range data; (2) synthesizing several AHRs obtained from various views of an object to form a complete AHR of the object; and (3) recognizing any view of an object of finding the graph monomorphism between the AHR of that view and the complete AHR of a prototype object. This system is implemented on a Grinnell imaging system driven by a VAX 11/750 running VMS. >
---
paper_title: Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model

paper_content:

This paper describes a 3D shape model retrieval method that accepts, as a query, a 3D mesh obtained by a range scan from a viewpoint. The proposed method visually compares single depth map of the query with depth maps of a 3D model rendered from multiple viewpoints. Comparison of the depth maps employs bag-of local visual features extracted by using a modified version of Lowe's Scale-Invariant Feature Transform (SIFT). The method is capable of retrieving 3D models having diverse shape representations and is robust against articulation and global deformation of 3D shapes thanks to location-free integration of local visual features. Two modifications to the SIFT are made to avoid ill effects of range scanning artifacts, such as jagged edges and cracks, that exist in the query mesh. The two modifications are; (1) dense and random feature placement, and (2) importance sampling of low-frequency images in the SIFT's Gaussian image pyramid. Our experimental evaluation showed that the proposed method significantly outperforms previous methods.
---
paper_title: Efficient manifold learning for 3D model retrieval by using clustering-based training sample reduction

paper_content:

Retrieval accuracy in content-based multimedia retrieval can be improved by using distance metric learned from distribution of features in input feature space. One way to achieve this is by dimension reduction via manifold-learning, such as Locally Linear Embedding [8]. While effective in improving retrieval accuracy, these algorithms have high computational cost that depends on feature dimensionality d and number of training samples N. In this paper, we explore a clustering-based approach to reduce number of training samples; it uses L cluster centers (L<<N) computed from N input features as training samples. We propose to use extremely randomized clustering tree [3] for clustering. Experiments showed that the proposed approach produces better retrieval performance than random sampling, and that the randomized tree is much faster than the k-means algorithm.
---

Title: A Survey of Recent View-based 3D Model Retrieval Methods

Section 1: Introduction
Description 1: Introduce the overall topic of 3D model retrieval methods, dividing them into model-based and view-based methods, and explain the advantages of view-based methods. Also, provide an overview of what this survey will cover.

Section 2: Multiple View Matching
Description 2: Discuss the various methods used for matching multiple views in view-based 3D model retrieval, including early work and recent advancements. Highlight notable methods like Light Field Descriptor (LFD), bipartite graph matching, and probabilistic frameworks.

Section 3: Application of the Bag-of-Words Method in 3D Model Retrieval
Description 3: Review how the bag-of-visual-words method is applied to view-based 3D model retrieval. Include discussions on key techniques, such as SIFT feature extraction, manifold learning, and dense sampling.

Section 4: Discussion and Future Work
Description 4: Summarize the findings from the surveyed methods and discuss open issues and potential future research directions in the field of view-based 3D model retrieval, particularly in view representation and relationship estimation among 3D models.