A review of learning vector quantization classifiers
---
paper_title: A novel kernel prototype-based learning algorithm

paper_content:

We propose a novel kernel prototype-based learning algorithm, called kernel generalized learning vector quantization (KGLYQ) algorithm, which can significantly improve the classification performance of the original generalized learning vector quantization algorithm in complex pattern classification tasks. In addition, the KGLVQ can also serve as a good general kernel learning framework for further investigation.
---
paper_title: Classification of respiratory sounds based on wavelet packet decomposition and learning vector quantization

paper_content:

In this paper, a wavelet packet-based method is used for detection of abnormal respiratory sounds. The sound signal is divided into segments, and a feature vector for classification is formed using the results of the search for the best wavelet packet decomposition. The segments are classified as containing crackles, wheezes or normal lung sounds, using Learning Vector Quantization. The method is tested using a small set of real patient data which was also analysed by an expert observer. The preliminary results are promising, although not yet good enough for clinical use.
---
paper_title: Evaluation of learning vector quantization to classify cotton trash

paper_content:

The cotton industry needs a method to identify the type of trash (nonlint material (NLM)) in cotton samples; learning vector quanti- zation (LVQ) is evaluated as that method. LVQ is a classification tech- nique that defines reference vectors (group prototypes) in an N-dimensional feature space (R N ). Normalized trash object features ex- tracted from images of compressed cotton samples defineR N.A n un- known NLM object is given the label of the closest reference vector (as defined by Euclidean distance). Different normalized feature spaces and NLM classifications are evaluated and accuracies reported for correctly identifying the NLM type. LVQ is used to partition cotton trash into: (1) bark (B), leaf (L), pepper (P), or stick (S); (2) bark and nonbark (N); or (3) bark, combined leaf and pepper (LP), or stick. Percentage accuracy for correctly identifying 139 pieces of test trash placed on laboratory pre- pared samples for the three scenarios are (B:95, L:87, P:100, S:88), (B:100, N:97), and (B:95, LP:99, S:88), respectively. Also, LVQ results are compared to previous work using backpropagating neural networks. © 1997 Society of Photo-Optical Instrumentation Engineers.
---
paper_title: Kernel robust soft learning vector quantization

paper_content:

Prototype-based classification schemes offer very intuitive and flexible classifiers with the benefit of easy interpretability of the results and scalability of the model complexity. Recent prototype-based models such as robust soft learning vector quantization (RSLVQ) have the benefit of a solid mathematical foundation of the learning rule and decision boundaries in terms of probabilistic models and corresponding likelihood optimization. In its original form, they can be used for standard Euclidean vectors only. In this contribution, we extend RSLVQ towards a kernelized version which can be used for any positive semidefinite data matrix. We demonstrate the superior performance of the technique, kernel RSLVQ, in a variety of benchmarks where results competitive or even superior to state-of-the-art support vector machines are obtained.
---
paper_title: A supervised growing neural gas algorithm for cluster analysis

paper_content:

In this paper, a prototype-based supervised clustering algorithm is proposed. The proposed algorithm, called the Supervised Growing Neural Gas algorithm (SGNG), incorporates several techniques from some unsupervised GNG algorithms such as the adaptive learning rates and the cluster repulsion mechanisms of the Robust Growing Neural Gas algorithm, and the Type Two Learning Vector Quantization (LVQ2) technique. Furthermore, a new prototype insertion mechanism and a clustering validity index are proposed. These techniques are designed to utilize class labels of the training data to guide the clustering. The SGNG algorithm is capable of clustering adjacent regions of data objects labeled with different classes, formulating topological relationships among prototypes and automatically determining the optimal number of clusters using the proposed validity index. To evaluate the effectiveness of the SGNG algorithm, two experiments are conducted. The first experiment uses two synthetic data sets to graphically illustrate the potential with respect to growing ability, ability to cluster adjacent regions of different classes, and ability to determine the optimal number of prototypes. The second experiment evaluates the effectiveness using the UCI benchmark data sets. The results from the second experiment show that the SGNG algorithm performs better than other supervised clustering algorithms for both cluster impurities and total running times.
---
paper_title: A new generalized LVQ algorithm via harmonic to minimum distance measure transition

paper_content:

We present a novel generalized learning vector quantization (LVQ) framework called the harmonic to minimum generalized LVQ algorithm (H2M-GLVQ). Through incorporating the distance measure transition procedure from harmonic average distance to minimum distance, the H2M-GLVQ cost function is gradually changing from the soft model to the hard model. Our proposed method, at the early training stage, can effectively tackle the initialization sensitivity problem associated with the original generalized LVQ algorithm while the convergence of the algorithm can be ensured by the hard model in the later training stage. Experimental results have shown the superior performance of the H2M-GLVQ algorithm over the generalized LVQ and one of its variants on some artificial multi-modal datasets
---
paper_title: Entropy-constrained learning vector quantization algorithms and their application in image compression

paper_content:

This paper presents entropy-constrained learning vector quantization (ECLVQ) algorithms and their application in image compression. The development of these algorithms relies on reformulation, which is a powerful new methodology that essentially establishes a link between learning vector quantization and clustering algorithms developed using alternating optimization. ECLVQ algorithms are developed in this paper by reformulating entropyconstrained fuzzy clustering (ECFC) algorithms, which were developed by minimizing an objective function incorporating the partition entropy and the average distortion between the feature vectors and their prototypes. The proposed algorithms allow the gradual transition from a maximally fuzzy partition to a nearly crisp partition of the feature vectors during the learning process. This paper presents two alternative implementations of the proposed algorithms, which differ in terms of the strategy employed for updating the prototypes during learning. The proposed algorithms are tested and evaluated on the design of codebooks used for image data compression.
---
paper_title: An Axiomatic Approach to Soft Learning Vector Quantization and Clustering

paper_content:

This paper presents an axiomatic approach to soft learning vector quantization (LVQ) and clustering based on reformulation. The reformulation of the fuzzy c-means (FCM) algorithm provides the basis for reformulating entropy-constrained fuzzy clustering (ECFC) algorithms. According to the proposed approach, the development of specific algorithms reduces to the selection of a generator function. Linear generator functions lead to the FCM and fuzzy learning vector quantization algorithms while exponential generator functions lead to ECFC and entropy-constrained learning vector quantization algorithms. The reformulation of LVQ and clustering algorithms also provides the basis for developing uncertainty measures that can identify feature vectors equidistant from all prototypes. These measures are employed by a procedure developed to make soft LVQ and clustering algorithms capable of identifying outliers in the data set. This procedure is evaluated by testing the algorithms generated by linear and exponential generator functions on speech data.
---
paper_title: Soft Learning Vector Quantization

paper_content:

Learning vector quantization (LVQ) is a popular class of adaptive nearest prototype classifiers for multiclass classification, but learning algorithms from this family have so far been proposed on heuristic grounds. Here, we take a more principled approach and derive two variants of LVQ using a gaussian mixture ansatz. We propose an objective function based on a likelihood ratio and derive a learning rule using gradient descent. The new approach provides a way to extend the algorithms of the LVQ family to different distance measure and allows for the design of "soft" LVQ algorithms. Benchmark results show that the new methods lead to better classification performance than LVQ 2.1. An additional benefit of the new method is that model assumptions are made explicit, so that the method can be adapted more easily to different kinds of problems.
---
paper_title: Generalized clustering networks and Kohonen's self-organizing scheme

paper_content:

The relationship between the sequential hard c-means (SHCM) and learning vector quantization (LVQ) clustering algorithms is discussed. The impact and interaction of these two families of methods with Kohonen's self-organizing feature mapping (SOFM), which is not a clustering method but often lends ideas to clustering algorithms, are considered. A generalization of LVQ that updates all nodes for a given input vector is proposed. The network attempts to find a minimum of a well-defined objective function. The learning rules depend on the degree of distance match to the winner node; the lesser the degree of match with the winner, the greater the impact on nonwinner nodes. Numerical results indicate that the terminal prototypes generated by this modification of LVQ are generally insensitive to initialization and independent of any choice of learning coefficient. IRIS data obtained by E. Anderson's (1939) is used to illustrate the proposed method. Results are compared with the standard LVQ approach. >
---
paper_title: Relevance LVQ versus SVM

paper_content:

The support vector machine (SVM) constitutes one of the most successful current learning algorithms with excellent classification accuracy in large real-life problems and strong theoretical background. However, a SVM solution is given by a not intuitive classification in terms of extreme values of the training set and the size of a SVM classifier scales with the number of training data. Generalized relevance learning vector quantization (GRLVQ) has recently been introduced as a simple though powerful expansion of basic LVQ. Unlike SVM, it provides a very intuitive classification in terms of prototypical vectors the number of which is independent of the size of the training set. Here, we discuss GRLVQ in comparison to the SVM and point out its beneficial theoretical properties which are similar to SVM whereby providing sparse and intuitive solutions. In addition, the competitive performance of GRLVQ is demonstrated in one experiment from computational biology.
---
paper_title: Identification of ECG beats from cross-spectrum information aided learning vector quantization

paper_content:

Abstract This work describes the development of a computerized medical diagnostic tool for heart beat categorization. The main objective is to achieve an accurate, timely detection of cardiac arrhythmia for providing appropriate medical attention to a patient. The proposed scheme employs a feature extractor coupled with an Artificial Neural Network (ANN) classifier. The feature extractor is based on cross-correlation approach, utilizing the cross-spectral density information in frequency domain. The ANN classifier uses a Learning Vector Quantization (LVQ) scheme which classifies the ECG beats into three categories: normal beats, Premature Ventricular Contraction (PVC) beats and other beats. To demonstrate the generalization capability of the scheme, this classifier is developed utilizing a small training dataset and then tested with a large testing dataset. Our proposed scheme was employed for 40 benchmark ECG files of the MIT/BIH database. The system could produce classification accuracy as high as 95.24% and could outperform several competing algorithms.
---
paper_title: Forecasting time-series by Kohonen classification

paper_content:

In this paper, we propose a generic non-linear approach for time series forecasting. The main feature of this approach is the use of a simple statistical forecasting in small regions of an input space adequately chosen and quantized. The partition of the space is achieved by the Kohonen algorithm. The method is then applied to a widely known time-series from the SantaFe competition, and the results are compared with the best ones published for this series.
---
paper_title: Generalized relevance learning vector quantization

paper_content:

We propose a new scheme for enlarging generalized learning vector quantization (GLVQ) with weighting factors for the input dimensions. The factors allow an appropriate scaling of the input dimensions according to their relevance. They are adapted automatically during training according to the specific classification task whereby training can be interpreted as stochastic gradient descent on an appropriate error function. This method leads to a more powerful classifier and to an adaptive metric with little extra cost compared to standard GLVQ. Moreover, the size of the weighting factors indicates the relevance of the input dimensions. This proposes a scheme for automatically pruning irrelevant input dimensions. The algorithm is verified on artificial data sets and the iris data from the UCI repository. Afterwards, the method is compared to several well known algorithms which determine the intrinsic data dimension on real world satellite image data.
---
paper_title: Intelligent switching control of a pneumatic muscle robot arm using learning vector quantization neural network

paper_content:

Pneumatic cylinders are one of the low-cost actuation sources used in industrial and prosthetic application, since they have a high power/weight ratio, high-tension force and long durability. However, problems with the control, oscillatory motion and compliance of pneumatic systems have prevented their widespread use in advanced robotics. To overcome these shortcomings, a number of newer pneumatic actuators have been developed, such as the McKibben Muscle, Rubber Actuator and Pneumatic Artificial Muscle (PAM) Manipulators. In this paper, the solution for position control of a robot arm with slow motion driven by two pneumatic artificial muscles is presented. However, some limitations still exist, such as a deterioration of the performance of transient response due to the changes in the external load. To overcome this problem, a switching algorithm of the control parameter using a learning vector quantization neural network (LVQNN) is proposed in this paper. The LVQNN estimates the external load of the pneumatic artificial muscle manipulator. The effectiveness of the proposed control algorithm is demonstrated through experiments with different external working loads.
---
paper_title: Suppressed fuzzy-soft learning vector quantization for MRI segmentation

paper_content:

Objective: A self-organizing map (SOM) is a competitive artificial neural network with unsupervised learning. To increase the SOM learning effect, a fuzzy-soft learning vector quantization (FSLVQ) algorithm has been proposed in the literature, using fuzzy functions to approximate lateral neural interaction of the SOM. However, the computational performance of FSLVQ is still not good enough, especially for large data sets. In this paper, we propose a suppressed FSLVQ (S-FSLVQ) using suppression with a parameter learning schema. We then apply the S-FSLVQ to MRI segmentation and compare it with several existing methods. Methods and materials: The proposed S-FSLVQ algorithm and some existing methods, such as FSLVQ, generalized LVQ, revised generalized LVQ and alternative LVQ, are compared using numerical data and MRI images. The numerical data are generated by a mixture of normal distributions. The MRI data sets are from a 2-year-old female patient who was diagnosed with retinoblastoma of her left eye, a congenital malignant neoplasm of the retina with frequent metastasis beyond the lacrimal cribrosa. To evaluate the performance of these algorithms, two criteria for accuracy and computational efficiency are used. Results: Comparing S-FSLVQ with FSLVQ, generalized LVQ, revised generalized LVQ and alternative LVQ, the numerical results indicate that the S-FSLVQ algorithm is better than the other algorithms in accuracy and computational efficiency. Moreover, the proposed S-FSLVQ can reduce the computation time and increase accuracy compared to existing methods in segmenting these ophthalmological MRIs. Conclusions: The proposed S-FSLVQ is a good competitive learning algorithm that is very suitable for segmenting the ophthalmological MRI data sets. Therefore, the S-FSLVQ algorithm is highly recommended for use in MRI segmentation as an aid for supportive diagnoses.
---
paper_title: Fuzzy algorithms for learning vector quantization

paper_content:

This paper presents the development of fuzzy algorithms for learning vector quantization (FALVQ). These algorithms are derived by minimizing the weighted sum of the squared Euclidean distances between an input vector, which represents a feature vector, and the weight vectors of a competitive learning vector quantization (LVQ) network, which represent the prototypes. This formulation leads to competitive algorithms, which allow each input vector to attract all prototypes. The strength of attraction between each input and the prototypes is determined by a set of membership functions, which can be selected on the basis of specific criteria. A gradient-descent-based learning rule is derived for a general class of admissible membership functions which satisfy certain properties. The FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms are developed by selecting admissible membership functions with different properties. The proposed algorithms are tested and evaluated using the IRIS data set. The efficiency of the proposed algorithms is also illustrated by their use in codebook design required for image compression based on vector quantization.
---
paper_title: Feature-based classification of time-series data

paper_content:

In this paper we propose the use of statistical features for time-series classification. The classification is performed with a multi-layer perceptron (MLP) neural network. The proposed method is examined in the context of Control Chart Pattern data, which are time series used in Statistical Process Control. Experimental results verify the efficiency of the feature-based classification method, compared to previous methods which classify time series based on the values of each time point. Moreover, the results show the robustness of the proposed method against noise and time-series length.
---
paper_title: Prediction of laser butt joint welding parameters using back propagation and learning vector quantization networks

paper_content:

Abstract Laser welding parameters include not only the laser power, focused spot size, welding speed, focused position, etc., but also the welding gap and the alignment of the laser beam with the center of the welding gap, these latter two parameters being critical for a butt joint. These parameters are controllable in the actual operation of laser welding, but are interconnected and extremely non-linear, such problems limit the industrial applicability of the laser welding for butt joints. The neural network technique is a useful tool for predicting the operation parameters of a non-linear model. Back propagation (BP) and learning vector quantization (LVQ) networks are presented in this paper to predict the laser welding parameters for butt joints. The input parameters of the network include workpiece thickness and welding gap, whilst the output parameters include optimal focused position, acceptable welding parameters of laser power and welding speed, and welding quality, including weld width, undercut and distortion for the associated power and speed used. The results of this research show a comprehensive and usable prediction of the laser welding parameters for butt joints using BP and LVQ networks. As a result, the industrial applicability of laser welding for butt joints can be expanded widely.
---
paper_title: Generalized Learning Vector Quantization

paper_content:

We propose a new learning method, "Generalized Learning Vector Quantization (GLVQ)," in which reference vectors are updated based on the steepest descent method in order to minimize the cost function. The cost function is determined so that the obtained learning rule satisfies the convergence condition. We prove that Kohonen's rule as used in LVQ does not satisfy the convergence condition and thus degrades recognition ability. Experimental results for printed Chinese character recognition reveal that GLVQ is superior to LVQ in recognition ability.
---
paper_title: Initialization insensitive LVQ algorithm based on cost-function adaptation

paper_content:

A learning vector quantization (LVQ) algorithm called harmonic to minimum LVQ algorithm (H2M-LVQ) is presented to tackle the initialization sensitiveness problem associated with the original generalized LVQ (GLVQ) algorithm. Experimental results show superior performance of the H2M-LVQ algorithm over the GLVQ and one of its variants on several datasets. datasets.
---
paper_title: Prototype-based classification of dissimilarity data

paper_content:

Unlike many black-box algorithms in machine learning, prototype-based models offer an intuitive interface to given data sets, since prototypes can directly be inspected by experts in the field. Most techniques rely on Euclidean vectors such that their suitability for complex scenarios is limited. Recently, several unsupervised approaches have successfully been extended to general, possibly non-Euclidean data characterized by pairwise dissimilarities. In this paper, we shortly review a general approach to extend unsupervised prototype-based techniques to dissimilarities, and we transfer this approach to supervised prototypebased classification for general dissimilarity data. In particular, a new supervised prototype-based classification technique for dissimilarity data is proposed.
---
paper_title: Soft Nearest Prototype Classification

paper_content:

We propose a new method for the construction of nearest prototype classifiers which is based on a Gaussian mixture ansatz and which can be interpreted as an annealed version of learning vector quantization (LVQ). The algorithm performs a gradient descent on a cost-function minimizing the classification error on the training set. We investigate the properties of the algorithm and assess its performance for several toy data sets and for an optical letter classification task. Results show 1) that annealing in the dispersion parameter of the Gaussian kernels improves classification accuracy; 2) that classification results are better than those obtained with standard learning vector quantization (LVQ 2.1, LVQ 3) for equal numbers of prototypes; and 3) that annealing of the width parameter improved the classification capability. Additionally, the principled approach provides an explanation of a number of features of the (heuristic) LVQ methods.
---
paper_title: Supervised Neural Gas with General Similarity Measure

paper_content:

Prototype based classification offers intuitive and sparse models with excellent generalization ability. However, these models usually crucially depend on the underlying Euclidian metric; moreover, online variants likely suffer from the problem of local optima. We here propose a generalization of learning vector quantization with three additional features: (I) it directly integrates neighborhood cooperation, hence is less affected by local optima; (II) the method can be combined with any differentiable similarity measure whereby metric parameters such as relevance factors of the input dimensions can automatically be adapted according to the given data; (III) it obeys a gradient dynamics hence shows very robust behavior, and the chosen objective is related to margin optimization.
---
paper_title: Distance Learning in Discriminative Vector Quantization

paper_content:

Discriminative vector quantization schemes such as learning vector quantization (LVQ) and extensions thereof offer efficient and intuitive classifiers based on the representation of classes by prototypes. The original methods, however, rely on the Euclidean distance corresponding to the assumption that the data can be represented by isotropic clusters. For this reason, extensions of the methods to more general metric structures have been proposed, such as relevance adaptation in generalized LVQ (GLVQ) and matrix learning in GLVQ. In these approaches, metric parameters are learned based on the given classification task such that a data-driven distance measure is found. In this letter, we consider full matrix adaptation in advanced LVQ schemes. In particular, we introduce matrix learning to a recent statistical formalization of LVQ, robust soft LVQ, and we compare the results on several artificial and real-life data sets to matrix learning in GLVQ, a derivation of LVQ-like learning based on a (heuristic) cost function. In all cases, matrix adaptation allows a significant improvement of the classification accuracy. Interestingly, however, the principled behavior of the models with respect to prototype locations and extracted matrix dimensions shows several characteristic differences depending on the data sets.
---
paper_title: Detection of seizure activity in EEG by an artificial neural network: a preliminary study

paper_content:

Abstract Neural networks, inspired by the organizational principles of the human brain, have recently been used in various fields of application such as pattern recognition, identification, classification, speech, vision, signal processing, and control systems. In this study, a two-layered neural network has been trained for the recognition of temporal patterns of the electroencephalogram (EEG). This network is called a Learning Vector Quantization (LVQ) neural network since it learns the characteristics of the signal presented to it as a vector. The first layer is a competitive layer which learns to classify the input vectors. The second, linear, layer transforms the output of the competitive layer to target classes defined by the user. We have tested and evaluated the LVQ network. The network successfully detects epileptiform discharges (EDs) when trained using EEG records scored by a neurologist. Epochs of EEG containing EDs from one subject have been used for training the network, and EEGs of other subjects have been used for testing the network. The results demonstrate that the LVQ detector can generalize the learning to previously “unseen” records of subjects. This study shows that the LVQ network offers a practical solution for ED detection which is easily adjusted to an individual neurologist's style and is as sensitive and specific as an expert visual analysis.
---
paper_title: Two soft relatives of learning vector quantization

paper_content:

Abstract Learning vector quantization often requires extensive experimentation with the learning rate distribution and update neighborhood used during iteration towards good prototypes. A single winner prototype controls the updates. This paper discusses two soft relatives of LVQ: the soft competition scheme (SCS) of Yair et al. and fuzzy LVQ=FLVQ. These algorithms both extend the update neighborhood to all nodes in the network. SCS is a sequential, deterministic method with learning rates that are partially based on posterior probabilities. FLVQ is a batch algorithm whose learning rates are derived from fuzzy memberships. We show that SCS learning rates can be interpreted in terms of statistical decision theory, and derive several relationships between SCS and FLVQ. Limit analysis shows that the learning rates of these two algorithms have opposite tendencies. Numerical examples illustrate the difficulty of choosing good algorithmic parameters for SCS. Finally, we elaborate the relationship between FL VQ, Fuzzy c-Means, Hard c-Means, a batch version of LVQ and SCS.
---
paper_title: A Methodology for Constructing Fuzzy Algorithms for Learning Vector Quantization

paper_content:

This paper presents a general methodology for the development of fuzzy algorithms for learning vector quantization (FALVQ). The design of specific FALVQ algorithms according to existing approaches reduces to the selection of the membership function assigned to the weight vectors of an LVQ competitive neural network, which represent the prototypes. The development of a broad variety of FALVQ algorithms can be accomplished by selecting the form of the interference function that determines the effect of the nonwinning prototypes on the attraction between the winning prototype and the input of the network. The proposed methodology provides the basis for extending the existing FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms. This paper also introduces two quantitative measures which establish a relationship between the formulation that led to FALVQ algorithms and the competition between the prototypes during the learning process. The proposed algorithms and competition measures are tested and evaluated using the IRIS data set. The significance of the proposed competition measure is illustrated using FALVQ algorithms to perform segmentation of magnetic resonance images of the brain.
---
paper_title: Learning Vector Quantization Neural Networks for LED Wafer Defect Inspection

paper_content:

Automatic visual inspection of defects plays an important role in industrial manufacturing with the benefits of low-cost and high accuracy. In light-emitting diode (LED) manufacturing, each die on the LED wafer must be inspected to determine whether it has defects or not. Therefore, detection of defective regions is a significant issue to discuss. In this paper, a new approach for inspection of LED wafer defects using the learning vector quantization (LVQ) neural network is presented. In the wafer image, each die image and the region of interest (ROI) in them to handle can be acquired. Then, by analyzing the properties of every ROI, we can extract specific geometric features and texture features. Using these features, the LVQ neural network is presented to classify these dies as either acceptable or not. The experimental results confirm the usefulness of the approach for LED wafer defect inspection.
---
paper_title: Fault diagnosis of stamping process based on empirical mode decomposition and learning vector quantization

paper_content:

Sheet metal stamping process is widely used in industry due to its high accuracy and productivity. However, monitoring the process is a difficult task since the monitoring signals are typically non-stationary transient signals. In this paper, empirical mode decomposition (EMD) is applied to extract the main features of the strain signals. First, the signal is decomposed by EMD into intrinsic mode functions (IMF). Then the signal energy and the Hilbert marginal spectrum, which reflects the working condition and the fault pattern of the process, are computed. Finally, to identify the faulty conditions of process, the learning vector quantization (LVQ) network is used as a classifier with the Hilbert marginal spectrum as the input vectors. The performance of this method is tested by 107 experiments derived from different conditions in the sheet metal stamping process. The artificially created defects can be detected with a success rate of 96.3%. The method seems to be useful to monitor a sheet metal stamping process in practice.
---
paper_title: An introduction to neural computing

paper_content:

Abstract This article contains a brief survey of the motivations, fundamentals, and applications of artificial neural networks, as well as some detailed analytical expressions for their theory.
---
paper_title: Improved versions of learning vector quantization

paper_content:

The author introduces a variant of (supervised) learning vector quantization (LVQ) and discusses practical problems associated with the application of the algorithms. The LVQ algorithms work explicitly in the input domain of the primary observation vectors, and their purpose is to approximate the theoretical Bayes decision borders using piecewise linear decision surfaces. This is done by purported optimal placement of the class codebook vectors in signal space. As the classification decision is based on the nearest-neighbor selection among the codebook vectors, its computation is very fast. It has turned out that the differences between the presented algorithms in regard to the remaining discretization error are not significant, and thus the choice of the algorithm may be based on secondary arguments, such as stability in learning, in which respect the variant introduced (LVQ2.1) seems to be superior to the others. A comparative study of several methods applied to speech recognition is included
---
paper_title: Margin Analysis of the LVQ Algorithm

paper_content:

Prototypes based algorithms are commonly used to reduce the computational complexity of Nearest-Neighbour (NN) classifiers. In this paper we discuss theoretical and algorithmical aspects of such algorithms. On the theory side, we present margin based generalization bounds that suggest that these kinds of classifiers can be more accurate then the 1-NN rule. Furthermore, we derived a training algorithm that selects a good set of prototypes using large margin principles. We also show that the 20 years old Learning Vector Quantization (LVQ) algorithm emerges naturally from our framework.
---
paper_title: Generalized Learning Vector Quantization

paper_content:

We propose a new learning method, "Generalized Learning Vector Quantization (GLVQ)," in which reference vectors are updated based on the steepest descent method in order to minimize the cost function. The cost function is determined so that the obtained learning rule satisfies the convergence condition. We prove that Kohonen's rule as used in LVQ does not satisfy the convergence condition and thus degrades recognition ability. Experimental results for printed Chinese character recognition reveal that GLVQ is superior to LVQ in recognition ability.
---
paper_title: On the Generalization Ability of GRLVQ Networks

paper_content:

We derive a generalization bound for prototype-based classifiers with adaptive metric. The bound depends on the margin of the classifier and is independent of the dimensionality of the data. It holds for classifiers based on the Euclidean metric extended by adaptive relevance terms. In particular, the result holds for relevance learning vector quantization (RLVQ) [4] and generalized relevance learning vector quantization (GRLVQ) [19].
---
paper_title: Soft Learning Vector Quantization

paper_content:

Learning vector quantization (LVQ) is a popular class of adaptive nearest prototype classifiers for multiclass classification, but learning algorithms from this family have so far been proposed on heuristic grounds. Here, we take a more principled approach and derive two variants of LVQ using a gaussian mixture ansatz. We propose an objective function based on a likelihood ratio and derive a learning rule using gradient descent. The new approach provides a way to extend the algorithms of the LVQ family to different distance measure and allows for the design of "soft" LVQ algorithms. Benchmark results show that the new methods lead to better classification performance than LVQ 2.1. An additional benefit of the new method is that model assumptions are made explicit, so that the method can be adapted more easily to different kinds of problems.
---
paper_title: Soft Nearest Prototype Classification

paper_content:

We propose a new method for the construction of nearest prototype classifiers which is based on a Gaussian mixture ansatz and which can be interpreted as an annealed version of learning vector quantization (LVQ). The algorithm performs a gradient descent on a cost-function minimizing the classification error on the training set. We investigate the properties of the algorithm and assess its performance for several toy data sets and for an optical letter classification task. Results show 1) that annealing in the dispersion parameter of the Gaussian kernels improves classification accuracy; 2) that classification results are better than those obtained with standard learning vector quantization (LVQ 2.1, LVQ 3) for equal numbers of prototypes; and 3) that annealing of the width parameter improved the classification capability. Additionally, the principled approach provides an explanation of a number of features of the (heuristic) LVQ methods.
---
paper_title: Generalized relevance learning vector quantization

paper_content:

We propose a new scheme for enlarging generalized learning vector quantization (GLVQ) with weighting factors for the input dimensions. The factors allow an appropriate scaling of the input dimensions according to their relevance. They are adapted automatically during training according to the specific classification task whereby training can be interpreted as stochastic gradient descent on an appropriate error function. This method leads to a more powerful classifier and to an adaptive metric with little extra cost compared to standard GLVQ. Moreover, the size of the weighting factors indicates the relevance of the input dimensions. This proposes a scheme for automatically pruning irrelevant input dimensions. The algorithm is verified on artificial data sets and the iris data from the UCI repository. Afterwards, the method is compared to several well known algorithms which determine the intrinsic data dimension on real world satellite image data.
---
paper_title: Generalized Relevance LVQ for Time Series

paper_content:

An application of the recently proposed generalized relevance learning vector quantization (GRLVQ) to the analysis and modeling of time series data is presented. We use GRLVQ for two tasks: first, for obtaining a phase space embedding of a scalar time series, and second, for short term and long term data prediction. The proposed embedding method is tested with a signal from the wellknown Lorenz system. Afterwards, it is applied to daily lysimeter observations of water runoff. A one-stepp rediction of the runoff dynamic is obtained from the classification of high dimensional subseries data vectors, from which a promising technique for long term forecasts is derived.
---
paper_title: Adaptive Relevance Matrices in Learning Vector Quantization

paper_content:

We propose a new matrix learning scheme to extend relevance learning vector quantization (RLVQ), an efficient prototype-based classification algorithm, toward a general adaptive metric. By introducing a full matrix of relevance factors in the distance measure, correlations between different features and their importance for the classification scheme can be taken into account and automated, and general metric adaptation takes place during training. In comparison to the weighted Euclidean metric used in RLVQ and its variations, a full matrix is more powerful to represent the internal structure of the data appropriately. Large margin generalization bounds can be transferred to this case, leading to bounds that are independent of the input dimensionality. This also holds for local metrics attached to each prototype, which corresponds to piecewise quadratic decision boundaries. The algorithm is tested in comparison to alternative learning vector quantization schemes using an artificial data set, a benchmark multiclass problem from the UCI repository, and a problem from bioinformatics, the recognition of splice sites for C. elegans.
---
paper_title: Distance Learning in Discriminative Vector Quantization

paper_content:

Discriminative vector quantization schemes such as learning vector quantization (LVQ) and extensions thereof offer efficient and intuitive classifiers based on the representation of classes by prototypes. The original methods, however, rely on the Euclidean distance corresponding to the assumption that the data can be represented by isotropic clusters. For this reason, extensions of the methods to more general metric structures have been proposed, such as relevance adaptation in generalized LVQ (GLVQ) and matrix learning in GLVQ. In these approaches, metric parameters are learned based on the given classification task such that a data-driven distance measure is found. In this letter, we consider full matrix adaptation in advanced LVQ schemes. In particular, we introduce matrix learning to a recent statistical formalization of LVQ, robust soft LVQ, and we compare the results on several artificial and real-life data sets to matrix learning in GLVQ, a derivation of LVQ-like learning based on a (heuristic) cost function. In all cases, matrix adaptation allows a significant improvement of the classification accuracy. Interestingly, however, the principled behavior of the models with respect to prototype locations and extracted matrix dimensions shows several characteristic differences depending on the data sets.
---
paper_title: A novel kernel prototype-based learning algorithm

paper_content:

We propose a novel kernel prototype-based learning algorithm, called kernel generalized learning vector quantization (KGLYQ) algorithm, which can significantly improve the classification performance of the original generalized learning vector quantization algorithm in complex pattern classification tasks. In addition, the KGLVQ can also serve as a good general kernel learning framework for further investigation.
---
paper_title: Input space versus feature space in kernel-based methods

paper_content:

This paper collects some ideas targeted at advancing our understanding of the feature spaces associated with support vector (SV) kernel functions. We first discuss the geometry of feature space. In particular, we review what is known about the shape of the image of input space under the feature space map, and how this influences the capacity of SV methods. Following this, we describe how the metric governing the intrinsic geometry of the mapped surface can be computed in terms of the kernel, using the example of the class of inhomogeneous polynomial kernels, which are often used in SV pattern recognition. We then discuss the connection between feature space and input space by dealing with the question of how one can, given some vector in feature space, find a preimage (exact or approximate) in input space. We describe algorithms to tackle this issue, and show their utility in two applications of kernel methods. First, we use it to reduce the computational complexity of SV decision functions; second, we combine it with the kernel PCA algorithm, thereby constructing a nonlinear statistical denoising technique which is shown to perform well on real-world data.
---
paper_title: Efficient Kernelized prototype based classification.

paper_content:

Prototype based classifiers are effective algorithms in modeling classification problems and have been applied in multiple domains. While many supervised learning algorithms have been successfully extended to kernels to improve the discrimination power by means of the kernel concept, prototype based classifiers are typically still used with Euclidean distance measures. Kernelized variants of prototype based classifiers are currently too complex to be applied for larger data sets. Here we propose an extension of Kernelized Generalized Learning Vector Quantization (KGLVQ) employing a sparsity and approximation technique to reduce the learning complexity. We provide generalization error bounds and experimental results on real world data, showing that the extended approach is comparable to SVM on different public data.
---
paper_title: Relational extensions of learning vector quantization

paper_content:

Prototype-based models offer an intuitive interface to given data sets by means of an inspection of the model prototypes. Supervised classification can be achieved by popular techniques such as learning vector quantization (LVQ) and extensions derived from cost functions such as generalized LVQ (GLVQ) and robust soft LVQ (RSLVQ). These methods, however, are restricted to Euclidean vectors and they cannot be used if data are characterized by a general dissimilarity matrix. In this approach, we propose relational extensions of GLVQ and RSLVQ which can directly be applied to general possibly non-Euclidean data sets characterized by a symmetric dissimilarity matrix.
---
paper_title: The Dissimilarity Representation for Pattern Recognition: Foundations and Applications

paper_content:

# Spaces # Characterization of Dissimilarities # Learning Approaches # Dissimilarity Measures # Visualization # Further Data Exploration # One-Class Classifiers # Classification # Combining # Representation Review and Recommendations # Conclusions and Open Problems
---
paper_title: Prototype-based classification of dissimilarity data

paper_content:

Unlike many black-box algorithms in machine learning, prototype-based models offer an intuitive interface to given data sets, since prototypes can directly be inspected by experts in the field. Most techniques rely on Euclidean vectors such that their suitability for complex scenarios is limited. Recently, several unsupervised approaches have successfully been extended to general, possibly non-Euclidean data characterized by pairwise dissimilarities. In this paper, we shortly review a general approach to extend unsupervised prototype-based techniques to dissimilarities, and we transfer this approach to supervised prototypebased classification for general dissimilarity data. In particular, a new supervised prototype-based classification technique for dissimilarity data is proposed.
---
paper_title: Improved versions of learning vector quantization

paper_content:

The author introduces a variant of (supervised) learning vector quantization (LVQ) and discusses practical problems associated with the application of the algorithms. The LVQ algorithms work explicitly in the input domain of the primary observation vectors, and their purpose is to approximate the theoretical Bayes decision borders using piecewise linear decision surfaces. This is done by purported optimal placement of the class codebook vectors in signal space. As the classification decision is based on the nearest-neighbor selection among the codebook vectors, its computation is very fast. It has turned out that the differences between the presented algorithms in regard to the remaining discretization error are not significant, and thus the choice of the algorithm may be based on secondary arguments, such as stability in learning, in which respect the variant introduced (LVQ2.1) seems to be superior to the others. A comparative study of several methods applied to speech recognition is included
---
paper_title: Vector quantization using information theoretic concepts

paper_content:

The process of representing a large data set with a smaller number of vectors in the best possible way, also known as vector quantization, has been intensively studied in the recent years. Very efficient algorithms like the Kohonen self-organizing map (SOM) and the Linde Buzo Gray (LBG) algorithm have been devised. In this paper a physical approach to the problem is taken, and it is shown that by considering the processing elements as points moving in a potential field an algorithm equally efficient as the before mentioned can be derived. Unlike SOM and LBG this algorithm has a clear physical interpretation and relies on minimization of a well defined cost function. It is also shown how the potential field approach can be linked to information theory by use of the Parzen density estimator. In the light of information theory it becomes clear that minimizing the free energy of the system is in fact equivalent to minimizing a divergence measure between the distribution of the data and the distribution of the processing elements, hence, the algorithm can be seen as a density matching method.
---
paper_title: Feature Extraction by Non-Parametric Mutual Information Maximization

paper_content:

We present a method for learning discriminative feature transforms using as criterion the mutual information between class labels and transformed features. Instead of a commonly used mutual information measure based on Kullback-Leibler divergence, we use a quadratic divergence measure, which allows us to make an efficient non-parametric implementation and requires no prior assumptions about class densities. In addition to linear transforms, we also discuss nonlinear transforms that are implemented as radial basis function networks. Extensions to reduce the computational complexity are also presented, and a comparison to greedy feature selection is made.
---
paper_title: A supervised growing neural gas algorithm for cluster analysis

paper_content:

In this paper, a prototype-based supervised clustering algorithm is proposed. The proposed algorithm, called the Supervised Growing Neural Gas algorithm (SGNG), incorporates several techniques from some unsupervised GNG algorithms such as the adaptive learning rates and the cluster repulsion mechanisms of the Robust Growing Neural Gas algorithm, and the Type Two Learning Vector Quantization (LVQ2) technique. Furthermore, a new prototype insertion mechanism and a clustering validity index are proposed. These techniques are designed to utilize class labels of the training data to guide the clustering. The SGNG algorithm is capable of clustering adjacent regions of data objects labeled with different classes, formulating topological relationships among prototypes and automatically determining the optimal number of clusters using the proposed validity index. To evaluate the effectiveness of the SGNG algorithm, two experiments are conducted. The first experiment uses two synthetic data sets to graphically illustrate the potential with respect to growing ability, ability to cluster adjacent regions of different classes, and ability to determine the optimal number of prototypes. The second experiment evaluates the effectiveness using the UCI benchmark data sets. The results from the second experiment show that the SGNG algorithm performs better than other supervised clustering algorithms for both cluster impurities and total running times.
---
paper_title: A new generalized LVQ algorithm via harmonic to minimum distance measure transition

paper_content:

We present a novel generalized learning vector quantization (LVQ) framework called the harmonic to minimum generalized LVQ algorithm (H2M-GLVQ). Through incorporating the distance measure transition procedure from harmonic average distance to minimum distance, the H2M-GLVQ cost function is gradually changing from the soft model to the hard model. Our proposed method, at the early training stage, can effectively tackle the initialization sensitivity problem associated with the original generalized LVQ algorithm while the convergence of the algorithm can be ensured by the hard model in the later training stage. Experimental results have shown the superior performance of the H2M-GLVQ algorithm over the generalized LVQ and one of its variants on some artificial multi-modal datasets
---
paper_title: Divergence-Based Vector Quantization

paper_content:

Supervised and unsupervised vector quantization methods for classification and clustering traditionally use dissimilarities, frequently taken as Euclidean distances. In this article, we investigate the applicability of divergences instead, focusing on online learning. We deduce the mathematical fundamentals for its utilization in gradient-based online vector quantization algorithms. It bears on the generalized derivatives of the divergences known as Fréchet derivatives in functional analysis, which reduces in finite-dimensional problems to partial derivatives in a natural way. We demonstrate the application of this methodology for widely applied supervised and unsupervised online vector quantization schemes, including self-organizing maps, neural gas, and learning vector quantization. Additionally, principles for hyperparameter optimization and relevance learning for parameterized divergences in the case of supervised vector quantization are given to achieve improved classification accuracy.
---
paper_title: Initialization insensitive LVQ algorithm based on cost-function adaptation

paper_content:

A learning vector quantization (LVQ) algorithm called harmonic to minimum LVQ algorithm (H2M-LVQ) is presented to tackle the initialization sensitiveness problem associated with the original generalized LVQ (GLVQ) algorithm. Experimental results show superior performance of the H2M-LVQ algorithm over the GLVQ and one of its variants on several datasets. datasets.
---
paper_title: Fuzzy classification using information theoretic learning vector quantization

paper_content:

In this article we extend the (recently published) unsupervised information theoretic vector quantization approach based on the Cauchy-Schwarz-divergence for matching data and prototype densities to supervised learning and classification. In particular, first we generalize the unsupervised method to more general metrics instead of the Euclidean, as it was used in the original algorithm. Thereafter, we extend the model to a supervised learning method resulting in a fuzzy classification algorithm. Thereby, we allow fuzzy labels for both, data and prototypes. Finally, we transfer the idea of relevance learning for metric adaptation known from learning vector quantization to the new approach. We show the abilities and the power of the method for exemplary and real-world medical applications.
---
paper_title: Supervised Neural Gas with General Similarity Measure

paper_content:

Prototype based classification offers intuitive and sparse models with excellent generalization ability. However, these models usually crucially depend on the underlying Euclidian metric; moreover, online variants likely suffer from the problem of local optima. We here propose a generalization of learning vector quantization with three additional features: (I) it directly integrates neighborhood cooperation, hence is less affected by local optima; (II) the method can be combined with any differentiable similarity measure whereby metric parameters such as relevance factors of the input dimensions can automatically be adapted according to the given data; (III) it obeys a gradient dynamics hence shows very robust behavior, and the chosen objective is related to margin optimization.
---
paper_title: A Growing Neural Gas Network Learns Topologies

paper_content:

An incremental network model is introduced which is able to learn the important topological relations in a given set of input vectors by means of a simple Hebb-like learning rule. In contrast to previous approaches like the "neural gas" method of Martinetz and Schulten (1991, 1994), this model has no parameters which change over time and is able to continue learning, adding units and connections, until a performance criterion has been met. Applications of the model include vector quantization, clustering, and interpolation.
---
paper_title: ;Neural-gas' network for vector quantization and its application to time-series prediction.

paper_content:

A neural network algorithm based on a soft-max adaptation rule is presented. This algorithm exhibits good performance in reaching the optimum minimization of a cost function for vector quantization data compression. The soft-max rule employed is an extension of the standard K-means clustering procedure and takes into account a neighborhood ranking of the reference (weight) vectors. It is shown that the dynamics of the reference (weight) vectors during the input-driven adaptation procedure are determined by the gradient of an energy function whose shape can be modulated through a neighborhood determining parameter and resemble the dynamics of Brownian particles moving in a potential determined by the data point density. The network is used to represent the attractor of the Mackey-Glass equation and to predict the Mackey-Glass time series, with additional local linear mappings for generating output values. The results obtained for the time-series prediction compare favorably with the results achieved by backpropagation and radial basis function networks.
---
paper_title: Mutual Information in Learning Feature Transformations

paper_content:

We present feature transformations useful for exploratory data analysis or for pattern recognition. Transformations are learned from example data sets by maximizing the mutual information between transformed data and their class labels. We make use of Renyi’s quadratic entropy, and we extend the work of Principe et al. to mutual information between continuous multidimensional variables and discrete-valued class labels.
---
paper_title: Generalized relevance learning vector quantization

paper_content:

We propose a new scheme for enlarging generalized learning vector quantization (GLVQ) with weighting factors for the input dimensions. The factors allow an appropriate scaling of the input dimensions according to their relevance. They are adapted automatically during training according to the specific classification task whereby training can be interpreted as stochastic gradient descent on an appropriate error function. This method leads to a more powerful classifier and to an adaptive metric with little extra cost compared to standard GLVQ. Moreover, the size of the weighting factors indicates the relevance of the input dimensions. This proposes a scheme for automatically pruning irrelevant input dimensions. The algorithm is verified on artificial data sets and the iris data from the UCI repository. Afterwards, the method is compared to several well known algorithms which determine the intrinsic data dimension on real world satellite image data.
---
paper_title: Distance Learning in Discriminative Vector Quantization

paper_content:

Discriminative vector quantization schemes such as learning vector quantization (LVQ) and extensions thereof offer efficient and intuitive classifiers based on the representation of classes by prototypes. The original methods, however, rely on the Euclidean distance corresponding to the assumption that the data can be represented by isotropic clusters. For this reason, extensions of the methods to more general metric structures have been proposed, such as relevance adaptation in generalized LVQ (GLVQ) and matrix learning in GLVQ. In these approaches, metric parameters are learned based on the given classification task such that a data-driven distance measure is found. In this letter, we consider full matrix adaptation in advanced LVQ schemes. In particular, we introduce matrix learning to a recent statistical formalization of LVQ, robust soft LVQ, and we compare the results on several artificial and real-life data sets to matrix learning in GLVQ, a derivation of LVQ-like learning based on a (heuristic) cost function. In all cases, matrix adaptation allows a significant improvement of the classification accuracy. Interestingly, however, the principled behavior of the models with respect to prototype locations and extracted matrix dimensions shows several characteristic differences depending on the data sets.
---
paper_title: A novel kernel prototype-based learning algorithm

paper_content:

We propose a novel kernel prototype-based learning algorithm, called kernel generalized learning vector quantization (KGLYQ) algorithm, which can significantly improve the classification performance of the original generalized learning vector quantization algorithm in complex pattern classification tasks. In addition, the KGLVQ can also serve as a good general kernel learning framework for further investigation.
---
paper_title: Kernel robust soft learning vector quantization

paper_content:

Prototype-based classification schemes offer very intuitive and flexible classifiers with the benefit of easy interpretability of the results and scalability of the model complexity. Recent prototype-based models such as robust soft learning vector quantization (RSLVQ) have the benefit of a solid mathematical foundation of the learning rule and decision boundaries in terms of probabilistic models and corresponding likelihood optimization. In its original form, they can be used for standard Euclidean vectors only. In this contribution, we extend RSLVQ towards a kernelized version which can be used for any positive semidefinite data matrix. We demonstrate the superior performance of the technique, kernel RSLVQ, in a variety of benchmarks where results competitive or even superior to state-of-the-art support vector machines are obtained.
---
paper_title: Efficient Kernelized prototype based classification.

paper_content:

Prototype based classifiers are effective algorithms in modeling classification problems and have been applied in multiple domains. While many supervised learning algorithms have been successfully extended to kernels to improve the discrimination power by means of the kernel concept, prototype based classifiers are typically still used with Euclidean distance measures. Kernelized variants of prototype based classifiers are currently too complex to be applied for larger data sets. Here we propose an extension of Kernelized Generalized Learning Vector Quantization (KGLVQ) employing a sparsity and approximation technique to reduce the learning complexity. We provide generalization error bounds and experimental results on real world data, showing that the extended approach is comparable to SVM on different public data.
---
paper_title: Relational extensions of learning vector quantization

paper_content:

Prototype-based models offer an intuitive interface to given data sets by means of an inspection of the model prototypes. Supervised classification can be achieved by popular techniques such as learning vector quantization (LVQ) and extensions derived from cost functions such as generalized LVQ (GLVQ) and robust soft LVQ (RSLVQ). These methods, however, are restricted to Euclidean vectors and they cannot be used if data are characterized by a general dissimilarity matrix. In this approach, we propose relational extensions of GLVQ and RSLVQ which can directly be applied to general possibly non-Euclidean data sets characterized by a symmetric dissimilarity matrix.
---
paper_title: Prototype-based classification of dissimilarity data

paper_content:

Unlike many black-box algorithms in machine learning, prototype-based models offer an intuitive interface to given data sets, since prototypes can directly be inspected by experts in the field. Most techniques rely on Euclidean vectors such that their suitability for complex scenarios is limited. Recently, several unsupervised approaches have successfully been extended to general, possibly non-Euclidean data characterized by pairwise dissimilarities. In this paper, we shortly review a general approach to extend unsupervised prototype-based techniques to dissimilarities, and we transfer this approach to supervised prototypebased classification for general dissimilarity data. In particular, a new supervised prototype-based classification technique for dissimilarity data is proposed.
---
paper_title: Soft Learning Vector Quantization

paper_content:

Learning vector quantization (LVQ) is a popular class of adaptive nearest prototype classifiers for multiclass classification, but learning algorithms from this family have so far been proposed on heuristic grounds. Here, we take a more principled approach and derive two variants of LVQ using a gaussian mixture ansatz. We propose an objective function based on a likelihood ratio and derive a learning rule using gradient descent. The new approach provides a way to extend the algorithms of the LVQ family to different distance measure and allows for the design of "soft" LVQ algorithms. Benchmark results show that the new methods lead to better classification performance than LVQ 2.1. An additional benefit of the new method is that model assumptions are made explicit, so that the method can be adapted more easily to different kinds of problems.
---
paper_title: Soft Nearest Prototype Classification

paper_content:

We propose a new method for the construction of nearest prototype classifiers which is based on a Gaussian mixture ansatz and which can be interpreted as an annealed version of learning vector quantization (LVQ). The algorithm performs a gradient descent on a cost-function minimizing the classification error on the training set. We investigate the properties of the algorithm and assess its performance for several toy data sets and for an optical letter classification task. Results show 1) that annealing in the dispersion parameter of the Gaussian kernels improves classification accuracy; 2) that classification results are better than those obtained with standard learning vector quantization (LVQ 2.1, LVQ 3) for equal numbers of prototypes; and 3) that annealing of the width parameter improved the classification capability. Additionally, the principled approach provides an explanation of a number of features of the (heuristic) LVQ methods.
---
paper_title: Multiple Comparison Procedures

paper_content:

PROCEDURES BASED ON CLASSICAL APPROACHES FOR FIXED--EFFECTS LINEAR MODELS WITH NORMAL HOMOSCEDASTIC INDEPENDENT ERRORS. Some Theory of Multiple Comparisons Procedure Fixed--effects Linear Models. Single--step Procedures for Pairwise and More General Comparisons Among All Treatments. Stepwise Procedures for Pairwise and More General Comparisons Among All Treatments. Procedures for Some Other Nonhierarchical Finite Families of Comparisons. Designing Experiments for Multiple Comparisons. PROCEDURES FOR OTHER MODELS AND PROBLEMS, AND PROCEDURES BASED ON ALTERNATIVE APPROACHES. Procedures for One--way Layouts with Unequal Variances. Procedures for Some Mixed--effects Models. Distribution--free and Robust Procedures. Some Miscellaneous Multiple Comparison Problems. Optimal Procedures Using Decision--theoretic, Bayesian, and Other Approaches. Appendixes. Tables. References. Index.
---
paper_title: Initialization insensitive LVQ algorithm based on cost-function adaptation

paper_content:

A learning vector quantization (LVQ) algorithm called harmonic to minimum LVQ algorithm (H2M-LVQ) is presented to tackle the initialization sensitiveness problem associated with the original generalized LVQ (GLVQ) algorithm. Experimental results show superior performance of the H2M-LVQ algorithm over the GLVQ and one of its variants on several datasets. datasets.
---
paper_title: On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach

paper_content:

An important component of many data mining projects is finding a good classification algorithm, a process that requires very careful thought about experimental design. If not done very carefully, comparative studies of classification and other types of algorithms can easily result in statistically invalid conclusions. This is especially true when one is using data mining techniques to analyze very large databases, which inevitably contain some statistically unlikely data. This paper describes several phenomena that can, if ignored, invalidate an experimental comparison. These phenomena and the conclusions that follow apply not only to classification, but to computational experiments in almost any aspect of data mining. The paper also discusses why comparative analysis is more important in evaluating some types of algorithms than for others, and provides some suggestions about how to avoid the pitfalls suffered by many experimental studies.
---
paper_title: A probabilistic active support vector learning algorithm

paper_content:

The paper describes a probabilistic active learning strategy for support vector machine (SVM) design in large data applications. The learning strategy is motivated by the statistical query model. While most existing methods of active SVM learning query for points based on their proximity to the current separating hyperplane, the proposed method queries for a set of points according to a distribution as determined by the current separating hyperplane and a newly defined concept of an adaptive confidence factor. This enables the algorithm to have more robust and efficient learning capabilities. The confidence factor is estimated from local information using the k nearest neighbor principle. The effectiveness of the method is demonstrated on real-life data sets both in terms of generalization performance, query complexity, and training time.
---
paper_title: Margin based Active Learning for LVQ Networks

paper_content:

In this article, we extend a local prototype-based learning model by active learning, which gives the learner the capability to select training samples during the model adaptation procedure. The proposed active learning strategy aims on an improved generalization ability of the final model. This is achieved by usage of an adaptive query strategy which is more adequate for supervised learning than a simple random approach. Beside an improved generalization ability the method also improves the speed of the learning procedure which is especially beneficial for large data sets with multiple similar items. The algorithm is based on the idea of selecting a query on the borderline of the actual classification. This can be done by considering margins in an extension of learning vector quantization based on an appropriate cost function. The proposed active learning approach is analyzed for two kinds of learning vector quantizers the supervised relevance neural gas and the supervised nearest prototype classifier, but is applicable for a broader set of prototype-based learning approaches too. The performance of the query algorithm is demonstrated on synthetic and real life data taken from clinical proteomic studies. From the proteomic studies high-dimensional mass spectrometry measurements were calculated which are believed to contain features discriminating the different classes. Using the proposed active learning strategies, the generalization ability of the models could be kept or improved accompanied by a significantly improved learning speed. Both of these characteristics are important for the generation of predictive clinical models and were used in an initial biomarker discovery study.
---
paper_title: Efficient Approximations of Kernel Robust Soft LVQ

paper_content:

Robust soft learning vector quantization (RSLVQ) constitutes a probabilistic extension of learning vector quantization (LVQ) based on a labeled Gaussian mixture model of the data. Training optimizes the likelihood ratio of the model and recovers a variant similar to LVQ2.1 in the limit of small bandwidth. Recently, RSLVQ has been extended to a kernel version, thus opening the way towards more general data structures characterized in terms of a Gram matrix only. While leading to state of the art results, this extension has the drawback that models are no longer sparse, and quadratic training complexity is encountered. In this contribution, we investigate two approximation schemes which lead to sparse models: k-approximations of the prototypes and the Nystrom approximation of the Gram matrix. We investigate the behavior of these approximations in a couple of benchmarks.
---
paper_title: How to Visualize Large Data Sets

paper_content:

We address novel developments in the context of dimensionality reduction for data visualization. We consider nonlinear non-parametric techniques such as t-distributed stochastic neighbor embedding and discuss the difficulties which are encountered if large data sets are dealt with, in contrast to parametric approaches such as the self-organizing map. We focus on the following topics, which arise in this context: (i) how can dimensionality reduction be realized efficiently in at most linear time, (ii) how can nonparametric approaches be extended to provide an explicit mapping, (iii) how can techniques be extended to incorporate auxiliary information as provided by class labeling?
---
paper_title: Online Visualization of Prototypes and Receptive Fields Produced by LVQ Algorithms

paper_content:

A new approach is proposed to visualize online the training of learning vector quantization algorithms. The prototypes and data samples associated to each receptive field are projected onto a two-dimensional map by using a non-linear transformation of the input space. The mapping finds a set of projection vectors by minimizing a cost function, which preserves the local topology of the input space. The proposed visualization is tested on two datasets: image segmentation and pipeline. The usefulness of the method is demonstrated by studying the behavior of Generalized LVQ, Supervised Neural Gas and Harmonic to Minimum LVQ algorithms on high-dimensional datasets.
---
paper_title: Neural net algorithms that learn in polynomial time from examples and queries

paper_content:

An algorithm which trains networks using examples and queries is proposed. In a query, the algorithm supplies a y and is told t(y) by an oracle. Queries appear to be available in practice for most problems of interest, e.g. by appeal to a human expert. The author's algorithm is proved to PAC learn in polynomial time the class of target functions defined by layered, depth two, threshold nets having n inputs connected to k hidden threshold units connected to one or more output units, provided k >
---
paper_title: Using the Nyström Method to Speed Up Kernel Machines

paper_content:

A major problem for kernel-based predictors (such as Support Vector Machines and Gaussian processes) is that the amount of computation required to find the solution scales as O(n3), where n is the number of training examples. We show that an approximation to the eigendecomposition of the Gram matrix can be computed by the Nystrom method (which is used for the numerical solution of eigenproblems). This is achieved by carrying out an eigendecomposition on a smaller system of size m < n, and then expanding the results back up to n dimensions. The computational complexity of a predictor using this approximation is O(m2n). We report experiments on the USPS and abalone data sets and show that we can set m ≪ n without any significant decrease in the accuracy of the solution.
---

Title: A Review of Learning Vector Quantization Classifiers

Section 1: Introduction
Description 1: Introduce the scope and objectives of the paper, highlighting the significance and advantages of Learning Vector Quantization (LVQ) classifiers.

Section 2: Taxonomy of LVQ Classifiers
Description 2: Present a taxonomy of LVQ classifiers, describing the different families and their characteristics.

Section 3: Main LVQ Learning Rules
Description 3: Describe the main learning rules of LVQ algorithms, explaining the evolutionary improvements from the original LVQ to its modern variants.

Section 4: Margin Maximization
Description 4: Discuss LVQ methods based on margin maximization, detailing their cost functions and learning rules.

Section 5: Likelihood Ratio Maximization
Description 5: Explore LVQ methods based on likelihood ratio maximization, explaining their probabilistic approaches and cost functions.

Section 6: Distance Learning
Description 6: Describe distance learning approaches in LVQ, focusing on adaptive and generalized metrics.

Section 7: Kernelization
Description 7: Explain kernel approaches in LVQ, detailing how traditional LVQ algorithms are extended using kernel functions.

Section 8: Dis-/similarities
Description 8: Discuss methods based on dis-/similarities, focusing on how LVQ can be applied to non-Euclidean spaces using relational data.

Section 9: LVQ 2.1
Description 9: Provide a detailed description of LVQ 2.1, outlining its updating rules and improvements over earlier versions.

Section 10: Results
Description 10: Present experimental results comparing various LVQ algorithms on different datasets, summarizing the performance and sensitivity of each method.

Section 11: Open Problems
Description 11: Identify and discuss open problems and challenges in the field of LVQ classifiers, proposing areas for future research.

Section 12: Conclusions
Description 12: Summarize the key findings and contributions of the paper, highlighting the strengths and limitations of different LVQ methods.