Data Mining and Soft Computing using Support Vector Machine: A Survey
---
paper_title: The Nature of Statistical Learning Theory

paper_content:

Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.
---
paper_title: New empirical nonparametric kernels for support vector machine classification

paper_content:

Despite the excellent applicability of kernel methods, there seems to be no systematic way of choosing appropriate kernel functions or the optimum parameters. Therefore, the performance of support vector machines (SVMs) cannot be easily optimized. To address this problem, a general procedure is suggested to produce nonparametric and efficient kernels. This is achieved by finding an empirical and theoretical connection between positive semidefinite matrices and certain metric space properties. The Gaussian kernel turns out to be a special case of the new framework. Comprehensive experiments on eleven real-world datasets and seven synthetic datasets demonstrate a clear advantage in favor of the proposed kernels. However, several important problems remain unresolved.
---
paper_title: Structural Regularized Support Vector Machine: A Framework for Structural Large Margin Classifier

paper_content:

Support vector machine (SVM), as one of the most popular classifiers, aims to find a hyperplane that can separate two classes of data with maximal margin. SVM classifiers are focused on achieving more separation between classes than exploiting the structures in the training data within classes. However, the structural information, as an implicit prior knowledge, has recently been found to be vital for designing a good classifier in different real-world problems. Accordingly, using as much prior structural information in data as possible to help improve the generalization ability of a classifier has yielded a class of effective structural large margin classifiers, such as the structured large margin machine (SLMM) and the Laplacian support vector machine (LapSVM). In this paper, we unify these classifiers into a common framework from the concept of “structural granularity” and the formulation for optimization problems. We exploit the quadratic programming (QP) and second-order cone programming (SOCP) methods, and derive a novel large margin classifier, we call the new classifier the structural regularized support vector machine (SRSVM). Unlike both SLMM at the cross of the cluster granularity and SOCP and LapSVM at the cross of the point granularity and QP, SRSVM is located at the cross of the cluster granularity and QP and thus follows the same optimization formulation as LapSVM to overcome large computational complexity and non-sparse solution in SLMM. In addition, it integrates the compactness within classes with the separability between classes simultaneously. Furthermore, it is possible to derive generalization bounds for these algorithms by using eigenvalue analysis of the kernel matrices. Experimental results demonstrate that SRSVM is often superior in classification and generalization performances to the state-of-the-art algorithms in the framework, both with the same and different structural granularities.
---
paper_title: Matrix Pattern Based Minimum Within-Class Scatter Support Vector Machines

paper_content:

Based on minimum within-class scatter support vector machines(MCSVMs),a new matrix pattern based MCSVMs(MCSVMsmatrix) is presented.Accordingly,it is extended by introducing Mercer's kernels in order to solve the problem of nonlinear decision boundaries,which presents a significant matrix pattern based nonlinear support vector machines:Ker-MCSVMsmatrix.The above-mentioned approaches not only keep the merits of MCSVMs,but,owing to introducing matrix pattern based within-class scatter matrix into support vector machines,theoretically better solve the singular problem of within-class scatter matrix when small sample size problems are dealt with,reduce the time/place complexity when within-class scatter matrix,its invertible matrix and coefficient vector omega are calculated.Hence,the classification accuracy is improved to certain extent.Experimental results indicate the above advantages of the proposed methods:both MCSVMsmatrix and Ker-MCSVMsmatrix.
---
paper_title: Scaling Up Support Vector Machines Using Nearest Neighbor Condensation

paper_content:

In this brief, we describe the FCNN-SVM classifier, which combines the support vector machine (SVM) approach and the fast nearest neighbor condensation classification rule (FCNN) in order to make SVMs practical on large collections of data. As a main contribution, it is experimentally shown that, on very large and multidimensional data sets, the FCNN-SVM is one or two orders of magnitude faster than SVM, and that the number of support vectors (SVs) is more than halved with respect to SVM. Thus, a drastic reduction of both training and testing time is achieved by using the FCNN-SVM. This result is obtained at the expense of a little loss of accuracy. The FCNN-SVM is proposed as a viable alternative to the standard SVM in applications where a fast response time is a fundamental requirement.
---
paper_title: A sequential algorithm for sparse support vector classifiers

paper_content:

Support vector machines (SVMs), though accurate, are not preferred in applications requiring high classification speed or when deployed in systems of limited computational resources, due to the large number of support vectors involved in the model. To overcome this problem we have devised a primal SVM method with the following properties: (1) it solves for the SVM representation without the need to invoke the representer theorem, (2) forward and backward selections are combined to approach the final globally optimal solution, and (3) a criterion is introduced for identification of support vectors leading to a much reduced support vector set. In addition to introducing this method the paper analyzes the complexity of the algorithm and presents test results on three public benchmark problems and a human activity recognition application. These applications demonstrate the effectiveness and efficiency of the proposed algorithm.
---
paper_title: Determination of Global Minima of Some Common Validation Functions in Support Vector Machine

paper_content:

Tuning of the regularization parameter C is a well-known process in the implementation of a support vector machine (SVM) classifier. Such a tuning process uses an appropriate validation function whose value, evaluated over a validation set, has to be optimized for the determination of the optimal C. Unfortunately, most common validation functions are not smooth functions of C. This brief presents a method for obtaining the global optimal solution of these non-smooth validation functions. The method is guaranteed to find the global optimum and relies on the regularization solution path of SVM over a range of C values. When the solution path is available, the computation needed is minimal.
---
paper_title: Improved support vector clustering

paper_content:

Support vector clustering (SVC) is an important boundary-based clustering algorithm in multi applications. But SVC's popularity is degraded by its pricy computation and poor labeling performance. Different from existing modifications that only resolve one of two bottlenecks, this paper presents an improved SVC, iSVC, to address two bottlenecks simultaneously. iSVC's contributions are as follows: (1) It includes a reduction strategy that can help to develop clustering model on a qualified subset. The reduction strategy is based on the Schrodinger equation to find the crucial data towards model formulation. (2) The original objective is modified; it cooperates with the reduction strategy to produce the model with subtle loss of quality. (3) iSVC employs a new label approach to label data according to the geometric properties of feature space. The new approach labels data in a simple but effective way without suffering from the randomness originated in the old algorithm. (4) The geometric property is proofed to guarantee the new labeling approach's validation. Theoretical analysis and empirical evidence suggest that iSVC overcomes two bottlenecks well. And when compared with some common clustering methods, it does a good job in performance and efficiency, which opens a broad way of applications for SVC.
---
paper_title: New support vector-based design method for binary hierarchical classifiers for multi-class classification problems

paper_content:

We propose a new hierarchical design method, weighted support vector (WSV) k-means clustering, to design a binary hierarchical classification structure. This method automatically selects the classes to be separated at each node in the hierarchy, and allows visualization of clusters of high-dimensional support vector data; no prior hierarchical designs address this. At each node in the hierarchy, we use an SVRDM (support vector representation and discrimination machine) classifier, which offers generalization and good rejection of unseen false objects (rejection is not achieved with the standard SVMs). We give the basis and new insight into why a Gaussian kernel provides good rejection. Recognition and rejection test results on a real IR (infrared) database show that our proposed method outperforms the standard one-vs-rest methods and the use of standard SVM classifiers.
---
paper_title: TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition

paper_content:

A novel twin parametric-margin support vector machine (TPMSVM) for classification is proposed in this paper. This TPMSVM, in the spirit of the twin support vector machine (TWSVM), determines indirectly the separating hyperplane through a pair of nonparallel parametric-margin hyperplanes solved by two smaller sized support vector machine (SVM)-type problems. Similar to the parametric-margin @[email protected]?support vector machine ([email protected]@?SVM), this TPMSVM is suitable for many cases, especially when the data has heteroscedastic error structure, that is, the noise strongly depends on the input value. But there is an advantage in the learning speed compared with the [email protected]@?SVM. The experimental results on several artificial and benchmark datasets indicate that the TPMSVM not only obtains fast learning speed, but also shows good generalization.
---
paper_title: Density-Based Multifeature Background Subtraction with Support Vector Machine

paper_content:

Background modeling and subtraction is a natural technique for object detection in videos captured by a static camera, and also a critical preprocessing step in various high-level computer vision applications. However, there have not been many studies concerning useful features and binary segmentation algorithms for this problem. We propose a pixelwise background modeling and subtraction technique using multiple features, where generative and discriminative techniques are combined for classification. In our algorithm, color, gradient, and Haar-like features are integrated to handle spatio-temporal variations for each pixel. A pixelwise generative background model is obtained for each feature efficiently and effectively by Kernel Density Approximation (KDA). Background subtraction is performed in a discriminative manner using a Support Vector Machine (SVM) over background likelihood vectors for a set of features. The proposed algorithm is robust to shadow, illumination changes, spatial variations of background. We compare the performance of the algorithm with other density-based methods using several different feature combinations and modeling techniques, both quantitatively and qualitatively.
---
paper_title: Improvements on Twin Support Vector Machines

paper_content:

For classification problems, the generalized eigenvalue proximal support vector machine (GEPSVM) and twin support vector machine (TWSVM) are regarded as milestones in the development of the powerful SVMs, as they use the nonparallel hyperplane classifiers. In this brief, we propose an improved version, named twin bounded support vector machines (TBSVM), based on TWSVM. The significant advantage of our TBSVM over TWSVM is that the structural risk minimization principle is implemented by introducing the regularization term. This embodies the marrow of statistical learning theory, so this modification can improve the performance of classification. In addition, the successive overrelaxation technique is used to solve the optimization problems to speed up the training procedure. Experimental results show the effectiveness of our method in both computation time and classification accuracy, and therefore confirm the above conclusion further.
---
paper_title: Tuning Support Vector Machines for Minimax and Neyman-Pearson Classification

paper_content:

This paper studies the training of support vector machine (SVM) classifiers with respect to the minimax and Neyman-Pearson criteria. In principle, these criteria can be optimized in a straightforward way using a cost-sensitive SVM. In practice, however, because these criteria require especially accurate error estimation, standard techniques for tuning SVM parameters, such as cross-validation, can lead to poor classifier performance. To address this issue, we first prove that the usual cost-sensitive SVM, here called the 2C-SVM, is equivalent to another formulation called the 2\nu-SVM. We then exploit a characterization of the 2\nu-SVM parameter space to develop a simple yet powerful approach to error estimation based on smoothing. In an extensive experimental study, we demonstrate that smoothing significantly improves the accuracy of cross-validation error estimates, leading to dramatic performance gains. Furthermore, we propose coordinate descent strategies that offer significant gains in computational efficiency, with little to no loss in performance.
---
paper_title: Multitask multiclass support vector machines: Model and experiments

paper_content:

Multitask learning or learning multiple related tasks simultaneously has shown a better performance than learning these tasks independently. Most approaches to multitask multiclass problems decompose them into multiple multitask binary problems, and thus cannot effectively capture inherent correlations between classes. Although very elegant, traditional multitask support vector machines are restricted by the fact that different learning tasks have to share the same set of classes. In this paper, we present an approach to multitask multiclass support vector machines based on the minimization of regularization functionals. We cast multitask multiclass problems into a constrained optimization problem with a quadratic objective function. Therefore, our approach can learn multitask multiclass problems directly and effectively. This approach can learn in two different scenarios: label-compatible and label-incompatible multitask learning. We can easily generalize the linear multitask learning method to the non-linear case using kernels. A number of experiments, including comparisons with other multitask learning methods, indicate that our approach for multitask multiclass problems is very encouraging.
---
paper_title: Recursive Support Vector Machines for Dimensionality Reduction

paper_content:

The usual dimensionality reduction technique in supervised learning is mainly based on linear discriminant analysis (LDA), but it suffers from singularity or undersampled problems. On the other hand, a regular support vector machine (SVM) separates the data only in terms of one single direction of maximum margin, and the classification accuracy may be not good enough. In this letter, a recursive SVM (RSVM) is presented, in which several orthogonal directions that best separate the data with the maximum margin are obtained. Theoretical analysis shows that a completely orthogonal basis can be derived in feature subspace spanned by the training samples and the margin is decreasing along the recursive components in linearly separable cases. As a result, a new dimensionality reduction technique based on multilevel maximum margin components and then a classifier with high accuracy are achieved. Experiments in synthetic and several real data sets show that RSVM using multilevel maximum margin features can do efficient dimensionality reduction and outperform regular SVM in binary classification problems.
---
paper_title: Multiple Incremental Decremental Learning of Support Vector Machines

paper_content:

We propose a multiple incremental decremental algorithm of Support Vector Machine (SVM). Conventional single incremental decremental SVM can update the trained model efficiently when single data point is added to or removed from the training set. When we add and/or remove multiple data points, this algorithm is time-consuming because we need to repeatedly apply it to each data point. The proposed algorithm is computationally more efficient when multiple data points are added and/or removed simultaneously. The single incremental decremental algorithm is built on an optimization technique called parametric programming. We extend the idea and introduce multi-parametric programming for developing the proposed algorithm. Experimental results on synthetic and real data sets indicate that the proposed algorithm can significantly reduce the computational cost of multiple incremental decremental operation. Our approach is especially useful for online SVM learning in which we need to remove old data points and add new data points in a short amount of time.
---
paper_title: New support vector algorithms with parametric insensitive/margin model

paper_content:

In this paper, a modification of v-support vector machines (v-SVM) for regression and classification is described, and the use of a parametric insensitive/margin model with an arbitrary shape is demonstrated. This can be useful in many cases, especially when the noise is heteroscedastic, that is, the noise strongly depends on the input value x. Like the previous v-SVM, the proposed support vector algorithms have the advantage of using the parameter [email protected][email protected]?1 for controlling the number of support vectors. To be more precise, v is an upper bound on the fraction of training errors and a lower bound on the fraction of support vectors. The algorithms are analyzed theoretically and experimentally.
---
paper_title: Efficient Classification for Additive Kernel SVMs

paper_content:

We show that a class of nonlinear kernel SVMs admits approximate classifiers with runtime and memory complexity that is independent of the number of support vectors. This class of kernels, which we refer to as additive kernels, includes widely used kernels for histogram-based image comparison like intersection and chi-squared kernels. Additive kernel SVMs can offer significant improvements in accuracy over linear SVMs on a wide variety of tasks while having the same runtime, making them practical for large-scale recognition or real-time detection tasks. We present experiments on a variety of datasets, including the INRIA person, Daimler-Chrysler pedestrians, UIUC Cars, Caltech-101, MNIST, and USPS digits, to demonstrate the effectiveness of our method for efficient evaluation of SVMs with additive kernels. Since its introduction, our method has become integral to various state-of-the-art systems for PASCAL VOC object detection/image classification, ImageNet Challenge, TRECVID, etc. The techniques we propose can also be applied to settings where evaluation of weighted additive kernels is required, which include kernelized versions of PCA, LDA, regression, k-means, as well as speeding up the inner loop of SVM classifier training algorithms.
---
paper_title: Solving Nonstationary Classification Problems With Coupled Support Vector Machines

paper_content:

Many learning problems may vary slowly over time: in particular, some critical real-world applications. When facing this problem, it is desirable that the learning method could find the correct input-output function and also detect the change in the concept and adapt to it. We introduce the time-adaptive support vector machine (TA-SVM), which is a new method for generating adaptive classifiers, capable of learning concepts that change with time. The basic idea of TA-SVM is to use a sequence of classifiers, each one appropriate for a small time window but, in contrast to other proposals, learning all the hyperplanes in a global way. We show that the addition of a new term in the cost function of the set of SVMs (that penalizes the diversity between consecutive classifiers) produces a coupling of the sequence that allows TA-SVM to learn as a single adaptive classifier. We evaluate different aspects of the method using appropriate drifting problems. In particular, we analyze the regularizing effect of changing the number of classifiers in the sequence or adapting the strength of the coupling. A comparison with other methods in several problems, including the well-known STAGGER dataset and the real-world electricity pricing domain, shows the good performance of TA-SVM in all tested situations.
---
paper_title: On-line Independent Support Vector Machines

paper_content:

Support vector machines (SVMs) are one of the most successful algorithms for classification. However, due to their space and time requirements, they are not suitable for on-line learning, that is, when presented with an endless stream of training observations. In this paper we propose a new on-line algorithm, called on-line independent support vector machines (OISVMs), which approximately converges to the standard SVM solution each time new observations are added; the approximation is controlled via a user-defined parameter. The method employs a set of linearly independent observations and tries to project every new observation onto the set obtained so far, dramatically reducing time and space requirements at the price of a negligible loss in accuracy. As opposed to similar algorithms, the size of the solution obtained by OISVMs is always bounded, implying a bounded testing time. These statements are supported by extensive experiments on standard benchmark databases as well as on two real-world applications, namely place recognition by a mobile robot in an indoor environment and human grasping posture classification.
---
paper_title: Hierarchical linear support vector machine

paper_content:

The increasing size and dimensionality of real-world datasets make it necessary to design efficient algorithms not only in the training process but also in the prediction phase. In applications such as credit card fraud detection, the classifier needs to predict an event in 10ms at most. In these environments the speed of the prediction constraints heavily outweighs the training costs. We propose a new classification method, called a Hierarchical Linear Support Vector Machine (H-LSVM), based on the construction of an oblique decision tree in which the node split is obtained as a Linear Support Vector Machine. Although other methods have been proposed to break the data space down in subregions to speed up Support Vector Machines, the H-LSVM algorithm represents a very simple and efficient model in training but mainly in prediction for large-scale datasets. Only a few hyperplanes need to be evaluated in the prediction step, no kernel computation is required and the tree structure makes parallelization possible. In experiments with medium and large datasets, the H-LSVM reduces the prediction cost considerably while achieving classification results closer to the non-linear SVM than that of the linear case.
---
paper_title: An Improved Algorithm for the Solution of the Regularization Path of Support Vector Machine

paper_content:

This paper describes an improved algorithm for the numerical solution to the support vector machine (SVM) classification problem for all values of the regularization parameter C . The algorithm is motivated by the work of Hastie and follows the main idea of tracking the optimality conditions of the SVM solution for ascending value of C . It differs from Hastie's approach in that the tracked path is not assumed to be 1-D. Instead, a multidimensional feasible space for the optimality condition is used to solve the tracking problem. Such a treatment allows the algorithm to properly handle data sets which Hastie's approach fails. These data sets are characterized by the presence of linearly dependent points (in the kernel space), duplicate points, or nearly duplicate points. Such data sets are quite common among many real-world data, especially those with nominal features. Other contributions of this paper include a unifying formulation of the tracking process in the form of a linear programming problem, update formula for the linear programs, considerations that guard against accumulation of errors resulting from the use of incremental updates, and routines to speed up the algorithm. The algorithm is implemented under the Matlab environment and is available for download. Experiments with several data sets including data set having up to several thousand data points are reported.
---
paper_title: Extended Input Space Support Vector Machine

paper_content:

In some applications, the probability of error of a given classifier is too high for its practical application, but we are allowed to gather more independent test samples from the same class to reduce the probability of error of the final decision. From the point of view of hypothesis testing, the solution is given by the Neyman-Pearson lemma. However, there is no equivalent result to the Neyman-Pearson lemma when the likelihoods are unknown, and we are given a training dataset. In this brief, we explore two alternatives. First, we combine the soft (probabilistic) outputs of a given classifier to produce a consensus labeling for test samples. In the second approach, we build a new classifier that directly computes the label for test samples. For this second approach, we need to define an extended input space training set and incorporate the known symmetries in the classifier. This latter approach gives more accurate results, as it only requires an accurate classification boundary, while the former needs an accurate posterior probability estimate for the whole input space. We illustrate our results with well-known databases.
---
paper_title: Optimal Feature Selection for Support Vector Machines

paper_content:

Selecting relevant features for support vector machine (SVM) classifiers is important for a variety of reasons such as generalization performance, computational efficiency, and feature interpretability. Traditional SVM approaches to feature selection typically extract features and learn SVM parameters independently. Independently performing these two steps might result in a loss of information related to the classification process. This paper proposes a convex energy-based framework to jointly perform feature selection and SVM parameter learning for linear and non-linear kernels. Experiments on various databases show significant reduction of features used while maintaining classification performance.
---
paper_title: Structural twin support vector machine for classification

paper_content:

It has been shown that the structural information of data may contain useful prior domain knowledge for training a classifier. How to apply the structural information of data to build a good classifier is a new research focus recently. As we all know, the all existing structural large margin methods are the common in considering all structural information within classes into one model. In fact, these methods do not balance all structural information's relationships both infra-class and inter-class, which directly results in these prior information not being exploited sufficiently. In this paper, we design a new Structural Twin Support Vector Machine (called S-TWSVM). Unlike existing methods based on structural information, S-TWSVM uses two hyperplanes to decide the category of new data, of which each model only considers one class's structural information and closer to the class at the same time far away from the other class. This makes S-TWSVM fully exploit these prior knowledge to directly improve the algorithm's the capacity of generalization. All experiments show that our proposed method is rigidly superior to the state-of-the-art algorithms based on structural information of data in both computation time and classification accuracy.
---
paper_title: Fast instance selection for speeding up support vector machines

paper_content:

Support vector machine (SVM) has shown prominent performance for binary classification. How to effectively apply it to massive datasets with large number of classes and instances is still a serious challenge. Instance selection methods have been proposed and shown significant efficacy for reducing the training complexity of SVM, but more or less trade off the generalization performance. This paper presents an instance selection method especially for multi-class problems. With cluster centers of positive class as reference points instances are selected for each one-versus-rest SVM model. The purpose of clustering here is to improve the efficiency of instance selection, other than to select instances directly from clusters as previous methods did. Experiments on a wide variety of datasets demonstrate that the proposed method selects fewer instances than most competitive algorithms and keeps the highest classification accuracy on most datasets. Additionally, experimental results show that this method also performs superiorly for binary problems.
---
paper_title: FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning

paper_content:

Support vector machines (SVMs) is a popular machine learning technique, which works effectively with balanced datasets. However, when it comes to imbalanced datasets, SVMs produce suboptimal classification models. On the other hand, the SVM algorithm is sensitive to outliers and noise present in the datasets. Therefore, although the existing class imbalance learning (CIL) methods can make SVMs less sensitive to class imbalance, they can still suffer from the problem of outliers and noise. Fuzzy SVMs (FSVMs) is a variant of the SVM algorithm, which has been proposed to handle the problem of outliers and noise. In FSVMs, training examples are assigned different fuzzy-membership values based on their importance, and these membership values are incorporated into the SVM learning algorithm to make it less sensitive to outliers and noise. However, like the normal SVM algorithm, FSVMs can also suffer from the problem of class imbalance. In this paper, we present a method to improve FSVMs for CIL (called FSVM-CIL), which can be used to handle the class imbalance problem in the presence of outliers and noise. We thoroughly evaluated the proposed FSVM-CIL method on ten real-world imbalanced datasets and compared its performance with five existing CIL methods, which are available for normal SVM training. Based on the overall results, we can conclude that the proposed FSVM-CIL method is a very effective method for CIL, especially in the presence of outliers and noise in datasets.
---
paper_title: A neural support vector machine

paper_content:

Support vector machines are state-of-the-art pattern recognition algorithms that are well founded in optimization and generalization theory but not obviously applicable to the brain. This paper presents Bio-SVM, a biologically feasible support vector machine. An unstable associative memory oscillates between support vectors and interacts with a feed-forward classification pathway. Kernel neurons blend support vectors and sensory input. Downstream temporal integration generates the classification. Instant learning of surprising events and off-line tuning of support vector weights trains the system. Emotion-based learning, forgetting trivia, sleep and brain oscillations are phenomena that agree with the Bio-SVM model. A mapping to the olfactory system is suggested.
---
paper_title: A Kernel Fuzzy c-Means Clustering-Based Fuzzy Support Vector Machine Algorithm for Classification Problems With Outliers or Noises

paper_content:

The support vector machine (SVM) has provided higher performance than traditional learning machines and has been widely applied in real-world classification problems and nonlinear function estimation problems. Unfortunately, the training process of the SVM is sensitive to the outliers or noises in the training set. In this paper, a common misunderstanding of Gaussian-function-based kernel fuzzy clustering is corrected, and a kernel fuzzy c-means clustering-based fuzzy SVM algorithm (KFCM-FSVM) is developed to deal with the classification problems with outliers or noises. In the KFCM-FSVM algorithm, we first use the FCM clustering to cluster each of two classes from the training set in the high-dimensional feature space. The farthest pair of clusters, where one cluster comes from the positive class and the other from the negative class, is then searched and forms one new training set with membership degrees. Finally, we adopt FSVM to induce the final classification results on this new training set. The computational complexity of the KFCM-FSVM algorithm is analyzed. A set of experiments is conducted on six benchmarking datasets and four artificial datasets for testing the generalization performance of the KFCM-FSVM algorithm. The results indicate that the KFCM-FSVM algorithm is robust for classification problems with outliers or noises.
---
paper_title: Rule extraction from support vector machines based on consistent region covering reduction

paper_content:

Due to good performance in classification and regression, support vector machines have attracted much attention and become one of the most popular learning machines in last decade. As a black box, the support vector machine is difficult for users' understanding and explanation. In many application domains including medical diagnosis or credit scoring, understandability and interpretability are very important for the practicability of the learned models. To improve the comprehensibility of SVMs, we propose a rule extraction technique from support vector machines via analyzing the distribution of samples. We define the consistent region of samples in terms of classification boundary, and form a consistent region covering of the sample space. Then a covering reduction algorithm is developed for extracting compact representation of classes, thus a minimal set of decision rules is derived. Experiment analysis shows that the extracted models perform well in comparison with decision tree algorithms and other support vector machine rule extraction methods.
---
paper_title: Feature selection in the Laplacian support vector machine

paper_content:

Traditional classifiers including support vector machines use only labeled data in training. However, labeled instances are often difficult, costly, or time consuming to obtain while unlabeled instances are relatively easy to collect. The goal of semi-supervised learning is to improve the classification accuracy by using unlabeled data together with a few labeled data in training classifiers. Recently, the Laplacian support vector machine has been proposed as an extension of the support vector machine to semi-supervised learning. The Laplacian support vector machine has drawbacks in its interpretability as the support vector machine has. Also it performs poorly when there are many non-informative features in the training data because the final classifier is expressed as a linear combination of informative as well as non-informative features. We introduce a variant of the Laplacian support vector machine that is capable of feature selection based on functional analysis of variance decomposition. Through synthetic and benchmark data analysis, we illustrate that our method can be a useful tool in semi-supervised learning.
---

Title: Data Mining and Soft Computing using Support Vector Machine: A Survey

Section 1: INTRODUCTION
Description 1: Write about the fundamentals of Support Vector Machine (SVM), including its application for linearly separable data, soft-margin extension, nonlinear decision boundary, and the kernel trick.

Section 2: DATA MINING AND SVM
Description 2: Discuss the role of SVM in data mining, covering different data mining techniques such as association, classification, clustering, regression, and outlier detection. Provide a review and summary of previous research papers and methodologies involving SVM in data mining.

Section 3: SOFT COMPUTING AND SVM
Description 3: Explain the relationship between SVM and soft computing, including optimization techniques like genetic algorithms, ant colony optimization, swarm optimization, and neural networks. Describe the application of SVM in various soft computing tasks.

Section 4: DISCUSSION
Description 4: Summarize the key points from the survey of data mining and soft computing applications using SVM. Discuss the future scope and potential research directions in these fields.

Section 5: CONCLUSION
Description 5: Provide a conclusion summarizing the effectiveness of SVM in multi-class classification, dimensionality reduction, optimization, and various applications highlighted in the survey.