Ensemble Classifiers and Their Applications: A Review
---
paper_title: Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

paper_content:

Diversity among the members of a team of classifiers is deemed to be a key issue in classifier combination. However, measuring diversity is not straightforward because there is no generally accepted formal definition. We have found and studied ten statistics which can measure diversity among binary classifier outputs (correct or incorrect vote for the class label): four averaged pairwise measures (the Q statistic, the correlation, the disagreement and the double fault) and six non-pairwise measures (the entropy of the votes, the difficulty index, the Kohavi-Wolpert variance, the interrater agreement, the generalized diversity, and the coincident failure diversity). Four experiments have been designed to examine the relationship between the accuracy of the team and the measures of diversity, and among the measures themselves. Although there are proven connections between diversity and accuracy in some special cases, our results raise some doubts about the usefulness of diversity measures in building classifier ensembles in real-life pattern recognition problems.
---
paper_title: An analysis of diversity measures

paper_content:

Diversity among the base classifiers is deemed to be important when constructing a classifier ensemble. Numerous algorithms have been proposed to construct a good classifier ensemble by seeking both the accuracy of the base classifiers and the diversity among them. However, there is no generally accepted definition of diversity, and measuring the diversity explicitly is very difficult. Although researchers have designed several experimental studies to compare different diversity measures, usually confusing results were observed. In this paper, we present a theoretical analysis on six existing diversity measures (namely disagreement measure, double fault measure, KW variance, inter-rater agreement, generalized diversity and measure of difficulty), show underlying relationships between them, and relate them to the concept of margin, which is more explicitly related to the success of ensemble learning algorithms. We illustrate why confusing experimental results were observed and show that the discussed diversity measures are naturally ineffective. Our analysis provides a deeper understanding of the concept of diversity, and hence can help design better ensemble learning algorithms.
---
paper_title: Ensemble classifier generation using non-uniform layered clustering and Genetic Algorithm

paper_content:

In this paper, we propose a novel cluster oriented ensemble classifier generation method and a Genetic Algorithm based approach to optimize the parameters. In the proposed method the data set is partitioned into a variable number of clusters at different layers. Base classifiers are trained on the clusters at different layers. Due to the variability of the number of clusters at different layers, the cluster compositions in one layer are different from that in another layer. Due to this difference in cluster contents, the base classifiers trained at different layers are diverse among each other. A test pattern is classified by the base classifier of the nearest cluster at each layer and the decisions from different layers are fused using majority voting. The accuracy of the proposed method depends on the number of layers and the number of clusters at the corresponding layer. A Genetic Algorithm based search is incorporated to obtain the optimal number of layers and clusters. The Genetic Algorithm is evaluated under three different objective functions: optimizing (i) accuracy, (ii) diversity, and (iii) accuracyxdiversity. We have conducted a number of experiments to evaluate the effectiveness of the different objective functions.
---
paper_title: Combining the Predictions of Multiple Classifiers: Using Competitive Learning to Initialize Neural Networks

paper_content:

The primary goal of inductive learning is to generalize well - that is, induce a function that accurately produces the correct output for future inputs. Hansen and Salamon showed that, under certain assumptions, combining the predictions of several separately trained neural networks will improve generalization. One of their key assumptions is that the individual networks should be independent in the errors they produce. In the standard way of performing backpropagation this assumption may be violated, because the standard procedure is to initialize network weights in the region of weight space near the origin. This means that backpropagation's gradient-descent search may only reach a small subset of the possible local minima. In this paper we present an approach to initializing neural networks that uses competitive learning to intelligently create networks that are originally located far from the origin of weight space, thereby potentially increasing the set of reachable local minima. We report experiments on two real-world datasets where combinations of networks initialized with our method generalize better than combinations of networks initialized the traditional way.
---
paper_title: Artificial neural network ensemble-based land-cover classifiers using MODIS data

paper_content:

Terra and Aqua, two satellites launched by the NASA-centered International Earth Observing System project, house MODIS (moderate resolution imaging spectroradiometer) sensors. Moderate-resolution remote sensing allows the quantifying of land-surface type and extent, which can be used to monitor changes in land cover and land use for extended periods of time. In this article, we propose land-surface classification by applying an ensemble technique based on fault masking among individual classifiers in N-version programming. An N-version programming ensemble of artificial neural networks is created, in which the majority vote result is used to predict land-surface cover from MODIS data. It is shown by experiment that an N-version programming ensemble of neural networks greatly improves the classification error rate of land-cover type.
---
paper_title: Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning

paper_content:

Negative Correlation Learning (NCL) [CHECK END OF SENTENCE], [CHECK END OF SENTENCE] is a neural network ensemble learning algorithm which introduces a correlation penalty term to the cost function of each individual network so that each neural network minimizes its mean-square-error (MSE) together with the correlation. This paper describes NCL in detail and observes that the NCL corresponds to training the entire ensemble as a single learning machine that only minimizes the MSE without regularization. This insight explains that NCL is prone to overfitting the noise in the training set. The paper analyzes this problem and proposes the multiobjective regularized negative correlation learning (MRNCL) algorithm which incorporates an additional regularization term for the ensemble and uses the evolutionary multiobjective algorithm to design ensembles. In MRNCL, we define the crossover and mutation operators and adopt nondominated sorting algorithm with fitness sharing and rank-based fitness assignment. The experiments on synthetic data as well as real-world data sets demonstrate that MRNCL achieves better performance than NCL, especially when the noise level is nontrivial in the data set. In the experimental discussion, we give three reasons why our algorithm outperforms others.
---
paper_title: Regularized Negative Correlation Learning for Neural Network Ensembles

paper_content:

Negative correlation learning (NCL) is a neural network ensemble learning algorithm that introduces a correlation penalty term to the cost function of each individual network so that each neural network minimizes its mean square error (MSE) together with the correlation of the ensemble. This paper analyzes NCL and reveals that the training of NCL (when ? = 1) corresponds to training the entire ensemble as a single learning machine that only minimizes the MSE without regularization. This analysis explains the reason why NCL is prone to overfitting the noise in the training set. This paper also demonstrates that tuning the correlation parameter ? in NCL by cross validation cannot overcome the overfitting problem. The paper analyzes this problem and proposes the regularized negative correlation learning (RNCL) algorithm which incorporates an additional regularization term for the whole ensemble. RNCL decomposes the ensemble's training objectives, including MSE and regularization, into a set of sub-objectives, and each sub-objective is implemented by an individual neural network. In this paper, we also provide a Bayesian interpretation for RNCL and provide an automatic algorithm to optimize regularization parameters based on Bayesian inference. The RNCL formulation is applicable to any nonlinear estimator minimizing the MSE. The experiments on synthetic as well as real-world data sets demonstrate that RNCL achieves better performance than NCL, especially when the noise level is nontrivial in the data set.
---
paper_title: The random subspace method for constructing decision forests

paper_content:

Much of previous attention on decision trees focuses on the splitting criteria and optimization of tree sizes. The dilemma between overfitting and achieving maximum accuracy is seldom resolved. A method to construct a decision tree based classifier is proposed that maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity. The classifier consists of multiple trees constructed systematically by pseudorandomly selecting subsets of components of the feature vector, that is, trees constructed in randomly chosen subspaces. The subspace method is compared to single-tree classifiers and other forest construction methods by experiments on publicly available datasets, where the method's superiority is demonstrated. We also discuss independence between trees in a forest and relate that to the combined classification accuracy.
---
paper_title: Random Subspace Ensembles for fMRI Classification

paper_content:

Classification of brain images obtained through functional magnetic resonance imaging (fMRI) poses a serious challenge to pattern recognition and machine learning due to the extremely large feature-to-instance ratio. This calls for revision and adaptation of the current state-of-the-art classification methods. We investigate the suitability of the random subspace (RS) ensemble method for fMRI classification. RS samples from the original feature set and builds one (base) classifier on each subset. The ensemble assigns a class label by either majority voting or averaging of output probabilities. Looking for guidelines for setting the two parameters of the method-ensemble size and feature sample size-we introduce three criteria calculated through these parameters: usability of the selected feature sets, coverage of the set of ?important? features, and feature set diversity. Optimized together, these criteria work toward producing accurate and diverse individual classifiers. RS was tested on three fMRI datasets from single-subject experiments: the Haxby data (Haxby, 2001.) and two datasets collected in-house. We found that RS with support vector machines (SVM) as the base classifier outperformed single classifiers as well as some of the most widely used classifier ensembles such as bagging, AdaBoost, random forest, and rotation forest. The closest rivals were the single SVM and bagging of SVM classifiers. We use kappa-error diagrams to understand the success of RS.
---
paper_title: Solving Multiclass Learning Problems via Error-Correcting Output Codes

paper_content:

Multiclass learning problems involve finding a definition for an unknown function f(x) whose range is a discrete set containing k > 2 values (i.e., k "classes"). The definition is acquired by studying collections of training examples of the form (xi, f(xi)). Existing approaches to multiclass learning problems include direct application of multiclass algorithms such as the decision-tree algorithms C4.5 and CART, application of binary concept learning algorithms to learn individual binary functions for each of the k classes, and application of binary concept learning algorithms with distributed output representations. This paper compares these three approaches to a new technique in which error-correcting codes are employed as a distributed output representation. We show that these output representations improve the generalization performance of both C4.5 and backpropagation on a wide range of multiclass learning tasks. We also demonstrate that this approach is robust with respect to changes in the size of the training sample, the assignment of distributed representations to particular classes, and the application of overfitting avoidance techniques such as decision-tree pruning. Finally, we show that--like the other methods--the error-correcting code technique can provide reliable class probability estimates. Taken together, these results demonstrate that error-correcting output codes provide a general-purpose method for improving the performance of inductive learning programs on multiclass problems.
---
paper_title: Class-switching Neural Network Ensembles

paper_content:

This article investigates the properties of class-switching ensembles composed of neural networks and compares them to class-switching ensembles of decision trees and to standard ensemble learning methods, such as bagging and boosting. In a class-switching ensemble, each learner is constructed using a modified version of the training data. This modification consists in switching the class labels of a fraction of training examples that are selected at random from the original training set. Experiments on 20 benchmark classification problems, including real-world and synthetic data, show that class-switching ensembles composed of neural networks can obtain significant improvements in the generalization accuracy over single neural networks and bagging and boosting ensembles. Furthermore, it is possible to build medium-sized ensembles (~200 networks) whose classification performance is comparable to larger class-switching ensembles (~1000 learners) of unpruned decision trees.
---
paper_title: Influence of unstable patterns in layered cluster oriented ensemble classifier

paper_content:

In this paper, we have investigated the influence of cluster instability on the performance of layered cluster oriented ensemble classifier. The final contents of clusters in some clustering algorithms like k-means depend on the initialization of clustering parameters like cluster centres. Layered cluster oriented ensemble classifier is based on this philosophy where the base classifiers are trained on clusters generated at multiple layers from random initialization of cluster centres. As the data is clustered into multiple layers some patterns move between clusters (unstable patterns). This instability of patterns brings in diversity among the base classifiers that in turn influences the accuracy of the ensemble classifier. There is thus a connection between the instability of the patterns and the accuracy of layered cluster oriented ensemble classifier. The research presented in this paper aims to find this connection by investigating the influence of unstable patterns on the overall ensemble classifier accuracy as well as diversity among the base classifiers. We have provided results from a number of experiments to quantify this influence.
---
paper_title: Non-uniform layered clustering for ensemble classifier generation and optimality

paper_content:

In this paper we present an approach to generate ensemble of classifiers using non-uniform layered clustering. In the proposed approach the dataset is partitioned into variable number of clusters at different layers. A set of base classifiers is trained on the clusters at different layers. The decision on a pattern at each layer is obtained from the classifier trained on the nearest cluster and the decisions from the different layers are fused using majority voting to obtain the final verdict. The proposed approach provides a mechanism to obtain the optimal number of layers and clusters using a Genetic Algorithm. Clustering identifies difficult-to-classify patterns and layered non-uniform clustering approach brings in diversity among the base classifiers at different layers. The proposed method performs relatively better than the other state-of-art ensemble classifier generation methods as evidenced from the experimental results.
---
paper_title: CLUSTER BASED ENSEMBLE CLASSIFIER GENERATION BY JOINT OPTIMIZATION OF ACCURACY AND DIVERSITY

paper_content:

This paper presents an algorithm to generate ensemble classifier by joint optimization of accuracy and diversity. It is expected that the base classifiers in an ensemble are accurate and diverse (i.e., complementary in terms of errors) among each other for the ensemble classifier to be more accurate. We adopt a multi-objective evolutionary algorithm (MOEA) for joint optimization of accuracy and diversity on our recently developed nonuniform layered cluster oriented ensemble classifier (NULCOEC). In NULCOEC, the data set is partitioned into a variable number of clusters at different layers. Base classifiers are then trained on the clusters at different layers. The performance of NULCOEC is a function of the vector of the number of layers and clusters. The research presented in this paper investigates the implication of applying MOEA to generate NULCOEC. Accuracy and diversity of the ensemble classifier is expressed as a function of layers and clusters. A MOEA then searches for the combination of layers and clusters to obtain the nondominated set of (accuracy, diversity). We have obtained the results of single objective optimization (i.e., optimizing either accuracy or diversity) and compared them with the results of MOEA on sixteen UCI data sets. The results show that the MOEA can improve the performance of ensemble classifier.
---
paper_title: Clustering based multi-label classification for image annotation and retrieval

paper_content:

This paper presents a novel multi-label classification framework for domains with large numbers of labels. Automatic image annotation is such a domain, as the available semantic concepts are typically hundreds. The proposed framework comprises an initial clustering phase that breaks the original training set into several disjoint clusters of data. It then trains a multi-label classifier from the data of each cluster. Given a new test instance, the framework first finds the nearest cluster and then applies the corresponding model. Empirical results using two clustering algorithms, four multi-label classification algorithms and three image annotation data sets suggest that the proposed approach can improve the performance and reduce the training time of standard multi-label classification algorithms, particularly in the case of large number of labels.
---
paper_title: Space decomposition in data mining - a clustering approach

paper_content:

Decomposition may divide the database horizontally (subsets of rows or tuples) or vertically. It may be aimed at minimizing space and time needed for the classification of a dataset (e.g. sampling, windowing) or rather attempt to improve accuracy (e.g. bagging, boosting). This paper presents a horizontal space-decomposition algorithm, exploiting the K-means clustering algorithm. It is aimed at decreasing error rate compared to the simple classifier embedded in it while being rather understandable.
---
paper_title: Cluster-space classification: a fast k-nearest neighbour classification for remote sensing hyperspectral data

paper_content:

In this paper a fast k-nearest neighbour (k-NN) algorithm is presented which combines k-NN with a cluster-space data representation. Implementation of the algorithm is easier and classification time can be significantly reduced. Results from tests carried out with a Hyperion data set demonstrate that the simplification has little effect on classification performance and yet efficiency is greatly improved.
---
paper_title: Novel Layered Clustering-Based Approach for Generating Ensemble of Classifiers

paper_content:

This paper introduces a novel concept for creating an ensemble of classifiers. The concept is based on generating an ensemble of classifiers through clustering of data at multiple layers. The ensemble classifier model generates a set of alternative clustering of a dataset at different layers by randomly initializing the clustering parameters and trains a set of base classifiers on the patterns at different clusters in different layers. A test pattern is classified by first finding the appropriate cluster at each layer and then using the corresponding base classifier. The decisions obtained at different layers are fused into a final verdict using majority voting. As the base classifiers are trained on overlapping patterns at different layers, the proposed approach achieves diversity among the individual classifiers. Identification of difficult-to-classify patterns through clustering as well as achievement of diversity through layering leads to better classification results as evidenced from the experimental results.
---
paper_title: Soft partitions lead to better learned ensembles

paper_content:

Ensembles of classifiers often provide better classification accuracy than a single classifier. One approach to creating ensembles is to create different subsets of the training data. We present a method of creating ensembles of classifiers by partitioning the dataset into regions using clustering. Learners are assigned to each region and the ensemble classification occurs by querying the learned classifier. The first strategy considered for partitioning the training set is to generate a hard, non-overlapping partition. This approach is shown to perform worse than a single classifier using the entire training set. However, the use of soft partitions significantly improves the overall ensemble performance. Three different methods of creating soft partitions are considered: a simple distance ratio, and both the fuzzy c-means and possibilistic c-means membership functions. All three methods are found to improve overall classifier performance beyond hard partitioning and often perform better than the base classifier using the entire training set. Experiments on six datasets illustrate the improved accuracy from creating ensembles on soft partitions of data.
---
paper_title: Random Forests

paper_content:

Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, aaa, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.
---
paper_title: Bagging Predictors

paper_content:

Bagging predictors is a method for generating multiple versions of a predictor and using these to get an aggregated predictor. The aggregation averages over the versions when predicting a numerical outcome and does a plurality vote when predicting a class. The multiple versions are formed by making bootstrap replicates of the learning set and using these as new learning sets. Tests on real and simulated data sets using classification and regression trees and subset selection in linear regression show that bagging can give substantial gains in accuracy. The vital element is the instability of the prediction method. If perturbing the learning set can cause significant changes in the predictor constructed, then bagging can improve accuracy.
---
paper_title: Pasting Small Votes for Classification in Large Databases and On-Line

paper_content:

Many databases have grown to the point where they cannot fit into the fast memory of even large memory machines, to say nothing of current workstations. If what we want to do is to use these data bases to construct predictions of various characteristics, then since the usual methods require that all data be held in fast memory, various work-arounds have to be used. This paper studies one such class of methods which give accuracy comparable to that which could have been obtained if all data could have been held in core and which are computationally fast. The procedure takes small pieces of the data, grows a predictor on each small piece and then pastes these predictors together. A version is given that scales up to terabyte data sets. The methods are also applicable to on-line learning.
---
paper_title: Multiple classifier system for automated quality assessment of marine sensor data

paper_content:

Numerous sources of uncertainty are associated with the data acquisition process in marine sensor networks. It is thus required to assure that the data quality of sensors is fit for the intended purpose. We propose a supervised learning framework to infer the quality of sensor observations online. A problem with using supervised classification in quality assessment is that sensor observations from the class of uncertain data will be far out-weighed by class instances of good data quality. This leads to an imbalanced data set, which can potentially reduce the classification accuracy of uncertain data. A multiple classifier (or ensemble classifier) system is proposed to deal with this problem. Training sets are randomly undersampled to develop training subsets with balanced class membership. The process is repeated to produce multiple balanced training subsets. Individual classifiers are then trained upon each of these balanced data sets. The quality classifications from the individual classifiers are then combined using majority voting. We evaluated the ensemble classifier system using conductivity and temperature sensors from the Tasmanian Marine Analysis Network (TasMAN). Experiments demonstrate that the ensemble classifier balances the classification accuracy of the majority and minority classes, achieving a higher overall classification accuracy than its constituent classifiers.
---
paper_title: A Novel Machine Learning Approach Toward Quality Assessment of Sensor Data

paper_content:

A novel machine learning approach to assess the quality of sensor data using an ensemble classification framework is presented in this paper. The quality of sensor data is indicated by discrete quality flags that indicate the level of uncertainty associated with a sensor reading. Depending on the domain and the problem under consideration, the level of uncertainty is different and thus unsupervised methods like outlier detection fails to match the expectation. The quality flags are normally assigned by domain experts. Considering the volume of sensor data, manual assignment is a laborious task and subject to human error. Given a representative set of labelled data, a supervised classification approach is thus a feasible alternative. The nature of sensor data, however, poses some challenges to the classification task. Data of dubious quality exists in such data sets with very small frequency leading to the class imbalance problem. We thus adopt a cluster oriented sampling approach to address the imbalance issue. In addition, it is beneficial to train multiple classifiers to improve the overall classification accuracy. We thus produce multiple under-sampled training sets using cluster oriented sampling and train base classifiers on each of them. Decisions produced by the base classifiers are fused into a single decision using majority voting. We have evaluated the proposed ensemble classification framework by assessing the quality of marine sensor data obtained from sensors situated at Sullivans Cove, Hobart, Australia. Experimental results reveal that the proposed framework agrees with expert judgement with high accuracy and achieves superior classification performance than other state-of-the-art approaches.
---
paper_title: Multiple classifier system for automated quality assessment of marine sensor data

paper_content:

Numerous sources of uncertainty are associated with the data acquisition process in marine sensor networks. It is thus required to assure that the data quality of sensors is fit for the intended purpose. We propose a supervised learning framework to infer the quality of sensor observations online. A problem with using supervised classification in quality assessment is that sensor observations from the class of uncertain data will be far out-weighed by class instances of good data quality. This leads to an imbalanced data set, which can potentially reduce the classification accuracy of uncertain data. A multiple classifier (or ensemble classifier) system is proposed to deal with this problem. Training sets are randomly undersampled to develop training subsets with balanced class membership. The process is repeated to produce multiple balanced training subsets. Individual classifiers are then trained upon each of these balanced data sets. The quality classifications from the individual classifiers are then combined using majority voting. We evaluated the ensemble classifier system using conductivity and temperature sensors from the Tasmanian Marine Analysis Network (TasMAN). Experiments demonstrate that the ensemble classifier balances the classification accuracy of the majority and minority classes, achieving a higher overall classification accuracy than its constituent classifiers.
---
paper_title: Ensemble Feature Ranking for Shellfish Farm Closure Cause Identification

paper_content:

Shellfish farms must be closed if there is suspected contamination during production to avoid serious health hazards. The authorities monitor a number of environmental and water quality variables through a set of sensors to check the health of shellfish farms and to decide on the closure of the farms. The research presented in this paper aims to develop an ensemble feature ranking algorithm to identify the cause of closure. We have presented and analysed the results obtained using the proposed algorithm to demonstrate its effectiveness.
---
paper_title: A Novel Machine Learning Approach Toward Quality Assessment of Sensor Data

paper_content:

A novel machine learning approach to assess the quality of sensor data using an ensemble classification framework is presented in this paper. The quality of sensor data is indicated by discrete quality flags that indicate the level of uncertainty associated with a sensor reading. Depending on the domain and the problem under consideration, the level of uncertainty is different and thus unsupervised methods like outlier detection fails to match the expectation. The quality flags are normally assigned by domain experts. Considering the volume of sensor data, manual assignment is a laborious task and subject to human error. Given a representative set of labelled data, a supervised classification approach is thus a feasible alternative. The nature of sensor data, however, poses some challenges to the classification task. Data of dubious quality exists in such data sets with very small frequency leading to the class imbalance problem. We thus adopt a cluster oriented sampling approach to address the imbalance issue. In addition, it is beneficial to train multiple classifiers to improve the overall classification accuracy. We thus produce multiple under-sampled training sets using cluster oriented sampling and train base classifiers on each of them. Decisions produced by the base classifiers are fused into a single decision using majority voting. We have evaluated the proposed ensemble classification framework by assessing the quality of marine sensor data obtained from sensors situated at Sullivans Cove, Hobart, Australia. Experimental results reveal that the proposed framework agrees with expert judgement with high accuracy and achieves superior classification performance than other state-of-the-art approaches.
---
paper_title: Effect of ensemble classifier composition on offline cursive character recognition

paper_content:

In this paper we present novel ensemble classifier architectures and investigate their influence for offline cursive character recognition. Cursive characters are represented by feature sets that portray different aspects of character images for recognition purposes. The recognition accuracy can be improved by training ensemble of classifiers on the feature sets. Given the feature sets and the base classifiers, we have developed multiple ensemble classifier compositions under four architectures. The first three architectures are based on the use of multiple feature sets whereas the fourth architecture is based on the use of a unique feature set. Type-1 architecture is composed of homogeneous base classifiers and Type-2 architecture is constructed using heterogeneous base classifiers. Type-3 architecture is based on hierarchical fusion of decisions. In Type-4 architecture a unique feature set is learned by a set of homogeneous base classifiers with different learning parameters. The experimental results demonstrate that the recognition accuracy achieved using the proposed ensemble classifier (with best composition of base classifiers and feature sets) is better than the existing recognition accuracies for offline cursive character recognition.
---
paper_title: Benthic Habitat Mapping from Seabed Images using Ensemble of Color, Texture, and Edge Features

paper_content:

Abstract In this paper we present a novel approach to produce benthic habitat maps from sea floor images in Derwent estuary. We have developed a step–by–step segmentation method to separate sea–grass, sand, and rock from the sea floor image. The sea–grass was separated first using color filtering. The remaining image was classified into rock and sand based on color, texture, and edge features. The features were fed into an ensemble classifier to produce better classification results. The base classifiers in the ensemble were made complementary by changing the weight (i.e. cost of misclassification) of the classes. The habitat maps were produced for three regions in Derwent estuary. Experimental results demonstrate that the proposed method can indentify different objects and produce habitat maps from the sea–floor images with very high accuracy.
---
paper_title: ALGAE GROWTH PREDICTION THROUGH IDENTIFICATION OF INFLUENTIAL ENVIRONMENTAL VARIABLES: A MACHINE LEARNING APPROACH

paper_content:

In this paper, we present an approach for predicting algae growth through the selection of influential environmental variables. Chlorophyll a is considered to be an indicator for algal biomass and we predict this as a proxy for algae growth. Environmental variables like water temperature, salinity, etc. have influence upon algae growth. Depending on the geographic location, the influence of these environmental variables will vary. Given a set of relevant environmental variables we perform feature selection using a number of algorithms to identify the variables relevant to the growth. We have developed an influence matrix-based approach to select the relevant features. The selected features are then used for predicting algae growth using different regression algorithms to identify their relative strength. The approach is tested on the algae data of Derwent estuary in Tasmania. The experimental results demonstrate that the accuracy of algae growth prediction with influence matrix-based feature selection is superior to using all the features.
---
paper_title: Dealing with missing sensor values in predicting shellfish farm closure

paper_content:

Shellfish farms need to be closed from harvesting when the water body is contaminated to avoid a serious health hazard. We have designed a sensor network framework to monitor a number of water quality variables to check the health of shellfish farms and predict closure if hazardous. Because of the uncertainty associated with the data acquisition process, a full set of sensor values are not always available for decision making purposes. The prediction system thus needs to deal with missing values. Statistical approaches are commonly used to generate an artificial value to approximate a missing sensor reading and predictions are made on the then complete set of sensor values. In this paper we present a new method that is capable of making predictions without making artificial approximations of missing values. The idea is to train a set of classifiers on different subsets of sensor values. Given a set of available sensor values, a prediction is made by the classifier trained on the corresponding set of sensor values. We have evaluated the system on the data obtained from a number of shellfish farms in Tasmania. Experimental results demonstrate that the proposed method to deal with missing values can predict closures with high accuracy.
---

Title: Ensemble Classifiers and Their Applications: A Review

Section 1: INTRODUCTION
Description 1: Introduce the concept of supervised classification problems and the motivation behind using ensemble classifiers over individual classifiers.

Section 2: BASE CLASSIFIERS
Description 2: Provide an overview of commonly used base classifiers like Neural Networks, Support Vector Machines, and k-NN classifiers, including their training and testing processes.

Section 3: ENSEMBLE CLASSIFIERS
Description 3: Discuss different methods for generating ensemble classifiers, focusing on the six broad categories of manipulations used.

Section 4: Ensemble Classifier Generation by Manipulation of the Training Parameters
Description 4: Explain how diversity in base classifiers can be achieved by manipulating training parameters and provide relevant examples.

Section 5: Ensemble Classifier Generation by Manipulation of the Error Function
Description 5: Describe methods that create ensemble classifiers by modifying the error functions of base classifiers.

Section 6: Ensemble Classifier Generation by Manipulation of The Feature Space
Description 6: Detail how manipulating the input feature space can generate diverse base classifiers and mention specific techniques.

Section 7: Ensemble Classifier Generation by Manipulation of the Output Labels
Description 7: Cover the techniques of ensemble generation by altering the output targets, including specific methodologies.

Section 8: Ensemble Classifier Generation by Clustering
Description 8: Discuss the creation of ensemble classifiers through clustering methods and the associated benefits and challenges.

Section 9: Ensemble Classifier Generation by Manipulation of the Training Patterns
Description 9: Elaborate on the largest set of methods for generating ensemble classifiers by creating different subsets of training patterns, with examples like bagging and boosting.

Section 10: ENSEMBLE CLASSIFIER APPLICATIONS
Description 10: Present various applications of ensemble classifiers, focusing on specific domains and case studies.

Section 11: Sensor Data Quality Assessment
Description 11: Showcase the use of ensemble classifiers in assessing the quality of sensor data, addressing challenges like class imbalance.

Section 12: Shellfish Farm Closure Prediction and Cause Identification
Description 12: Discuss the application of ensemble classifiers in predicting and identifying causes for shellfish farm closures based on environmental monitoring data.

Section 13: Handwriting Recognition
Description 13: Review the application of ensemble classifiers in offline cursive character recognition, highlighting different ensemble architectures and their impact.

Section 14: Benthic Habitat Mapping
Description 14: Explain the use of ensemble classifiers in generating benthic habitat maps from sea floor images, detailing the segmentation and classification processes.

Section 15: Dealing with Missing Sensor Data
Description 15: Describe approaches for handling missing sensor data using ensemble classifiers and demonstrate their effectiveness in predictions, such as algae growth.

Section 16: CONCLUSIONS
Description 16: Summarize the various ensemble classifier generation methods and their applications discussed in the paper, and outline future research directions.