Collective decision efficiency and optimal voting mechanisms: A comprehensive overview for multi-classifier models
---
paper_title: Optimal linear combination of neural networks for improving classification performance

paper_content:

This paper presents a new method for linearly combining multiple neural network classifiers based on the statistical pattern recognition theory. In our approach, several neural networks are first selected based on which works best for each class in terms of minimizing classification errors. Then, they are linearly combined to form an ideal classifier that exploits the strengths of the individual classifiers. In this approach, the minimum classification error criterion is utilized to estimate the optimal linear weights. In this formulation, because the classification decision rule is incorporated into the cost function, a more suitable better combination of weights for the classification objective could be obtained. Experimental results using artificial and real data sets show that the proposed method can construct a better combined classifier that outperforms the best single classifier in terms of overall classification errors for test data.
---
paper_title: Rank and response combination from confusion matrix data

paper_content:

Abstract The use of prior behavior of a classifier, as measured by the confusion matrix, can yield useful information for merging multiple classifiers. In particular, response vectors can be estimated and a ranking of possible classes can be produced which can allow Borda type reconciliation methods to be applied. A combination of real data and the simulation of multiple classifiers is used to evaluate this idea, and to compare with 11 other classifier combination techniques. Millions of classifications were used in the evaluation.
---
paper_title: Classifier combination based on confidence transformation

paper_content:

This paper investigates the effects of confidence transformation in combining multiple classifiers using various combination rules. The combination methods were tested in handwritten digit recognition by combining varying classifier sets. The classifier outputs are transformed to confidence measures by combining three scaling functions (global normalization, Gaussian density modeling, and logistic regression) and three confidence types (linear, sigmoid, and evidence). The combination rules include fixed rules (sum-rule, product-rule, median-rule, etc.) and trained rules (linear discriminants and weighted combination with various parameter estimation techniques). The experimental results justify that confidence transformation benefits the combination performance of either fixed rules or trained rules. Trained rules mostly outperform fixed rules, especially when the classifier set contains weak classifiers. Among the trained rules, the support vector machine with linear kernel (linear SVM) performs best while the weighted combination with optimized weights performs comparably well. I have also attempted the joint optimization of confidence parameters and combination weights but its performance was inferior to that of cascaded confidence transformation-combination. This justifies that the cascaded strategy is a right way of multiple classifier combination.
---
paper_title: Classifier combination based on confidence transformation

paper_content:

This paper investigates the effects of confidence transformation in combining multiple classifiers using various combination rules. The combination methods were tested in handwritten digit recognition by combining varying classifier sets. The classifier outputs are transformed to confidence measures by combining three scaling functions (global normalization, Gaussian density modeling, and logistic regression) and three confidence types (linear, sigmoid, and evidence). The combination rules include fixed rules (sum-rule, product-rule, median-rule, etc.) and trained rules (linear discriminants and weighted combination with various parameter estimation techniques). The experimental results justify that confidence transformation benefits the combination performance of either fixed rules or trained rules. Trained rules mostly outperform fixed rules, especially when the classifier set contains weak classifiers. Among the trained rules, the support vector machine with linear kernel (linear SVM) performs best while the weighted combination with optimized weights performs comparably well. I have also attempted the joint optimization of confidence parameters and combination weights but its performance was inferior to that of cascaded confidence transformation-combination. This justifies that the cascaded strategy is a right way of multiple classifier combination.
---
paper_title: Indirect voting systems: Banzhaf numbers, majority functions and collective competence

paper_content:

Abstract This paper discusses the reliability of group judgment in the context of indirect voting systems and the Condorcet jury theorem. To describe such systems, composite majority functions are introduced and their properties studied. Associated Banzhaf numbers appear as a natural measure of decisional power. The paper examines how collective judgmental competence depends on structural properties of the decision making system. Formulas are developed to measure the loss (or gain) in judgmental competence due to simple changes in system structure, such as overlapping membership.
---
paper_title: Weighted Voting, Multicameral Representation, and Power

paper_content:

Abstract A simple game ( P, W ) can serve as a model of a voting system in which an alternative is pitted against the status quo. In what follows, we investigate the following three aspects of such games as they apply to four real-world examples of voting systems: a characterization of weighted voting systems in terms of the ways in which coalitions can gain or lose by trading players; the application of a graph-theoretic notion of dimension to simple games and voting systems; and the consideration of a way to measure the power of a player as an interval of real numbers. American Mathematical Society Classification Numbers: 90A28, 90D80, 05C65. Journal of Economic Literature Classification Number: C71.
---
paper_title: Indirect voting systems: Banzhaf numbers, majority functions and collective competence

paper_content:

Abstract This paper discusses the reliability of group judgment in the context of indirect voting systems and the Condorcet jury theorem. To describe such systems, composite majority functions are introduced and their properties studied. Associated Banzhaf numbers appear as a natural measure of decisional power. The paper examines how collective judgmental competence depends on structural properties of the decision making system. Formulas are developed to measure the loss (or gain) in judgmental competence due to simple changes in system structure, such as overlapping membership.
---
paper_title: Indirect voting systems: Banzhaf numbers, majority functions and collective competence

paper_content:

Abstract This paper discusses the reliability of group judgment in the context of indirect voting systems and the Condorcet jury theorem. To describe such systems, composite majority functions are introduced and their properties studied. Associated Banzhaf numbers appear as a natural measure of decisional power. The paper examines how collective judgmental competence depends on structural properties of the decision making system. Formulas are developed to measure the loss (or gain) in judgmental competence due to simple changes in system structure, such as overlapping membership.
---
paper_title: Weighted Voting, Multicameral Representation, and Power

paper_content:

Abstract A simple game ( P, W ) can serve as a model of a voting system in which an alternative is pitted against the status quo. In what follows, we investigate the following three aspects of such games as they apply to four real-world examples of voting systems: a characterization of weighted voting systems in terms of the ways in which coalitions can gain or lose by trading players; the application of a graph-theoretic notion of dimension to simple games and voting systems; and the consideration of a way to measure the power of a player as an interval of real numbers. American Mathematical Society Classification Numbers: 90A28, 90D80, 05C65. Journal of Economic Literature Classification Number: C71.
---
paper_title: Rank and response combination from confusion matrix data

paper_content:

Abstract The use of prior behavior of a classifier, as measured by the confusion matrix, can yield useful information for merging multiple classifiers. In particular, response vectors can be estimated and a ranking of possible classes can be produced which can allow Borda type reconciliation methods to be applied. A combination of real data and the simulation of multiple classifiers is used to evaluate this idea, and to compare with 11 other classifier combination techniques. Millions of classifications were used in the evaluation.
---
paper_title: Indirect voting systems: Banzhaf numbers, majority functions and collective competence

paper_content:

Abstract This paper discusses the reliability of group judgment in the context of indirect voting systems and the Condorcet jury theorem. To describe such systems, composite majority functions are introduced and their properties studied. Associated Banzhaf numbers appear as a natural measure of decisional power. The paper examines how collective judgmental competence depends on structural properties of the decision making system. Formulas are developed to measure the loss (or gain) in judgmental competence due to simple changes in system structure, such as overlapping membership.
---
paper_title: Weighted Voting, Multicameral Representation, and Power

paper_content:

Abstract A simple game ( P, W ) can serve as a model of a voting system in which an alternative is pitted against the status quo. In what follows, we investigate the following three aspects of such games as they apply to four real-world examples of voting systems: a characterization of weighted voting systems in terms of the ways in which coalitions can gain or lose by trading players; the application of a graph-theoretic notion of dimension to simple games and voting systems; and the consideration of a way to measure the power of a player as an interval of real numbers. American Mathematical Society Classification Numbers: 90A28, 90D80, 05C65. Journal of Economic Literature Classification Number: C71.
---
paper_title: Weighted Voting, Multicameral Representation, and Power

paper_content:

Abstract A simple game ( P, W ) can serve as a model of a voting system in which an alternative is pitted against the status quo. In what follows, we investigate the following three aspects of such games as they apply to four real-world examples of voting systems: a characterization of weighted voting systems in terms of the ways in which coalitions can gain or lose by trading players; the application of a graph-theoretic notion of dimension to simple games and voting systems; and the consideration of a way to measure the power of a player as an interval of real numbers. American Mathematical Society Classification Numbers: 90A28, 90D80, 05C65. Journal of Economic Literature Classification Number: C71.
---
paper_title: The Network of Weighted Majority Rules and Weighted Majority Games

paper_content:

Abstract The network organizes in space the weighted majority rules (WMR) and weighted majority games (WMG). The WMRs are the potentially optimal decision rules in uncertain dichotomous choice situations. The WMGs are the well-known subgroup of coalitional games. The nodes are the WMRs or the WMGs, and the directed edges are the decision profiles of the decision makers or the coalitions among the players, respectively. For any group of decision makers, the network yields the ranking of all the WMRs by their efficiency; for any group of players, it yields the differences among the WMGs in terms of winning and losing coalitions. Journal of Economic Literature Classification Numbers: C45, C70, C71, D81.
---
paper_title: Optimal decision rules for fixed-size committees in polychotomous choice situations

paper_content:

Abstract. This paper derives optimal decision rules for fixed-size committees in polychotomous choice situations. Earlier studies focus on the dichotomous choice model and thus the present extension broadens the scope of applications of the theory of collective decision making.
---
paper_title: Combining Pattern Classifiers: Methods and Algorithms

paper_content:

This book, which is wholly devoted to the subject of model combination, is divided into ten chapters. In addition to the first two introductory chapters, the book covers some of the following topics: multiple classifier systems; combination methods when the base classifier outputs are 0/1; methods when the outputs are continuous, e.g., posterior probabilities; methods for classifier selection; bagging and boosting; the theory of fixed combination rules; and the concept of diversity. Overall, it is a very well-written monograph. It explains and analyzes different approaches comparatively so that the reader can see how they are similar and how they differ. The literature survey is extensive. The MATLAB code for many methods is given in chapter appendices allowing readers to play with the explained methods or apply them quickly to their own data. The book is a must-read for researchers and practitioners alike.
---
paper_title: The network of weighted majority rules and its geometric realizations

paper_content:

Abstract In a previous work the first author considered the network G ( n , p ) of weighted majority rules (WMR) for n decision makers whose competencies are given by their probabilities p =( p 1 ,…, p n ) of making a correct decision. On this paper we consider chains of decision profiles, which must occur in G ( n , p ) in a fixed order, and show that they can be mapped onto straight lines in a low-dimensional geometric realization. The minimal number of directions which must used to separate all edges is given as the chromatic number of a certain incidence graph. We also define degenerate networks in which several nodes coalesce.
---

Title: Collective decision efficiency and optimal voting mechanisms: A comprehensive overview for multi-classifier models

Section 1: Introduction
Description 1: Provide an overview of the emergence of multi-classifier methods and the need for combination rules.

Section 2: Rank and response combination from classifier confidence transformation
Description 2: Discuss different types of classifiers and how their outputs can be transformed and combined.

Section 3: Cascaded versus joint parameter optimization in combination schemes
Description 3: Explain the different combination rules and discuss cascaded versus joint optimization of parameters.

Section 4: Elements of Game Theory
Description 4: Introduce the basics of game theory and its application to classifier combination schemes.

Section 5: Coalitions, Stable Sets and Indices of Power
Description 5: Discuss coalition theory and indices such as the Shapley and Banzhaf power indices.

Section 6: Collective competence and the Condorcet criterion
Description 6: Explain the Condorcet Jury Theorem and its application in evaluating voting systems.

Section 7: Optimal scoring rules and Condorcet efficiency
Description 7: Describe optimal scoring rules such as the Borda count and wBorda models to maximize Condorcet efficiency.

Section 8: Majority functions and Banzhaf numbers
Description 8: Discuss dichotomous choice situations and the application of majority functions and Banzhaf numbers.

Section 9: Weighted Majority Games and Weighted Majority Rules
Description 9: Explore the design and application of weighted majority games (WMG) and weighted majority rules (WMR).

Section 10: Weighted Majority Rules and Condorcet efficiency
Description 10: Examine the efficiency of WMRs in maximizing Condorcet efficiency through optimal weighting profiles and thresholds.

Section 11: Basic framework
Description 11: Realize and implement the game-theoretic approach and risk-based models within pattern recognition systems.

Section 12: Adaptive realizations of WMRs
Description 12: Discuss adaptive methods for WMRs that account for statistical class distribution properties.

Section 13: A link with the theory of SVM
Description 13: Connect SVM classifiers to the theory of WMRs and their implementation in coalition games.

Section 14: Conclusion and further work
Description 14: Summarize the key findings and propose areas for future research to extend the theoretical and practical aspects of the study.