Neuro – Fuzzy Rule Generation : Survey in Soft Computing Framework
---
paper_title: Knowledge-based connectionism for revising domain theories

paper_content:

A knowledge-based connectionist model for machine learning referred to as KBCNN is presented. In the KBCNN learning model, useful domain attributes and concepts are first identified and linked in a way consistent with initial domain knowledge, and then the links are weighted properly so as to maintain the semantics. Hidden units and additional connections may be introduced into this initial connectionist structure as appropriate. Then, this primitive structure evolves to minimize empirical error. The KBCNN learning model allows the theory learned or revised to be translated into the symbolic rule-based language that describes the initial theory. Thus, a domain theory can be pushed onto the network, revised empirically over time, and decoded in symbolic form. The domain of molecular genetics is used to demonstrate the validity of the KBCNN learning model and its superiority over related learning methods. >
---
paper_title: Rough fuzzy MLP: knowledge encoding and classification

paper_content:

A scheme of knowledge encoding in a fuzzy multilayer perceptron (MLP) using rough set-theoretic concepts is described. Crude domain knowledge is extracted from the data set in the form of rules. The syntax of these rules automatically determines the appropriate number of hidden nodes while the dependency factors are used in the initial weight encoding. The network is then refined during training. Results on classification of speech and synthetic data demonstrate the superiority of the system over the fuzzy and conventional versions of the MLP (involving no initial knowledge).
---
paper_title: Survey and critique of techniques for extracting rules from trained artificial neural networks

paper_content:

It is becoming increasingly apparent that, without some form of explanation capability, the full potential of trained artificial neural networks (ANNs) may not be realised. This survey gives an overview of techniques developed to redress this situation. Specifically, the survey focuses on mechanisms, procedures, and algorithms designed to insert knowledge into ANNs (knowledge initialisation), extract rules from trained ANNs (rule extraction), and utilise ANNs to refine existing rule bases (rule refinement). The survey also introduces a new taxonomy for classifying the various techniques, discusses their modus operandi, and delineates criteria for evaluating their efficacy.
---
paper_title: Knowledge-Based Artificial Neural Networks

paper_content:

Abstract Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN ( Knowledge-Based Artificial Neural Networks ) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific “domain theories”, represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.
---
paper_title: Foundations Of Neuro-Fuzzy Systems

paper_content:

From the Publisher: ::: Foundations of Neuro-Fuzzy Systems reflects the current trend in intelligent systems research towards the integration of neural networks and fuzzy technology. The authors demonstrate how a combination of both techniques enhances the performance of control, decision-making and data analysis systems. Smarter and more applicable structures result from marrying the learning capability of the neural network with the transparency and interpretability of the rule-based fuzzy system. Foundations of Neuro-Fuzzy Systems highlights the advantages of integration making it a valuable resource for graduate students and researchers in control engineering, computer science and applied mathematics. The authors' informed analysis of practical neuro-fuzzy applications will be an asset to industrial practitioners using fuzzy technology and neural networks for control systems, data analysis and optimization tasks.
---
paper_title: Neuro-Fuzzy Pattern Recognition: Methods in Soft Computing

paper_content:

From the Publisher: ::: The authors consolidate a wealth of information previously scattered in disparate articles, journals, and edited volumes, explaining both the theory of neuro-fuzzy computing and the latest methodologies for performing different pattern recognition tasks in the neuro-fuzzy network - classification, feature evaluation, rule generation, knowledge extraction, and hybridization. Special emphasis is given to the integration of neuro-fuzzy methods with rough sets and genetic algorithms (GAs) to ensure more efficient recognition systems.
---
paper_title: The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks

paper_content:

To date, the preponderance of techniques for eliciting the knowledge embedded in trained artificial neural networks (ANN's) has focused primarily on extracting rule-based explanations from feedforward ANN's. The ADT taxonomy for categorizing such techniques was proposed in 1995 to provide a basis for the systematic comparison of the different approaches. This paper shows that not only is this taxonomy applicable to a cross section of current techniques for extracting rules from trained feedforward ANN's but also how the taxonomy can be adapted and extended to embrace a broader range of ANN types (e,g., recurrent neural networks) and explanation structures. In addition we identify some of the key research questions in extracting the knowledge embedded within ANN's including the need for the formulation of a consistent theoretical basis for what has been, until recently, a disparate collection of empirical results.
---
paper_title: Are artificial neural networks black boxes

paper_content:

Artificial neural networks are efficient computing models which have shown their strengths in solving hard problems in artificial intelligence. They have also been shown to be universal approximators. Notwithstanding, one of the major criticisms is their being black boxes, since no satisfactory explanation of their behavior has been offered. In this paper, we provide such an interpretation of neural networks so that they will no longer be seen as black boxes. This is stated after establishing the equality between a certain class of neural nets and fuzzy rule-based systems. This interpretation is built with fuzzy rules using a new fuzzy logic operator which is defined after introducing the concept of f-duality. In addition, this interpretation offers an automated knowledge acquisition procedure.
---
paper_title: Approximations between fuzzy expert systems and neural networks

paper_content:

Abstract The fuzzy expert system we are concerned about in this paper is a rule-based fuzzy expert system using any method of approximate reasoning to evaluate the rules when given new data. In this paper we argue that: (1) any continuous fuzzy expert system may be approximated by a neural net; and (2) any continuous neural net (feedforward, multilayered) may be approximated by a fuzzy expert system. We show how to train the neural net and how to write down the rules in the fuzzy expert system.
---
paper_title: An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller

paper_content:

This paper describes an experiment on the “linguistic” synthesis of a controller for a model industrial plant (a steam engine). Fuzzy logic is used to convert heuristic control rules stated by a human operator into an automatic control strategy. The experiment was initiated to investigate the possibility of human interaction with a learning controller. However, the control strategy set up linguistically proved to be far better than expected in its own right, and the basic experiment of linguistic control synthesis in a non-learning controller is reported here.
---
paper_title: Neural fuzzy systems: a neuro-fuzzy synergism to intelligent systems

paper_content:

Neural Fuzzy Systems provides a comprehensive, up-to-date introduction to the basic theories of fuzzy systems and neural networks, as well as an exploration of how these two fields can be integrated to create Neural-Fuzzy Systems. It includes Matlab software, with a Neural Network Toolkit, and a Fuzzy System Toolkit.
---
paper_title: Computational Intelligence: An Introduction

paper_content:

From the Publisher: ::: Computational Intelligence: An Introduction consists of a highly readable and systematic exposure of the fundamentals of computational intelligence, along with the coherent presentation of sound and comprehensive analysis and design practices. Provides a balanced introduction to computational intelligence, emphasizing equally the important analysis and design aspects of the emerging technology; text is organized in a way that allows for the easy use of the book as a basic course material; presents a design-oriented approach toward the use of computational intelligence; organizes exercises and problems of different levels of difficulty following each chapter; and complete algorithms are presented in a structured fashion, easing understanding and implementation.
---
paper_title: On the equivalence of neural nets and fuzzy expert systems

paper_content:

Abstract We show, under the assumptions described in the paper, that: (1) we can approximate a neural net to any degree of accuracy using a fuzzy expert system; and conversely (2) we may approximate a fuzzy expert system to any degree of accuracy with a neural net.
---
paper_title: Hybrid neural nets can be fuzzy controllers and fuzzy expert systems

paper_content:

Abstract Given a discrete fuzzy expert system we show how to construct a hybrid neural net computationally identical to the fuzzy expert system. Given a Sugeno, Mamdani, or expert system type of controller we build a hybrid neural net that is computationally the same as the controller. This improves on previous results that show a (regular) neural net can approximate continuous fuzzy controllers and continuous fuzzy expert systems to any degree of accuracy.
---
paper_title: Neuro-Fuzzy Pattern Recognition: Methods in Soft Computing

paper_content:

From the Publisher: ::: The authors consolidate a wealth of information previously scattered in disparate articles, journals, and edited volumes, explaining both the theory of neuro-fuzzy computing and the latest methodologies for performing different pattern recognition tasks in the neuro-fuzzy network - classification, feature evaluation, rule generation, knowledge extraction, and hybridization. Special emphasis is given to the integration of neuro-fuzzy methods with rough sets and genetic algorithms (GAs) to ensure more efficient recognition systems.
---
paper_title: Functional Equivalence between Radial Basis Function Networks and Fuzzy Inference Systems

paper_content:

It is shown that, under some minor restrictions, the functional behavior of radial basis function networks (RBFNs) and that of fuzzy inference systems are actually equivalent. This functional equivalence makes it possible to apply what has been discovered (learning rule, representational power, etc.) for one of the models to the other, and vice versa. It is of interest to observe that two models stemming from different origins turn out to be functionally equivalent. >
---
paper_title: Bidirectional bridge between neural networks and linguistic knowledge: linguistic rule extraction and learning from linguistic rules

paper_content:

The aim of the paper is to clearly demonstrate that the relation between neural networks and linguistic knowledge is bidirectional. First we show how neural networks can be trained by linguistic knowledge, which is represented by a set of fuzzy rules. Next we show how linguistic knowledge can be extracted from neural networks. Then we discuss the design of classification systems when numerical data and linguistic knowledge are available. Since the relation between neural networks and linguistic knowledge is bidirectional, we can simultaneously utilize these two kinds of information for designing classification systems. For example, neural network-based classification systems can be trained by numerical data and linguistic knowledge. Fuzzy rule-based classification systems can be designed by linguistic knowledge and fuzzy rules extracted from neural networks. The performance of these classification systems is examined by computer simulations.
---
paper_title: Numerical relationships between neural networks, continuous functions, and fuzzy systems

paper_content:

Abstract In this survey paper we first discuss computational equivalence between continuous functions, regular neural nets, fuzzy controllers, and discrete fuzzy expert systems. We also discuss how to build hybrid neural nets numerically identical to a fuzzy controller or a discrete fuzzy expert system. We then discuss computational equivalence between continuous fuzzy functions, regular fuzzy neural nets, and fuzzy systems. We also discuss how to construct hybrid fuzzy neural nets to approximate continuous fuzzy functions.
---
paper_title: ANFIS: Adaptive-Network-Based Fuzzy Inference System

paper_content:

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >
---
paper_title: Fuzzy basis functions, universal approximation, and orthogonal least-squares learning.

paper_content:

Fuzzy systems are represented as series expansions of fuzzy basis functions which are algebraic superpositions of fuzzy membership functions. Using the Stone-Weierstrass theorem, it is proved that linear combinations of the fuzzy basis functions are capable of uniformly approximating any real continuous function on a compact set to arbitrary accuracy. Based on the fuzzy basis function representations, an orthogonal least-squares (OLS) learning algorithm is developed for designing fuzzy systems based on given input-output pairs; then, the OLS algorithm is used to select significant fuzzy basis functions which are used to construct the final fuzzy system. The fuzzy basis function expansion is used to approximate a controller for the nonlinear ball and beam system, and the simulation results show that the control performance is improved by incorporating some common-sense fuzzy control rules.
---
paper_title: Logical operation based fuzzy MLP for classification and rule generation

paper_content:

A fuzzy layered neural network for classification and rule generation is proposed using logical neurons. It can handle uncertainty and/or impreciseness in the input as well as the output. Logical operators, namely, t-norm T and t-conorm S involving And and Or neurons, are employed in place of the weighted sum and sigmoid functions. Various fuzzy implication operators are introduced to incorporate different amounts of mutual interaction during the back propagation of erros. In case of partial inputs the model is capable of querying the user for the more important feature information, if and when required. Justification for an inferred decision may be produced in rule form. The built-in And-Or structure of the network enables the generation of appropriate rules expressed as the disjunction of conjunctive clauses. The effectiveness of the model is tested on a speech recognition problem and on some artificially generated pattern sets.
---
paper_title: A gradient descent learning algorithm for fuzzy neural networks

paper_content:

In order to train fuzzy neural nets fuzzy number weights have to be adjusted. Since fuzzy arithmetic automatically leads to monotonic increasing outputs a direct fuzzification of the backpropagation method does not work. Therefore, other strategies like evolutionary algorithms are being considered in the literature. In this paper we suggest a backpropagation based method of adjusting the weights. Furthermore, we show that by using the proposed method convergence can be guaranteed.
---
paper_title: Constructing fuzzy model by self-organizing counterpropagation network

paper_content:

This paper describes a general and systematic approach to constructing a multivariable fuzzy model from numerical data through a self-organizing counterpropagation network (SOCPN). Two self-organizing algorithms USOCPN and SSOCPN, being unsupervised and supervised respectively, are introduced. SOCPN can be employed in two ways. In the first place, it can be used as a knowledge extractor by which a set of rules are generated from the available numerical data set. The generated rule-base is then utilized by a fuzzy reasoning model. The second use of the SOCPN is as an online adaptive fuzzy model in which the rule-base in terms of connection weights is updated successively in response to the incoming measured data. The comparative results on three well studied examples suggest that the method has merits of simple structure, fast learning speed, and good modeling accuracy. >
---
paper_title: Fuzzy Neural Networks

paper_content:

In this paper, the McCulloch-Pitts model of a neuron is extended to a more general model which allows the activity of a neuron to be a “fuzzy” rather than an “all-or-none” process. The generalized model is called a fuzzy neuron. Some basic properties of fuzzy neural networks as well as their applications to the synthesis of fuzzy automata are investigated. It is shown that any n-state minimal fuzzy automatan can be realized by a network of m fuzzy neurons, where ⌈log2n⌉<m<2n. Examples are given to illustrate the procedure. As an example of application, a realization of fuzzy language recognizer using a fuzzy neural network is presented. The techniques described in this paper may be of use in the study of neural networks as well as in language, pattern recognition, and learning.
---
paper_title: A learning algorithm of fuzzy neural networks with triangular fuzzy weights

paper_content:

Abstract In this paper, first we propose an architecture of fuzzy neural networks with triangular fuzzy weights. The proposed fuzzy neural network can handle fuzzy input vectors as well as real input vectors. In both cases, outputs from the fuzzy neural network are fuzzy vectors. The input-output relation of each unit of the fuzzy neural network is defined by the extension principle of Zadeh. Next we define a cost function for the level sets (i.e., α-cuts) of fuzzy outputs and fuzzy targets. Then we derive a learning algorithm from the cost function for adjusting three parameters of each triangular fuzzy weight. Finally, we illustrate our approach by computer simulations on numerical examples.
---
paper_title: Self-organization for object extraction using a multilayer neural network and fuzziness measures

paper_content:

The feedforward multilayer perceptron (MLP) with back-propagation of error is described. Since use of this network requires a set of labeled input-output, as such it cannot be used for segmentation of images when only one image is available. (However, if images to be processed are of similar nature, one can use a set of known images for learning and then use the network for processing of other images.) A self-organizing multilayer neural network architecture suitable for image processing is proposed. The proposed architecture is also a feedforward one with back-propagation of errors; but like MLP it does not require any supervised learning. Each neuron is connected to the corresponding neuron in the previous layer and the set of neighbors of that neuron. The output status of neurons in the output layer is described as a fuzzy set. A fuzziness measure of this fuzzy set is used as a measure of error in the system (instability of the network). Learning rates for various measures of fuzziness have been theoretically and experimentally studied. An application of the proposed network in object extraction from noisy scenes is also demonstrated.
---
paper_title: Fuzzy self-organization, inferencing, and rule generation

paper_content:

A connectionist inferencing network, based on the fuzzy version of Kohonen's model already developed by the authors, is proposed. It is capable of handling uncertainty and/or impreciseness in the input representation provided in quantitative, linguistic and/or set forms. The output class membership value of an input pattern is inferred by the trained network. A measure of certainty expressing confidence m the decision is also defined. The model is capable of querying the user for the more important input feature information, if required, in case of partial inputs. Justification for an inferred decision may be produced in rule form, when so desired by the user. The connection weight magnitude of the trained neural network are utilized in every stage of the proposed inferencing procedure. The antecedent and consequent parts of the justificatory rules are provided in natural forms. The effectiveness of the algorithm is tested on the vowel recognition problem and on two sets of artificially generated nonconvex pattern classes.
---
paper_title: Neural network implementation of fuzzy logic

paper_content:

Abstract Fuzzy logic has gained increased attention as a methodology for managing uncertainty in a rule-based structure. In a fuzzy logic inference system, more rules can fire at any given time than in a crisp expert system. Since the propositions are modelled as possibility distributions, there is a considerable computation load on the inference engine. In this paper, a neural network structure is proposed as a means of performing fuzzy logic inference. Three variations of the network are described, but in each case, the knowledge of the rule (i.e., the antecedent and consequent clauses) are explicitly encoded in the weights of the net. The theoretical properties of this structure are developed. In fact, the network reduces to crisp modus ponens when the inputs are crisp sets. Also, under suitable conditions the degree of specificity of the consequences of the inference is a monotone function of the degree of specificity of the input. Several simulation studies are included to illustrate the performance of the fuzzy logic inference networks.
---
paper_title: Learning and tuning fuzzy logic controllers through reinforcements

paper_content:

A method for learning and tuning a fuzzy logic controller based on reinforcements from a dynamic system is presented. It is shown that: the generalized approximate-reasoning-based intelligent control (GARIC) architecture learns and tunes a fuzzy logic controller even when only weak reinforcement, such as a binary failure signal, is available; introduces a new conjunction operator in computing the rule strengths of fuzzy control rules; introduces a new localized mean of maximum (LMOM) method in combining the conclusions of several firing control rules; and learns to produce real-valued control actions. Learning is achieved by integrating fuzzy inference into a feedforward network, which can then adaptively improve performance by using gradient descent methods. The GARIC architecture is applied to a cart-pole balancing system and demonstrates significant improvements in terms of the speed of learning and robustness to changes in the dynamic system's parameters over previous schemes for cart-pole balancing. >
---
paper_title: Fuzzy neural networks: a survey

paper_content:

Abstract In this paper a fuzzy neural network will be a layered, feedforward, neural net that has fuzzy signals and/or fuzzy weights. We survey recent results on learning algorithms and applications for fuzzy neural networks.
---
paper_title: Implementation of conjunctive and disjunctive fuzzy logic rules with neural networks

paper_content:

Abstract The use of fuzzy logic to model and manage uncertainty in a rule-based system places high computational demands on an inference engine. In an earlier paper, we introduced trainable neural network structures for fuzzy logic. These networks can learn and extrapolate complex relationships between possibility distributions for the antecedents and consequents in the rules. In this paper, the power of these networks is further explored. The sensitivity of the output to noisy input distributions (which are likely if the clauses are generated from real data) is demonstrated as well as the ability of the networks to internalize multiple conjunctive clause and disjunctive clause rules. Since different rules (with the same variables) can be encoded in a single network, this approach to fuzzy logic inference provides a natural mechanism for rule conflict resolution.
---
paper_title: Neural fuzzy systems: a neuro-fuzzy synergism to intelligent systems

paper_content:

Neural Fuzzy Systems provides a comprehensive, up-to-date introduction to the basic theories of fuzzy systems and neural networks, as well as an exploration of how these two fields can be integrated to create Neural-Fuzzy Systems. It includes Matlab software, with a Neural Network Toolkit, and a Fuzzy System Toolkit.
---
paper_title: Multilayer perceptron, fuzzy sets, and classification

paper_content:

A fuzzy neural network model based on the multilayer perceptron, using the backpropagation algorithm, and capable of fuzzy classification of patterns is described. The input vector consists of membership values to linguistic properties while the output vector is defined in terms of fuzzy class membership values. This allows efficient modeling of fuzzy uncertain patterns with appropriate weights being assigned to the backpropagated errors depending upon the membership values at the corresponding outputs. During training, the learning rate is gradually decreased in discrete steps until the network converges to a minimum error solution. The effectiveness of the algorithm is demonstrated on a speech recognition problem. The results are compared with those of the conventional MLP, the Bayes classifier, and other related models. >
---
paper_title: Evidence aggregation networks for fuzzy logic inference

paper_content:

Fuzzy logic has been applied in many engineering disciplines. The problem of fuzzy logic inference is investigated as a question of aggregation of evidence. A fixed network architecture employing general fuzzy unions and intersections is proposed as a mechanism to implement fuzzy logic inference. It is shown that these networks possess desirable theoretical properties. Networks based on parameterized families of operators (such as Yager's union and intersection) have extra predictable properties and admit a training algorithm which produces sharper inference results than were earlier obtained. Simulation studies corroborate the theoretical properties. >
---
paper_title: Foundations Of Neuro-Fuzzy Systems

paper_content:

From the Publisher: ::: Foundations of Neuro-Fuzzy Systems reflects the current trend in intelligent systems research towards the integration of neural networks and fuzzy technology. The authors demonstrate how a combination of both techniques enhances the performance of control, decision-making and data analysis systems. Smarter and more applicable structures result from marrying the learning capability of the neural network with the transparency and interpretability of the rule-based fuzzy system. Foundations of Neuro-Fuzzy Systems highlights the advantages of integration making it a valuable resource for graduate students and researchers in control engineering, computer science and applied mathematics. The authors' informed analysis of practical neuro-fuzzy applications will be an asset to industrial practitioners using fuzzy technology and neural networks for control systems, data analysis and optimization tasks.
---
paper_title: Fuzzy neural network with fuzzy signals and weights

paper_content:

We discuss the direct fuzzification of a standard layered, feedforward, neural network where the signals and weights are fuzzy sets. A fuzzified delta rule is presented for learning. Three applications are given including fuzzy expert systems, fuzzy hierarchical analysis, and fuzzy systems modeling. © 1993 John Wiley & Sons, Inc.
---
paper_title: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps

paper_content:

A neural network architecture is introduced for incremental supervised learning of recognition categories and multidimensional maps in response to arbitrary sequences of analog or binary input vectors, which may represent fuzzy or crisp sets of features. The architecture, called fuzzy ARTMAP, achieves a synthesis of fuzzy logic and adaptive resonance theory (ART) neural networks by exploiting a close formal similarity between the computations of fuzzy subsethood and ART category choice, resonance, and learning. Four classes of simulation illustrated fuzzy ARTMAP performance in relation to benchmark backpropagation and generic algorithm systems. These simulations include finding points inside versus outside a circle, learning to tell two spirals apart, incremental approximation of a piecewise-continuous function, and a letter recognition database. The fuzzy ARTMAP system is also compared with Salzberg's NGE systems and with Simpson's FMMC system. >
---
paper_title: Interpolation of fuzzy if-then rules by neural networks

paper_content:

Abstract A number of approaches have been proposed for implementing fuzzy if-then rules with trainable multilayer feedforward neural networks. In these approaches, learning of neural networks is performed for fuzzy inputs and fuzzy targets. Because the standard back-propagation (BP) algorithm cannot be directly applied to fuzzy data, transformation of fuzzy data into non-fuzzy data or modification of the learning algorithm is required. Therefore the approaches for implementing fuzzy if-then rules can be classified into two main categories: introduction of preprocessors of fuzzy data and modification of the learning algorithm. In the first category, the standard BP algorithm can be employed after generating non-fuzzy data from fuzzy data by preprocessors. Two kinds of preprocessors based on membership values and level sets are examined in this paper. In the second category, the standard BP algorithm is modified to directly handle the level sets (i.e., intervals) of fuzzy data. This paper examines the ability of each approach to interpolate sparse fuzzy if-then rules. By computer simulations, high fitting ability of approaches in the first category and high interpolating ability of those in the second category are demonstrated.
---
paper_title: Neuro-Fuzzy Pattern Recognition: Methods in Soft Computing

paper_content:

From the Publisher: ::: The authors consolidate a wealth of information previously scattered in disparate articles, journals, and edited volumes, explaining both the theory of neuro-fuzzy computing and the latest methodologies for performing different pattern recognition tasks in the neuro-fuzzy network - classification, feature evaluation, rule generation, knowledge extraction, and hybridization. Special emphasis is given to the integration of neuro-fuzzy methods with rough sets and genetic algorithms (GAs) to ensure more efficient recognition systems.
---
paper_title: Fuzzy Kohonen clustering networks

paper_content:

The authors propose a fuzzy Kohonen clustering network which integrates the fuzzy c-means (FCM) model into the learning rate and updating strategies of the Kohonen network. This yields an optimization problem related to FCM, and the numerical results show improved convergence as well as reduced labeling errors. It is proved that the proposed scheme is equivalent to the c-means algorithms. The new method can be viewed as a Kohonen type of FCM, but it is self-organizing, since the size of the update neighborhood and the learning rate in the competitive layer are automatically adjusted during learning. Anderson's IRIS data were used to illustrate this method. The results are compared with the standard Kohonen approach. >
---
paper_title: Review Neuro-fuzzy computing for image processing and pattern recognition

paper_content:

The relevance of integration of the merits of fuzzy set theory and neural network models for designing an efficient decision making system is explained. The feasibility of such systems and different ways of integration, so far made, in the context of image processing and pattern recognition are described. Scope for further research and development is outlined. An extensive bibliography is also provided.
---
paper_title: A referential scheme of fuzzy decision making and its neural network structure

paper_content:

The author introduces a method for dealing with imprecise objectives involved in the process of decision-making. A three-stage form of the system is proposed. It comprises three basic functional components realizing matching, nonlinear transformation, and inverse matching. The proposed scheme has a referential structure which shows that the fuzzy set of a decision is not determined by the objectives themselves, but by the levels of the matching with some prototype decision situations. Both matching and inverse matching procedures involve some logic-based mechanisms (equality indices). Neural nets are used to realize the nonlinear mapping indicated in the general scheme. Several advantages of the referential model, including exhaustive usage of knowledge about the decision problem conveyed by prototype situations, and an introduction of mechanisms of evaluation of the relevancy of fuzzy decisions, are highlighted. Additional indices expressing consistency of decision scenarios are developed. Detailed numerical studies demonstrate the performance of the method and provide some additional background concerning an evaluation of the results. >
---
paper_title: Neural networks designed on approximate reasoning architecture and their applications

paper_content:

The NARA (neural networks based on approximate reasoning architecture) model is proposed and its composition procedure and evaluation are described. NARA is a neural network (NN) based on the structure of fuzzy inference rules. The distinctive feature of NARA is that its internal state can be analyzed according to the rule structure, and the problematic portion can be easily located and improved. The ease with which performance can be improved is shown by applying the NARA model to pattern classification problems. The NARA model is shown to be more efficient than ordinary NN models. In NARA, characteristics of the application task can be built into the NN model in advance by employing the logic structure, in the form of fuzzy inference rules. Therefore, it is easier to improve the performance of NARA, in which the internal state can be observed because of its structure, than that of an ordinary NN model, which is like a black box. Examples are introduced by applying the NARA model to the problems of auto adjustment of VTR tape running mechanisms and alphanumeric character recognition. >
---
paper_title: Incorporating Fuzzy Membership Functions into the Perceptron Algorithm

paper_content:

The perceptron algorithm, one of the class of gradient descent techniques, has been widely used in pattern recognition to determine linear decision boundaries. While this algorithm is guaranteed to converge to a separating hyperplane if the data are linearly separable, it exhibits erratic behavior if the data are not linearly separable. Fuzzy set theory is introduced into the perceptron algorithm to produce a ``fuzzy algorithm'' which ameliorates the convergence problem in the nonseparable case. It is shown that the fuzzy perceptron, like its crisp counterpart, converges in the separable case. A method of generating membership functions is developed, and experimental results comparing the crisp to the fuzzy perceptron are presented.
---
paper_title: Fuzzy logic, neural networks, and soft computing

paper_content:

In retrospect, the year 1990 may well be viewed as the beginning of a new trend in the design of household appliances, consumer electronics, cameras, and other types of widely used consumer products. The trend in question relates to a marked increase in what might be called the Machine Intelligence Quotient (MIQ) of such products compared to what it was before 1990. Today, we have microwave ovens and washing machines that can figure out on their own what settings to use to perform their tasks optimally; cameras that come close to professional photographers in picture-taking ability; and many other products that manifest an impressive capability to reason, make intelligent decisions, and learn from experience
---
paper_title: Rough fuzzy MLP: knowledge encoding and classification

paper_content:

A scheme of knowledge encoding in a fuzzy multilayer perceptron (MLP) using rough set-theoretic concepts is described. Crude domain knowledge is extracted from the data set in the form of rules. The syntax of these rules automatically determines the appropriate number of hidden nodes while the dependency factors are used in the initial weight encoding. The network is then refined during training. Results on classification of speech and synthetic data demonstrate the superiority of the system over the fuzzy and conventional versions of the MLP (involving no initial knowledge).
---
paper_title: Combining rough sets learning- and neural learning-method to deal with uncertain and imprecise information

paper_content:

Abstract Any system designed to reason about the real world must be capable of dealing with uncertainty. The complexity of the real world and the finite size of most knowledge bases pose significant difficulties for the traditional concept of the learning system. Experience has shown that many learning paradigms fail to scale up to those problems. One response to these failures has been to construct systems which use multiple learning paradigms. Thus the strengths of one paradigm counterbalance some of the weaknesses of the others. As a result the effectiveness of the overall system will be enhanced. Consequently, integrated techniques have been widespread over the last years. A multistrategy which addresses those issues is presented. This approach joins two forms of learning, the technique of neural networks and rough sets. These seem at first quite different but they share the common ability to work well in a natural environment. In a closed loop fashion we will achieve more robust concept learning capabilities for a variety of difficult classification tasks. The objective of integration is twofold: (i) to improve the overall classification effectiveness of learned objects' description, (ii) to refine the dependency factors of the rules.
---
paper_title: Real-time stable self-learning FNN controller using genetic algorithm

paper_content:

Abstract A kind of real-time stable self-learning fuzzy neural network (FNN) control system is proposed in this paper. The control system is composed of two parts: (1) A FNN controller which use genetic algorithm (GA) to search optimal fuzzy rules and membership functions for the unknown controlled plant; (2) A supervisor which can guarantee the stability of the control system during the real-time learning stage, since the GA has some random property which may cause control system unstable. The approach proposed in this paper combine a priori knowledge of designer and the learning ability of FNN to achieve optimal fuzzy control for an unknown plant in real-time. The efficiency of the approach is verified by computer simulation.
---
paper_title: Linguistic rule extraction from neural networks and genetic-algorithm-based rule selection

paper_content:

This paper proposes a hybrid approach to the design of a compact fuzzy rule-based classification system with a small number of linguistic rules. The proposed approach consists of two procedures: rule extraction from a trained neural network and rule selection by a genetic algorithm. We first describe how linguistic rules can be extracted from a multilayer feedforward neural network that has been already trained for a classification problem with many continuous attributes. In our rule extraction procedure, a linguistic input vector corresponding to the antecedent part of a linguistic rule is presented to the trained neural network, and the fuzzy output vector front the trained neural network is examined for determining the consequent part and the grade of certainty of that linguistic rule. Next we explain how a genetic algorithm can be utilized for selecting a small number of significant linguistic rules from a large number of extracted rules. Our rule selection problem has two objectives: to minimize the number of selected linguistic rules and to maximize the number of correctly classified patterns by the selected linguistic rules. A multi-objective genetic algorithm is employed for finding a set of non-dominated solutions with respect to these two objectives. Finally we illustrate our hybrid approach by computer simulations on real-world test problems.
---
paper_title: Extracting Rules from Composite Neural Networks for Medical Diagnostic Problems

paper_content:

Recently, neural networks have been applied to many medical diagnostic problems because of their appealing properties, robustness, capability of generalization and fault tolerance. Although the predictive accuracy of neural networks may be higher than that of traditional methods (e.g., statistical methods) or human experts, the lack of explanation from a trained neural network leads to the difficulty that users would hesitate to take the advise of a black box on faith alone. This paper presents a class of composite neural networks which are trained in such a way that the values of the network parameters can be utilized to generate If-Then rules on the basis of preselected meaningful coordinates. The concepts and methods presented in the paper are illustrated through one practical example from medical diagnosis.
---
paper_title: Evolutionary Modular Design of Rough Knowledge-based Network using Fuzzy Attributes

paper_content:

This article describes a way of integrating rough set theory with a fuzzy MLP using a modular evolutionary algorithm, for classification and rule generation in soft computing paradigm. The novelty of the method lies in applying rough set theory for extracting dependency rules directly from a real-valued attribute table consisting of fuzzy membership values. This helps in preserving all the class representative points in the dependency rules by adaptively applying a threshold that automatically takes care of the shape of membership functions. An l-class classification problem is split into l two-class problems. Crude subnetwork modules are initially encoded from the dependency rules. These subnetworks are then combined and the final network is evolved using a GA with restricted mutation operator which utilizes the knowledge of the modular structure already generated, for faster convergence. The GA tunes the fuzzification parameters, and network weight and structure simultaneously, by optimising a single fitness function. This methodology helps in imposing a structure on the weights, which results in a network more suitable for rule generation. Performance of the algorithm is compared with related techniques.
---
paper_title: Computational Intelligence: An Introduction

paper_content:

From the Publisher: ::: Computational Intelligence: An Introduction consists of a highly readable and systematic exposure of the fundamentals of computational intelligence, along with the coherent presentation of sound and comprehensive analysis and design practices. Provides a balanced introduction to computational intelligence, emphasizing equally the important analysis and design aspects of the emerging technology; text is organized in a way that allows for the easy use of the book as a basic course material; presents a design-oriented approach toward the use of computational intelligence; organizes exercises and problems of different levels of difficulty following each chapter; and complete algorithms are presented in a structured fashion, easing understanding and implementation.
---
paper_title: Staging of cervical cancer with soft computing

paper_content:

Describes a way of designing a hybrid decision support system in soft computing paradigm for detecting the different stages of cervical cancer. Hybridization includes the evolution of knowledge-based subnetwork modules with genetic algorithms (CIAs) using rough set theory and the Interactive Dichotomizer 3 (ID3) algorithm. Crude subnetworks obtained via rough set theory and the ID3 algorithm are evolved using CAs. The evolution uses a restricted mutation operator which utilizes the knowledge of the modular structure, already generated, for faster convergence. The CA tunes the network weights and structure simultaneously. The aforesaid integration enhances the performance in terms of classification score, network size and training time, as compared to the conventional multilayer perceptron. This methodology also helps in imposing a structure on the weights, which results in a network more suitable for extraction of logical rules and human interpretation of the inferencing procedure.
---
paper_title: Rough knowledge-based network, fuzziness and classification

paper_content:

A method of integrating rough sets and fuzzy multilayer perceptron (MLP) for designing a knowledge-based network for pattern recognition problems is described. Rough set theory is used to extract crude knowledge from the input domain in the form of rules. The syntax of these rules automatically determines the optimal number of hidden nodes while the dependency factors are used in the initial weight encoding. Results on classification of speech data demonstrate the superiority of the system over the fuzzy and conventional versions of the MLP.
---
paper_title: A genetic-based neuro-fuzzy approach for modeling and control of dynamical systems

paper_content:

Linguistic modeling of complex irregular systems constitutes the heart of many control and decision making systems, and fuzzy logic represents one of the most effective algorithms to build such linguistic models. In this paper, a linguistic (qualitative) modeling approach is proposed. The approach combines the merits of the fuzzy logic theory, neural networks, and genetic algorithms (GAs). The proposed model is presented in a fuzzy-neural network (FNN) form which can handle both quantitative (numerical) and qualitative (linguistic) knowledge. The learning algorithm of a FNN is composed of three phases. The first phase is used to find the initial membership functions of the fuzzy model. In the second phase, a new algorithm is developed and used to extract the linguistic-fuzzy rules. In the third phase, a multiresolutional dynamic genetic algorithm (MRD-GA) is proposed and used for optimized tuning of membership functions of the proposed model. Two well-known benchmarks are used to evaluate the performance of the proposed modeling approach, and compare it with other modeling approaches.
---
paper_title: Knowledge extraction and the integration by artificial life approach

paper_content:

Artificial life (A-Life) is a new paradigm to realize a phenomena of life and to extract the hidden principles. One of the most attractive features in the A-Life approach is emergence: simple elements interact with each other based on lower level rules, and then the higher level complex phenomena can emerge by interaction. The paper proposes a method for knowledge extraction and integration based on an A-Life approach. The proposed system has two parts: the knowledge extraction network and the A-Life environment. The simple elements interact in the A-Life environment and the data is transferred to the knowledge extraction network. The knowledge is extracted in the form of rules in the rule layer and then they are fed back to the simple elements in the A-Life environment. We dealt with a path planning problem as an example of A-Life environment. In the simulation, we assumed a severe condition: the position of the goal was unknown to the robots. Since the robots did not know the goal in the initial condition, the trajectory by the first robot that reached the goal is very complicated. The trajectory data which the robots had taken were inputted to the knowledge extraction network to extract rules. The trajectories become smooth step by step because of the extracted rules. We extracted various kinds of the rules using several different simple environments. By using the rules extracted from the simpler environments, the robot could reach the goal in a more complex environment.
---
paper_title: Induction of fuzzy rules and membership functions from training examples

paper_content:

Abstract Most fuzzy controllers and fuzzy expert systems must predefine membership functions and fuzzy inference rules to map numeric data into linguistic variable terms and to make fuzzy reasoning work. In this paper, we propose a general learning method as a framework for automatically deriving membership functions and fuzzy if-then rules from a set of given training examples to rapidly build a prototype fuzzy expert system. Based on the membership functions and the fuzzy rules derived, a corresponding fuzzy inference procedure to process inputs is also developed.
---
paper_title: Distributed representation of fuzzy rules and its application to pattern classification

paper_content:

Abstract This paper introduces the concept of distributed representation of fuzzy rules and applies it to classification problems. Distributed representation is implemented by superimposing many fuzzy rules corresponding to different fuzzy partitions of a pattern space. This means that we simulatenously employ many fuzzy rule tables corresponding to different fuzzy partitions in fuzzy inference. In order to apply distributed representation of fuzzy rules to pattern classification problems, we first propose an algorithm to generate fuzzy rules from numerical data. Next we propose a fuzzy inference method using the generated fuzzy rules. The classification power of distributed representation is compared with that of ordinary fuzzy rules which can be viewed as local representation.
---
paper_title: Fuzzy rules extraction directly from numerical data for function approximation

paper_content:

In our previous work (1993) we developed a method for extracting fuzzy rules directly from numerical input-output data for pattern classification. In this paper we extend the method to function approximation. For function approximation, first, the universe of discourse of an output variable is divided into multiple intervals, and each interval is treated as a class. Then the same as for pattern classification, using the input data for each interval, fuzzy rules are recursively defined by activation hyperboxes which show the existence region of the data for the interval and inhibition hyperboxes which inhibit the existence region of data for that interval. The approximation accuracy of the fuzzy system derived by this method is empirically studied using an operation learning application of a water purification plant. Additionally, we compare the approximation performance of the fuzzy system with the function approximation approach based on neural networks. >
---
paper_title: Efficient fuzzy partition of pattern space for classification problems

paper_content:

Abstract This paper proposes an efficient fuzzy partition method of a pattern space for classification problems. The proposed method is based on the sequential subdivision of fuzzy subspaces and the generated fuzzy subspaces have different sizes. In the proposed method, first an n -dimensional pattern space is divided into 2 n fuzzy subspaces with the same size. Next one of the fuzzy subspaces is selected and subdivided into 2 n fuzzy subspaces. This procedure is iterated until a stopping condition is satisfied. Some criteria for selecting a fuzzy subspace to be subdivided are proposed and compared with each other by computer simulations. The proposed method is also compared with other fuzzy classification methods.
---
paper_title: Fuzzy sets of rules for system identification

paper_content:

The synthesis of fuzzy systems involves the identification of a structure and its specialization by means of parameter optimization. In doing this, symbolic approaches which encode the structure information in the form of high-level rules allow further manipulation of the system to minimize its complexity, and possibly its implementation cost, while all-parametric methodologies often achieve better approximation performance. In this paper, we rely on the concept of a fuzzy set of rules to tackle the rule induction problem at an intermediate level. An online adaptive algorithm is developed which almost surely learns the extent to which inclusion of a rule in the rule set significantly contributes to the reproduction of the target behavior. Then, the resulting fuzzy set of rules can be defuzzified to give a conventional rule set with similar behavior. Comparisons with high-level and low-level methodologies show that this approach retains the most positive features of both.
---
paper_title: Generating fuzzy rules by learning from examples

paper_content:

A general method is developed for generating fuzzy rules from numerical data. The method consists of five steps: dividing the input and output spaces of the given numerical data into fuzzy regions; generating fuzzy rules from the given data; assigning a degree to each of the generated rules for the purpose of resolving conflicts among the generated rules; creating a combined fuzzy-associative-memory (FAM) bank based on both the generated rules and linguistic rules of human experts; and determining a mapping from input space to output space based on the combined FAM bank using a defuzzifying procedure. The mapping is proved to be capable of approximating any real continuous function on a compact set to arbitrary accuracy. The method is applied to predicting a chaotic time series. >
---
paper_title: Simultaneous design of membership functions and rule sets for fuzzy controllers using genetic algorithms

paper_content:

This paper examines the applicability of genetic algorithms (GA's) in the simultaneous design of membership functions and rule sets for fuzzy logic controllers. Previous work using genetic algorithms has focused on the development of rule sets or high performance membership functions; however, the interdependence between these two components suggests a simultaneous design procedure would be a more appropriate methodology. When GA's have been used to develop both, it has been done serially, e.g., design the membership functions and then use them in the design of the rule set. This, however, means that the membership functions were optimized for the initial rule set and not the rule set designed subsequently. GA's are fully capable of creating complete fuzzy controllers given the equations of motion of the system, eliminating the need for human input in the design loop. This new method has been applied to two problems, a cart controller and a truck controller. Beyond the development of these controllers, we also examine the design of a robust controller for the cart problem and its ability to overcome faulty rules. >
---
paper_title: Extracting fuzzy rules for system modeling using a hybrid of genetic algorithms and Kalman filter

paper_content:

Abstract This paper proposes a hybrid algorithm for extracting important fuzzy rules from a given rule base to construct a “parsimonious” fuzzy model with a high generalization ability. This algorithm combines the advantages of genetic algorithms' strong search capacity and Kalman filter's fast convergence merit. Each random combination of the rules in the rule base is coded into a binary string and treated as a chromosome in genetic algorithms. The binary string indicates the structure of a fuzzy model. The parameters of the model are then estimated using the Kalman filter. In order to achieve a trade-off between the accuracy and the complexity of a fuzzy model, the Schwarz-Rissanen Criterion is used as an evaluation function in the hybrid algorithm. The practical applicability of the proposed algorithm is examined by computer simulations on a human operator modeling problem and a nonlinear system modeling problem.
---
paper_title: Selecting fuzzy if-then rules for classification problems using genetic algorithms

paper_content:

This paper proposes a genetic-algorithm-based method for selecting a small number of significant fuzzy if-then rules to construct a compact fuzzy classification system with high classification power. The rule selection problem is formulated as a combinatorial optimization problem with two objectives: to maximize the number of correctly classified patterns and to minimize the number of fuzzy if-then rules. Genetic algorithms are applied to this problem. A set of fuzzy if-then rules is coded into a string and treated as an individual in genetic algorithms. The fitness of each individual is specified by the two objectives in the combinatorial optimization problem. The performance of the proposed method for training data and test data is examined by computer simulations on the iris data of Fisher. >
---
paper_title: Genetic-based new fuzzy reasoning models with application to fuzzy control

paper_content:

The successful application of fuzzy reasoning models to fuzzy control systems depends on a number of parameters, such as fuzzy membership functions, that are usually decided upon subjectively. It is shown in this paper that the performance of fuzzy control systems may be improved if the fuzzy reasoning model is supplemented by a genetic-based learning mechanism. The genetic algorithm enables us to generate an optimal set of parameters for the fuzzy reasoning model based either on their initial subjective selection or on a random selection. It is shown that if knowledge of the domain is available, it is exploited by the genetic algorithm leading to an even better performance of the fuzzy controller. >
---
paper_title: Fuzzy control of pH using genetic algorithms

paper_content:

Abstruct- Establishing suitable control of pH, a requirement in a number of mineral and chemical industries, poses a difficult problem because of inherent nonlinearities and frequently changing process dynamics. Researchers at the U.S. Bureau of Mines have developed a technique for producing adaptive fuzzy logic controllers (FLC’s) that are capable of effectively managing such systems. In this technique, a genetic algorithm (GA) alters the membership functions employed by a conventional FLC, an approach that is contrary to the tactic generally used to provide FLC’s with adaptive capabilities in which the rule set is altered. GA’s are search algorithms based on the mechanics of natural genetics that are able to rapidly locate near-optimal solutions to difficult problems. The Bureau-developed technique is used to produce an adaptive GA-FLC for a laboratory acid-base experiment. Nonlinearities in the laboratory system are associated with the logarithmic pH scale (pH is proportional to the logarithm of HJO’ ions) and changing process dynamics are introduced by altering system parameters such as the desired set point and the concentration and buffering capacity of input solutions. Results indicate that FLC’s augmented with GA’s offer a powerful alternative to conventional process control techniques in the nonlinear, rapidly changing pH systems commonly found in industry.
---
paper_title: Knowledge-based connectionism for revising domain theories

paper_content:

A knowledge-based connectionist model for machine learning referred to as KBCNN is presented. In the KBCNN learning model, useful domain attributes and concepts are first identified and linked in a way consistent with initial domain knowledge, and then the links are weighted properly so as to maintain the semantics. Hidden units and additional connections may be introduced into this initial connectionist structure as appropriate. Then, this primitive structure evolves to minimize empirical error. The KBCNN learning model allows the theory learned or revised to be translated into the symbolic rule-based language that describes the initial theory. Thus, a domain theory can be pushed onto the network, revised empirically over time, and decoded in symbolic form. The domain of molecular genetics is used to demonstrate the validity of the KBCNN learning model and its superiority over related learning methods. >
---
paper_title: NeuroLinear: From neural networks to oblique decision rules

paper_content:

Abstract We present NeuroLinear, a system for extracting oblique decision rules from neural networks that have been trained for classification of patterns. Each condition of an oblique decision rule corresponds to a partition of the attribute space by a hyperplane that is not necessarily axisparallel. Allowing a set of such hyperplanes to form the boundaries of the decision regions leads to a significant reduction in the number of rules generated while maintaining the accuracy rates of the networks. We describe the components of NeuroLinear in detail by way of two examples using artificial datasets. Our experimental results on real-world datasets show that the system is effective in extracting compact and comprehensible rules with high predictive accuracy from neural networks.
---
paper_title: Extraction of Logical Rules from Neural Networks

paper_content:

Three neural-based methods for extraction of logical rules from data are presented. These methods facilitate conversion of graded response neural networks into networks performing logical functions. MLP2LN method tries to convert a standard MLP into a network performing logical operations (LN). C-MLP2LN is a constructive algorithm creating such MLP networks. Logical interpretation is assured by adding constraints to the cost function, forcing the weights to ±1 or 0. Skeletal networks emerge ensuring that a minimal number of logical rules are found. In both methods rules covering many training examples are generated before more specific rules covering exceptions. The third method, FSM2LN, is based on the probability density estimation. Several examples of performance of these methods are presented.
---
paper_title: Cascade ARTMAP: Integrating Neural Computation and Symbolic Knowledge Processing

paper_content:

This paper introduces a hybrid system termed cascade adaptive resonance theory mapping (ARTMAP) that incorporates symbolic knowledge into neural-network learning and recognition. Cascade ARTMAP, a generalization of fuzzy ARTMAP, represents intermediate attributes and rule cascades of rule-based knowledge explicitly and performs multistep inferencing. A rule insertion algorithm translates if-then symbolic rules into cascade ARTMAP architecture. Besides that initializing networks with prior knowledge can improve predictive accuracy and learning efficiency, the inserted symbolic knowledge can be refined and enhanced by the cascade ARTMAP learning algorithm. By preserving symbolic rule form during learning, the rules extracted from cascade ARTMAP can be compared directly with the originally inserted rules. Simulations on an animal identification problem indicate that a priori symbolic knowledge always improves system performance, especially with a small training set. Benchmark study on a DNA promoter recognition problem shows that with the added advantage of fast learning, cascade ARTMAP rule insertion and refinement algorithms produce performance superior to those of other machine learning systems and an alternative hybrid system known as knowledge-based artificial neural network (KBANN). Also, the rules extracted from cascade ARTMAP are more accurate and much cleaner than the NofM rules extracted from KBANN.
---
paper_title: C4.5: Programs for Machine Learning

paper_content:

From the Publisher: ::: Classifier systems play a major role in machine learning and knowledge-based systems, and Ross Quinlan's work on ID3 and C4.5 is widely acknowledged to have made some of the most significant contributions to their development. This book is a complete guide to the C4.5 system as implemented in C for the UNIX environment. It contains a comprehensive guide to the system's use , the source code (about 8,800 lines), and implementation notes. The source code and sample datasets are also available on a 3.5-inch floppy diskette for a Sun workstation. ::: ::: C4.5 starts with large sets of cases belonging to known classes. The cases, described by any mixture of nominal and numeric properties, are scrutinized for patterns that allow the classes to be reliably discriminated. These patterns are then expressed as models, in the form of decision trees or sets of if-then rules, that can be used to classify new cases, with emphasis on making the models understandable as well as accurate. The system has been applied successfully to tasks involving tens of thousands of cases described by hundreds of properties. The book starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting. Advantages and disadvantages of the C4.5 approach are discussed and illustrated with several case studies. ::: ::: This book and software should be of interest to developers of classification-based intelligent systems and to students in machine learning and expert systems courses.
---
paper_title: Structural learning with forgetting

paper_content:

Abstract It is widely known that, despite its popularity, back propagation learning suffers from various difficulties. There have been many studies aiming at the solution of these. Among them there are a class of learning algorithms, which I call structural learning, aiming at small-sized networks requiring less computational cost. Still more important is the discovery of regularities in or the extraction of rules from training data. For this purpose I propose a learning method called structural learning with forgetting. It is applied to various examples: the discovery of Boolean functions, classification of irises, discovery of recurrent networks, prediction of time series and rule extraction from mushroom data. These results demonstrate the effectiveness of structural learning with forgetting. A comparative study on various structural learning methods also supports its effectiveness.
---
paper_title: Learning in certainty-factor-based multilayer neural networks for classification

paper_content:

The computational framework of rule-based neural networks inherits from the neural network and the inference engine of an expert system. In one approach, the network activation function is based on the certainty factor (CF) model of MYCIN-like systems. In this paper, it is shown theoretically that the neural network using the CF-based activation function requires relatively small sample sizes for correct generalization. This result is also confirmed by empirical studies in several independent domains.
---
paper_title: A new rule extraction method from neural networks

paper_content:

This paper presents a method of extracting rules from multilayered neural networks (NN) formed using a random optimization (search) method (ROM). The objective of this study is to extract rules from NN, achieving 100% recognition accuracy in a pattern recognition system. NNs to be extracted rules are formed using ROM. A hybrid algorithm of NN and ROM performs a formation of a small-sized NN system, which is suitable for a rule extraction. In this paper iris data is used as inputs. ROM is utilized to reduce the number of connection weights in NN. The network weights survived after the ROM training represent regularities to perform pattern classification. The rules are then extracted from the networks in which hidden units use signum and sigmoid functions to produce binary outputs. It enables us to extract simple logical functions from the network. By means of computer simulation, the effectiveness of this approach is examined.
---
paper_title: Survey and critique of techniques for extracting rules from trained artificial neural networks

paper_content:

It is becoming increasingly apparent that, without some form of explanation capability, the full potential of trained artificial neural networks (ANNs) may not be realised. This survey gives an overview of techniques developed to redress this situation. Specifically, the survey focuses on mechanisms, procedures, and algorithms designed to insert knowledge into ANNs (knowledge initialisation), extract rules from trained ANNs (rule extraction), and utilise ANNs to refine existing rule bases (rule refinement). The survey also introduces a new taxonomy for classifying the various techniques, discusses their modus operandi, and delineates criteria for evaluating their efficacy.
---
paper_title: Connectionist expert systems

paper_content:

Connectionist networks can be used as expert system knowledge bases. Furthermore, such networks can be constructed from training examples by machine learning techniques. This gives a way to automate the generation of expert systems for classification problems.
---
paper_title: Learning rules for neuro-controller via simultaneous perturbation

paper_content:

This paper describes learning rules using simultaneous perturbation for a neurocontroller that controls an unknown plant. When we apply a direct control scheme by a neural network, the neural network must learn an inverse system of the unknown plant. In this case, we must know the sensitivity function of the plant using a kind of the gradient method as a learning rule of the neural network. On the other hand, the learning rules described here do not require information about the sensitivity function. Some numerical simulations of a two-link planar arm and a tracking problem for a nonlinear dynamic plant are shown.
---
paper_title: A search technique for rule extraction from trained neural networks

paper_content:

Abstract Search methods for rule extraction from neural networks work by finding those combinations of inputs that make the neuron active. By sorting the input weights to a neuron and ordering the weights suitably, it is possible to prune the search space. Based on this observation, we present an algorithm for rule extraction from feedforward neural networks with boolean inputs and analyze its properties.
---
paper_title: Knowledge-Based Artificial Neural Networks

paper_content:

Abstract Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN ( Knowledge-Based Artificial Neural Networks ) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific “domain theories”, represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.
---
paper_title: Extracting Rules from Neural Networks by Pruning and Hidden-Unit Splitting

paper_content:

An algorithm for extracting rules from a standard three-layer feedforward neural network is proposed. The trained network is first pruned not only to remove redundant connections in the network but, more important, to detect the relevant inputs. The algorithm generates rules from the pruned network by considering only a small number of activation values at the hidden units. If the number of inputs connected to a hidden unit is sufficiently small, then rules that describe how each of its activation values is obtained can be readily generated. Otherwise the hidden unit will be split and treated as output units, with each output unit corresponding to an activation value. A hidden layer is inserted and a new subnetwork is formed, trained, and pruned. This process is repeated until every hidden unit in the network has a relatively small number of input units connected to it. Examples on how the proposed algorithm works are shown using real-world data arising from molecular biology and signal processing. Our re...
---
paper_title: Medical diagnostic expert system based on PDP model

paper_content:

The applicability of PDP (parallel distributed processing) models to knowledge processing is clarified. The authors evaluate the diagnostic capabilities of a prototype medical diagnostic expert system based on a multilayer network. After having been trained on only 300 patients, the prototype system shows diagnostic capabilities almost equivalent to those of a symbolic expert system. Symbolic knowledge is extracted from what the multilayer network has learned. The extracted knowledge is compared with doctors' knowledge. Moreover, a method to extract rules from the network and usage of the rules in a confirmation process are proposed. >
---
paper_title: Extracting M-of-N Rules from Trained Neural Networks

paper_content:

An effective algorithm for extracting M-of-N rules from trained feedforward neural networks is proposed. First, we train a network where each input of the data can only have one of the two possible values, -1 or one. Next, we apply the hyperbolic tangent function to each connection from the input layer to the hidden layer of the network. By applying this squashing function, the activation values at the hidden units are effectively computed as the hyperbolic tangent (or the sigmoid) of the weighted inputs, where the weights have magnitudes that are equal one. By restricting the inputs and the weights to binary values either -1 or one, the extraction of M-of-N rules from the networks becomes trivial. We demonstrate the effectiveness of the proposed algorithm on several widely tested datasets. For datasets consisting of thousands of patterns with many attributes, the rules extracted by the algorithm are simple and accurate.
---
paper_title: FERNN: An Algorithm for Fast Extraction of Rules from Neural Networks

paper_content:

Before symbolic rules are extracted from a trained neural network, the network is usually pruned so as to obtain more concise rules. Typical pruning algorithms require retraining the network which incurs additional cost. This paper presents FERNN, a fast method for extracting rules from trained neural networks without network retraining. Given a fully connected trained feedforward network with a single hidden layer, FERNN first identifies the relevant hidden units by computing their information gains. For each relevant hidden unit, its activation values is divided into two subintervals such that the information gain is maximized. FERNN finds the set of relevant network connections from the input units to this hidden unit by checking the magnitudes of their weights. The connections with large weights are identified as relevant. Finally, FERNN generates rules that distinguish the two subintervals of the hidden activation values in terms of the network inputs. Experimental results show that the size and the predictive accuracy of the tree generated are comparable to those extracted by another method which prunes and retrains the network.
---
paper_title: Grammatical Inference using an Adaptive Recurrent Neural Network

paper_content:

In this study, we proposed an adaptive recurrent neural network that is capable of inferring a regular grammar, and at the same time of extracting the underlying grammatical rules emulated by a finite-state automata. Our proposed network adapts from an initial analog phase, which has good training behavior, to a discrete phase for automatic rule extraction. A modified objective function is proposed to accomplish the discretisation process as well as logic learning. Comparison on learning Tomita grammars shows that our network has a significant advantage over other approaches.
---
paper_title: Rule-extraction by backpropagation of polyhedra.

paper_content:

The core problem of rule-extraction from feed-forward networks is an inversion problem. In this article, we solve this inversion problem by backpropagating unions of polyhedra. We obtain as a by-product a new rule-extraction technique for which the fidelity of the extracted rules can be made arbitrarily high.
---
paper_title: Extraction of Rules from Discrete-time Recurrent Neural Networks

paper_content:

The extraction of symbolic knowledge from trained neural networks and the direct encoding of (partial) knowledge into networks prior to training are important issues. They allow the exchange of information between symbolic and connectionist knowledge representations. The focus of this paper is on the quality of the rules that are extracted from recurrent neural networks. Discrete-time recurrent neural networks can be trained to correctly classify strings of a regular language. Rules defining the learned grammar can be extracted from networks in the form of deterministic finite-state automata (DFAs) by applying clustering algorithms in the output space of recurrent state neurons. Our algorithm can extract different finite-state automata that are consistent with a training set from the same network. We compare the generalization performances of these different models and the trained network and we introduce a heuristic that permits us to choose among the consistent DFAs the model which best approximates the learned regular grammar.
---
paper_title: The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks

paper_content:

To date, the preponderance of techniques for eliciting the knowledge embedded in trained artificial neural networks (ANN's) has focused primarily on extracting rule-based explanations from feedforward ANN's. The ADT taxonomy for categorizing such techniques was proposed in 1995 to provide a basis for the systematic comparison of the different approaches. This paper shows that not only is this taxonomy applicable to a cross section of current techniques for extracting rules from trained feedforward ANN's but also how the taxonomy can be adapted and extended to embrace a broader range of ANN types (e,g., recurrent neural networks) and explanation structures. In addition we identify some of the key research questions in extracting the knowledge embedded within ANN's including the need for the formulation of a consistent theoretical basis for what has been, until recently, a disparate collection of empirical results.
---
paper_title: Rule extraction from recurrent neural networks using a symbolic machine learning algorithm

paper_content:

Addresses the extraction of knowledge from recurrent neural networks trained to behave like deterministic finite-state automata (DFAs). To date, methods used to extract knowledge from such networks have relied on the hypothesis that network states tend to cluster and that clusters of network states correspond to DFA states. The computational complexity of such a cluster analysis has led to heuristics which either limit the number of clusters that may form during training or limit the exploration of the output space of hidden recurrent state neurons. These limitations, while necessary, may lead to reduced fidelity, i.e. the extracted knowledge may not model the true behavior of a trained network, perhaps not even for the training set. The method proposed uses a polynomial-time symbolic learning algorithm to infer DFAs solely from the observation of a trained network's input/output behavior. Thus, this method has the potential to increase the fidelity of the extracted knowledge.
---
paper_title: A Neural Expert System with Automated Extraction of Fuzzy If-Then Rules and Its Application to Medical Diagnosis

paper_content:

This paper proposes a fuzzy neural expert system (FNES) with the following two functions: (1) Generalization of the information derived from the training data and embodiment of knowledge in the form of the fuzzy neural network; (2) Extraction of fuzzy If-Then rules with linguistic relative importance of each proposition in an antecedent (If-part) from a trained neural network. This paper also gives a method to extract automatically fuzzy If-Then rules from the trained neural network. To prove the effectiveness and validity of the proposed fuzzy neural expert system, a fuzzy neural expert system for medical diagnosis has been developed.
---
paper_title: Fuzzy self-organization, inferencing, and rule generation

paper_content:

A connectionist inferencing network, based on the fuzzy version of Kohonen's model already developed by the authors, is proposed. It is capable of handling uncertainty and/or impreciseness in the input representation provided in quantitative, linguistic and/or set forms. The output class membership value of an input pattern is inferred by the trained network. A measure of certainty expressing confidence m the decision is also defined. The model is capable of querying the user for the more important input feature information, if required, in case of partial inputs. Justification for an inferred decision may be produced in rule form, when so desired by the user. The connection weight magnitude of the trained neural network are utilized in every stage of the proposed inferencing procedure. The antecedent and consequent parts of the justificatory rules are provided in natural forms. The effectiveness of the algorithm is tested on the vowel recognition problem and on two sets of artificially generated nonconvex pattern classes.
---
paper_title: Connectionist expert systems

paper_content:

Connectionist networks can be used as expert system knowledge bases. Furthermore, such networks can be constructed from training examples by machine learning techniques. This gives a way to automate the generation of expert systems for classification problems.
---
paper_title: Neural expert system using fuzzy teaching input and its application to medical diagnosis

paper_content:

Abstract Genetic algorithms (GAs) are inspired by Darwin's the survival of the fittest theory. This paper discusses a genetic algorithm that can automatically generate test cases to test a selected path. This algorithm takes a selected path as a target and executes sequences of operators iteratively for test cases to evolve. The evolved test case will lead the program execution to achieve the target path. To determine which test cases should survive to produce the next generation of fitter test cases, a metric named normalized extended Hamming distance (NEHD, which is used to determine whether the final test case is found) is developed. Based on NEHD, a fitness function named SIMILARITY is defined to determine which test cases should survive if the final test case has not been found. Even when there are loops in the target path, SIMILARITY can help the algorithm to lead the execution to flow along the target path.
---
paper_title: Use of fuzzy-logic-inspired features to improve bacterial recognition through classifier fusion

paper_content:

Escherichia coli O157:H7 has been found to cause serious health problems. Traditional methods to identify the organism are quite slow, pulsed-held gel electrophoresis (PFGE) images contain "banding pattern" information which can be used to recognize the bacteria. A fuzzy logic rule-based system is used as a guide to find a good feature set for the recognition of E. coli O157:H7. While the fuzzy rule-based system achieved good recognition, the human inspired features used in the rules were incorporated into a multiple neural network fusion approach which gave excellent separation of the target bacteria. The fuzzy integral was utilized in the fusion of neural networks trained with different feature sets to reach an almost perfect classification rate of E. coli O157:H7 PFGE patterns made available for the experiments.
---
paper_title: Use of neural network techniques in a medical expert system

paper_content:

Expert systems in medicine have relied heavily upon knowledge-based techniques, in which decision making rules or strategies are derived through consultation with experts. These techniques, coupled with methods of approximate reasoning, have produced systems which model the human decision making process. This approach has the disadvantage of requiring extensive interviewing of experts for each new application. It is desirable to be able to supplement this information by extracting information directly from data bases, without expert intervention. In this article, a neural network model is used to extract this information, and then use it in conjunction with rule-based knowledge, incorporating techniques of approximate reasoning.
---
paper_title: Review Neuro-fuzzy computing for image processing and pattern recognition

paper_content:

The relevance of integration of the merits of fuzzy set theory and neural network models for designing an efficient decision making system is explained. The feasibility of such systems and different ways of integration, so far made, in the context of image processing and pattern recognition are described. Scope for further research and development is outlined. An extensive bibliography is also provided.
---
paper_title: Fuzzy basis functions, universal approximation, and orthogonal least-squares learning.

paper_content:

Fuzzy systems are represented as series expansions of fuzzy basis functions which are algebraic superpositions of fuzzy membership functions. Using the Stone-Weierstrass theorem, it is proved that linear combinations of the fuzzy basis functions are capable of uniformly approximating any real continuous function on a compact set to arbitrary accuracy. Based on the fuzzy basis function representations, an orthogonal least-squares (OLS) learning algorithm is developed for designing fuzzy systems based on given input-output pairs; then, the OLS algorithm is used to select significant fuzzy basis functions which are used to construct the final fuzzy system. The fuzzy basis function expansion is used to approximate a controller for the nonlinear ball and beam system, and the simulation results show that the control performance is improved by incorporating some common-sense fuzzy control rules.
---
paper_title: Constructing fuzzy model by self-organizing counterpropagation network

paper_content:

This paper describes a general and systematic approach to constructing a multivariable fuzzy model from numerical data through a self-organizing counterpropagation network (SOCPN). Two self-organizing algorithms USOCPN and SSOCPN, being unsupervised and supervised respectively, are introduced. SOCPN can be employed in two ways. In the first place, it can be used as a knowledge extractor by which a set of rules are generated from the available numerical data set. The generated rule-base is then utilized by a fuzzy reasoning model. The second use of the SOCPN is as an online adaptive fuzzy model in which the rule-base in terms of connection weights is updated successively in response to the incoming measured data. The comparative results on three well studied examples suggest that the method has merits of simple structure, fast learning speed, and good modeling accuracy. >
---
paper_title: Nonlinear system modeling by competitive learning and adaptive fuzzy inference system

paper_content:

Modeling nonlinear systems by neural networks and fuzzy systems encounters problems such as the conflict between overfitting and good generalization and low reliability, which requires a great number of fuzzy rules or neural nodes and uses very complicated learning algorithms. A new adaptive fuzzy inference system, combined with a learning algorithm, is proposed to cope with these problems. First, the algorithm partitions the input space into some local regions by competitive learning, then it determines the decision boundaries for local input regions, and finally, based on the decision boundaries, it learns the fuzzy rule for each local region by recursive least squares (RLS). In the learning algorithm, the key role of the decision boundaries is highly emphasized. To demonstrate the validity of the proposed learning approach and the new adaptive fuzzy inference system, four examples are studied by the proposed method and compared with the previous results.
---
paper_title: A learning algorithm of fuzzy neural networks with triangular fuzzy weights

paper_content:

Abstract In this paper, first we propose an architecture of fuzzy neural networks with triangular fuzzy weights. The proposed fuzzy neural network can handle fuzzy input vectors as well as real input vectors. In both cases, outputs from the fuzzy neural network are fuzzy vectors. The input-output relation of each unit of the fuzzy neural network is defined by the extension principle of Zadeh. Next we define a cost function for the level sets (i.e., α-cuts) of fuzzy outputs and fuzzy targets. Then we derive a learning algorithm from the cost function for adjusting three parameters of each triangular fuzzy weight. Finally, we illustrate our approach by computer simulations on numerical examples.
---
paper_title: Learning by fuzzified neural networks

paper_content:

Abstract We derive a general learning algorithm for training a fuzzified feedforward neural networks that has fuzzy inputs, fuzzy targets, and fuzzy conncetion weights. The derived algorithm is applicable to the learning of fuzzy connection weights with various shapes such as triangular and trapezoid. First we briefly describe how a feedforward neural network can be fuzzified. Inputs, targets, and connection weights in the fiuzzified neural network can be fuzzy numbers. Next we define a cost function that measures the differences between a fuzzy target vector and an actual fuzzy output vector. Then we derive a learning algorithm from the cost function for adjusting fuzzy connection weights. Finally we show some results of computer simulations.
---
paper_title: Neural network implementation of fuzzy logic

paper_content:

Abstract Fuzzy logic has gained increased attention as a methodology for managing uncertainty in a rule-based structure. In a fuzzy logic inference system, more rules can fire at any given time than in a crisp expert system. Since the propositions are modelled as possibility distributions, there is a considerable computation load on the inference engine. In this paper, a neural network structure is proposed as a means of performing fuzzy logic inference. Three variations of the network are described, but in each case, the knowledge of the rule (i.e., the antecedent and consequent clauses) are explicitly encoded in the weights of the net. The theoretical properties of this structure are developed. In fact, the network reduces to crisp modus ponens when the inputs are crisp sets. Also, under suitable conditions the degree of specificity of the consequences of the inference is a monotone function of the degree of specificity of the input. Several simulation studies are included to illustrate the performance of the fuzzy logic inference networks.
---
paper_title: Neural networks that learn from fuzzy if-then rules

paper_content:

An architecture for neural networks that can handle fuzzy input vectors is proposed, and learning algorithms that utilize fuzzy if-then rules as well as numerical data in neural network learning for classification problems and for fuzzy control problems are derived. The learning algorithms can be viewed as an extension of the backpropagation algorithm to the case of fuzzy input vectors and fuzzy target outputs. Using the proposed methods, linguistic knowledge from human experts represented by fuzzy if-then rules and numerical data from measuring instruments can be integrated into a single information processing system (classification system or fuzzy control system). It is shown that the scheme works well for simple examples. >
---
paper_title: Implementation of conjunctive and disjunctive fuzzy logic rules with neural networks

paper_content:

Abstract The use of fuzzy logic to model and manage uncertainty in a rule-based system places high computational demands on an inference engine. In an earlier paper, we introduced trainable neural network structures for fuzzy logic. These networks can learn and extrapolate complex relationships between possibility distributions for the antecedents and consequents in the rules. In this paper, the power of these networks is further explored. The sensitivity of the output to noisy input distributions (which are likely if the clauses are generated from real data) is demonstrated as well as the ability of the networks to internalize multiple conjunctive clause and disjunctive clause rules. Since different rules (with the same variables) can be encoded in a single network, this approach to fuzzy logic inference provides a natural mechanism for rule conflict resolution.
---
paper_title: Interpolation of fuzzy if-then rules by neural networks

paper_content:

Abstract A number of approaches have been proposed for implementing fuzzy if-then rules with trainable multilayer feedforward neural networks. In these approaches, learning of neural networks is performed for fuzzy inputs and fuzzy targets. Because the standard back-propagation (BP) algorithm cannot be directly applied to fuzzy data, transformation of fuzzy data into non-fuzzy data or modification of the learning algorithm is required. Therefore the approaches for implementing fuzzy if-then rules can be classified into two main categories: introduction of preprocessors of fuzzy data and modification of the learning algorithm. In the first category, the standard BP algorithm can be employed after generating non-fuzzy data from fuzzy data by preprocessors. Two kinds of preprocessors based on membership values and level sets are examined in this paper. In the second category, the standard BP algorithm is modified to directly handle the level sets (i.e., intervals) of fuzzy data. This paper examines the ability of each approach to interpolate sparse fuzzy if-then rules. By computer simulations, high fitting ability of approaches in the first category and high interpolating ability of those in the second category are demonstrated.
---
paper_title: Fuzzy classifications using fuzzy inference networks

paper_content:

In this paper, fuzzy inference models for pattern classifications have been developed and fuzzy inference networks based on these models are proposed. Most of the existing fuzzy rule-based systems have difficulties in deriving inference rules and membership functions directly from training data. Rules and membership functions are obtained from experts. Some approaches use backpropagation (BP) type learning algorithms to learn the parameters of membership functions from training data. However, BP algorithms take a long time to converge and they require an advanced setting of the number of inference rules. The work to determine the number of inference rules demands lots of experiences from the designer. In this paper, self-organizing learning algorithms are proposed for the fuzzy inference networks. In the proposed learning algorithms, the number of inference rules and the membership functions in the inference rules will be automatically determined during the training procedure. The learning speed is fast. The proposed fuzzy inference network (FIN) classifiers possess both the structure and the learning ability of neural networks, and the fuzzy classification ability of fuzzy algorithms. Simulation results on fuzzy classification of two-dimensional data are presented and compared with those of the fuzzy ARTMAP. The proposed fuzzy inference networks perform better than the fuzzy ARTMAP and need less training samples.
---
paper_title: Improving classification performance using fuzzy MLP and two-level selective partitioning of the feature space

paper_content:

A fuzzy MLP model, developed by one of the authors, is used for obtaining selective two-level partitioning of the feature space in order to improve its classification performance. The model can handle uncertainty and/or impreciseness in the input as well as the output. The input to the network is modelled in terms of linguistic pi-sets whose centres and radii along the feature axes in each partition are generated automatically from the distribution of the training data. The performance of the model at the end of the first stage is used as a criterion for guiding the selection of the appropriate partition to be subdivided at the second stage, in order to improve the effectiveness of the model. A comparative study of the performance of the two-level technique with other methods, viz., the conventional MLP, linear discriminant analysis and the k-nearest neighbours algorithms, is also provided to demonstrate its superiority.
---
paper_title: Some neural net realizations of fuzzy reasoning

paper_content:

In this paper we analyze the neural network implementation of fuzzy logic proposed by Keller et al. [Fuzzy Sets Syst., 45, 1-12 (1992)], derive a learning algorithm for obtaining an optimal α for the net, and, for a special case, we show how one can directly (avoiding training) compute the optimal α. We address how training data can be generated for such a system. Effectiveness of the optimal α is then established through numerical examples. In this regard, several indices for performance evaluation are discussed. Finally, we propose a new architecture and demonstrate its effectiveness with numerical examples.
---
paper_title: Neural networks designed on approximate reasoning architecture and their applications

paper_content:

The NARA (neural networks based on approximate reasoning architecture) model is proposed and its composition procedure and evaluation are described. NARA is a neural network (NN) based on the structure of fuzzy inference rules. The distinctive feature of NARA is that its internal state can be analyzed according to the rule structure, and the problematic portion can be easily located and improved. The ease with which performance can be improved is shown by applying the NARA model to pattern classification problems. The NARA model is shown to be more efficient than ordinary NN models. In NARA, characteristics of the application task can be built into the NN model in advance by employing the logic structure, in the form of fuzzy inference rules. Therefore, it is easier to improve the performance of NARA, in which the internal state can be observed because of its structure, than that of an ordinary NN model, which is like a black box. Examples are introduced by applying the NARA model to the problems of auto adjustment of VTR tape running mechanisms and alphanumeric character recognition. >
---
paper_title: ANFIS: Adaptive-Network-Based Fuzzy Inference System

paper_content:

The architecture and learning procedure underlying ANFIS (adaptive-network-based fuzzy inference system) is presented, which is a fuzzy inference system implemented in the framework of adaptive networks. By using a hybrid learning procedure, the proposed ANFIS can construct an input-output mapping based on both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs. In the simulation, the ANFIS architecture is employed to model nonlinear functions, identify nonlinear components on-line in a control system, and predict a chaotic time series, all yielding remarkable results. Comparisons with artificial neural networks and earlier work on fuzzy modeling are listed and discussed. Other extensions of the proposed ANFIS and promising applications to automatic control and signal processing are also suggested. >
---
paper_title: Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction

paper_content:

Abstract In this paper we describe a neuro-fuzzy system with adaptive capability to extract fuzzy If Then rules from input and output sample data through learning. The proposed system, called radial basis function (RBF) based adaptive fuzzy system (AFS), employs the Gaussian functions to represent the membership functions of the premise part of fuzzy rules. Three architectural deviations of the RBF based AFS are also presented according to different consequence types such as constant, first-order linear function, and fuzzy variable. These provide versatility of the network to handle arbitrary fuzzy inference schemes. We present examples of system identification and time series prediction to illustrate how to solve these problems and to demonstrate its validity and effectiveness using the RBF based AFS.
---
paper_title: Fuzzy inference system learning by reinforcement methods

paper_content:

Fuzzy Actor-Critic Learning (FACL) and Fuzzy Q-Learning (FQL) are reinforcement learning methods based on dynamic programming (DP) principles. In the paper, they are used to tune online the conclusion part of fuzzy inference systems (FIS). The only information available for learning is the system feedback, which describes in terms of reward and punishment the task the fuzzy agent has to realize. At each time step, the agent receives a reinforcement signal according to the last action it has performed in the previous state. The problem involves optimizing not only the direct reinforcement, but also the total amount of reinforcements the agent can receive in the future. To illustrate the use of these two learning methods, we first applied them to a problem that involves finding a fuzzy controller to drive a boat from one bank to another, across a river with a strong nonlinear current. Then, we used the well known Cart-Pole Balancing and Mountain-Car problems to be able to compare our methods to other reinforcement learning methods and focus on important characteristic aspects of FACL and FQL. We found that the genericity of our methods allows us to learn every kind of reinforcement learning problem (continuous states, discrete/continuous actions, various type of reinforcement functions). The experimental studies also show the superiority of these methods with respect to the other related methods we can find in the literature.
---
paper_title: Fuzzy Finite-state Automata Can Be Deterministically Encoded into Recurrent Neural Networks

paper_content:

There has been an increased interest in combining fuzzy systems with neural networks because fuzzy neural systems merge the advantages of both paradigms. On the one hand, parameters in fuzzy systems have clear physical meanings and rule-based and linguistic information can be incorporated into adaptive fuzzy systems in a systematic way. On the other hand, there exist powerful algorithms for training various neural network models. However, most of the proposed combined architectures are only able to process static input-output relationships; they are not able to process temporal input sequences of arbitrary length. Fuzzy finite-state automats (FFAs) can model dynamical processes whose current state depends on the current input and previous states. Unlike in the case of deterministic finite-state automats (DFAs), FFAs are not in one particular state, rather each state is occupied to some degree defined by a membership function. Based on previous work on encoding DFAs in discrete-time second-order recurrent neural networks, we propose an algorithm that constructs an augmented recurrent neural network that encodes a FFA and recognizes a given fuzzy regular language with arbitrary accuracy. We then empirically verify the encoding methodology by correct string recognition of randomly generated FFAs. In particular, we examine how the networks' performance varies as a function of synaptic weight strengths.
---
paper_title: Learning and tuning fuzzy logic controllers through reinforcements

paper_content:

A method for learning and tuning a fuzzy logic controller based on reinforcements from a dynamic system is presented. It is shown that: the generalized approximate-reasoning-based intelligent control (GARIC) architecture learns and tunes a fuzzy logic controller even when only weak reinforcement, such as a binary failure signal, is available; introduces a new conjunction operator in computing the rule strengths of fuzzy control rules; introduces a new localized mean of maximum (LMOM) method in combining the conclusions of several firing control rules; and learns to produce real-valued control actions. Learning is achieved by integrating fuzzy inference into a feedforward network, which can then adaptively improve performance by using gradient descent methods. The GARIC architecture is applied to a cart-pole balancing system and demonstrates significant improvements in terms of the speed of learning and robustness to changes in the dynamic system's parameters over previous schemes for cart-pole balancing. >
---
paper_title: An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller

paper_content:

This paper describes an experiment on the “linguistic” synthesis of a controller for a model industrial plant (a steam engine). Fuzzy logic is used to convert heuristic control rules stated by a human operator into an automatic control strategy. The experiment was initiated to investigate the possibility of human interaction with a learning controller. However, the control strategy set up linguistically proved to be far better than expected in its own right, and the basic experiment of linguistic control synthesis in a non-learning controller is reported here.
---
paper_title: Recurrent neuro-fuzzy networks for nonlinear process modeling

paper_content:

A type of recurrent neuro-fuzzy network is proposed in this paper to build long-term prediction models for nonlinear processes. The process operation is partitioned into several fuzzy operating regions. Within each region, a local linear model is used to model the process. The global model output is obtained through the centre of gravity defuzzification which is essentially the interpolation of local model outputs. This modeling strategy utilizes both process knowledge and process input/output data. Process knowledge is used to initially divide the process operation into several fuzzy operating regions and to set up the initial fuzzification layer weights. Process I/O data are used to train the network. Network weights are such trained so that the long-term prediction errors are minimized. Through training, membership functions of fuzzy operating regions are refined and local models are learnt. Based on the recurrent neuro-fuzzy network model, a novel type of nonlinear model-based long range predictive controller can be developed and it consists of several local linear model-based predictive controllers. Local controllers are constructed based on the corresponding local linear models and their outputs are combined to form a global control action by using their membership functions. This control strategy has the advantage that control actions can be calculated analytically avoiding the time consuming nonlinear programming procedures required in conventional nonlinear model-based predictive control. The techniques have been successfully applied to the modeling and control of a neutralization process.
---
paper_title: Fuzzy identification of systems and its applications to modeling and control

paper_content:

A mathematical tool to build a fuzzy model of a system where fuzzy implications and reasoning are used is presented. The premise of an implication is the description of fuzzy subspace of inputs and its consequence is a linear input-output relation. The method of identification of a system using its input-output data is then shown. Two applications of the method to industrial processes are also discussed: a water cleaning process and a converter in a steel-making process.
---
paper_title: A fuzzy neural network for rule acquiring on fuzzy control systems

paper_content:

This paper presents a layer-structured fuzzy neural network (FNN) for learning rules of fuzzy-logic control systems. Initially, FNN is constructed to contain all the possible fuzzy rules. We propose a two-phase learning procedure for this network. The first phase is a error-backprop (EBP) training, and the second phase is a rule pruning. Since some functions of the nodes in the FNN have the competitive characteristics, the EBP training will converge quickly. After the training, a pruning process is performed to delete redundant rules for obtaining a concise fuzzy rule base. Simulation results show that for the truck backer-upper control problem, the training phase learns the knowledge of fuzzy rules in several dozen epochs with an error rate of less than 1%. Moreover, the fuzzy rule base generated by the pruning process contains only 14% of the initial fuzzy rules and is identical to the target fuzzy rule base.
---
paper_title: An adaptive fuzzy neural network for MIMO system model approximation in high-dimensional spaces

paper_content:

An adaptive fuzzy system implemented within the framework of neural network is proposed. The integration of the fuzzy system into a neural network enables the new fuzzy system to have learning and adaptive capabilities. The proposed fuzzy neural network can locate its rules and optimize its membership functions by competitive learning, Kalman filter algorithm and extended Kalman filter algorithms. A key feature of the new architecture is that a high dimensional fuzzy system can be implemented with fewer number of rules than the Takagi-Sugeno fuzzy systems. A number of simulations are presented to demonstrate the performance of the proposed system including modeling nonlinear function, operator's control of chemical plant, stock prices and bioreactor (multioutput dynamical system).
---
paper_title: Foundations Of Neuro-Fuzzy Systems

paper_content:

From the Publisher: ::: Foundations of Neuro-Fuzzy Systems reflects the current trend in intelligent systems research towards the integration of neural networks and fuzzy technology. The authors demonstrate how a combination of both techniques enhances the performance of control, decision-making and data analysis systems. Smarter and more applicable structures result from marrying the learning capability of the neural network with the transparency and interpretability of the rule-based fuzzy system. Foundations of Neuro-Fuzzy Systems highlights the advantages of integration making it a valuable resource for graduate students and researchers in control engineering, computer science and applied mathematics. The authors' informed analysis of practical neuro-fuzzy applications will be an asset to industrial practitioners using fuzzy technology and neural networks for control systems, data analysis and optimization tasks.
---
paper_title: Neuro-Fuzzy Systems for Function Approximation

paper_content:

Abstract We present a neuro-fuzzy architecture for function approximation based on supervised learning. The learning algorithm is able to determine the structure and the parameters of a fuzzy system. The approach is an extension to our already published NEFCON and NEFCLASS models which are used for control or classification purposes. The proposed extended model, which we call NEFPROX, is more general and can be used for any application based on function approximation.
---
paper_title: Handling the nonlinearity of a fuzzy logic controller at the transition between rules

paper_content:

Abstract A fuzzy logic controller approximates a desired control surface by using the outputs of its fuzzy control rules. The shape of this control surface is mainly influenced by the linguistic variables of the rule base, the basic operators of the fuzzy sets, the implication, the inference and the defuzzification method. The linearity/nonlinearity of this control surface is discussed in many studies, however none of those studies has explicitly considered the linearity/nonlinearity at the transition between fuzzy logic rules. In this paper a simple approach to control the linearity/nonlinearity at this transition by a modified center of gravity defuzzification method is proposed, namely by introducing the so-called defuzzification weights to the overlapping areas of the consequent. Those defuzzification weights can be determined heuristically or automatically. Both cases will be demonstrated, for the latter a feedforward neural network is employed.
---
paper_title: Using Fuzzy Logic for Performance Evaluation in Reinforcement Learning

paper_content:

Abstract A new architecture is described which uses fuzzy rules to initialize its two neural networks: a neural network for performance evaluation and another for action selection. This architecture, applied to control of dynamic systems, demonstrates that it is possible to start with an approximate prior knowledge and learn to refine it through experiments using Reinforcement Learning (RL).
---
paper_title: Manufacturing process control through integration of neural networks and fuzzy model

paper_content:

Artificial neural networks (ANNs) and fuzzy logic have been widely applied in many areas. This research is trying to discuss the integration of these two technologies. Three fuzzy models are utilized to update dynamically the training parameters in order to speed up the training. In addition, a fuzzy model is proposed which is self-organizing and self-adjusting, and able to learn from experience. In a self-organizing and self-adjusting fuzzy model (SOSAFM), the inputs and outputs are partitioned by Kohonen's feature mapping and the premise and consequence parameters are updated through an error backpropagation (EBP)-type learning algorithm. Physical experiments for manufacturing process control are implemented to evaluate the proposed methods. The results showed that updating the training parameters by using fuzzy models can accelerate the training speed. Moreover, SOSAFM is better than the multiple regression and artificial neural network both in speed and accuracy for the purpose of multi-sensor integration.
---
paper_title: A new type of fuzzy neural network based on a truth space approach for automatic acquisition of fuzzy rules with linguistic hedges

paper_content:

Abstract Fuzzy reasoning methods are generally classified into two approaches: the direct approach and the truth space approach. Several researches on the relationships between these approaches have been reported. There has been, however, no research which discusses their utility. The authors have previously proposed four types of fuzzy neural networks (FNNs) called Type I, II, III, and IV. The FNNs can identify the fuzzy rules and tune the membership functions of fuzzy reasoning automatically, utilizing the learning capability of neural networks. Types III and IV, which are based on the truth space approach, can acquire linguistic fuzzy rules with the fuzzy variables in the consequences labeled according to their linguistic truth values (LTVs). However, the expressions available for the linguistic labeling are limited, since the LTVs are singletons. This paper presents a new type of FNN based on the truth space approach for automatic acquisition of the fuzzy rules with linguistic hedges. The new FNN, called Type V, has the LTVs defined by fuzzy sets for fuzzy rules and can express the identified fuzzy rules linguistically using the fuzzy variables in the consequences with linguistic hedges. Two simulations are done for demonstrating the feasibility of the new method. The results show that the truth space approach makes the fuzzy rules easy to understand.
---
paper_title: Extraction of Rules from Discrete-time Recurrent Neural Networks

paper_content:

The extraction of symbolic knowledge from trained neural networks and the direct encoding of (partial) knowledge into networks prior to training are important issues. They allow the exchange of information between symbolic and connectionist knowledge representations. The focus of this paper is on the quality of the rules that are extracted from recurrent neural networks. Discrete-time recurrent neural networks can be trained to correctly classify strings of a regular language. Rules defining the learned grammar can be extracted from networks in the form of deterministic finite-state automata (DFAs) by applying clustering algorithms in the output space of recurrent state neurons. Our algorithm can extract different finite-state automata that are consistent with a training set from the same network. We compare the generalization performances of these different models and the trained network and we introduce a heuristic that permits us to choose among the consistent DFAs the model which best approximates the learned regular grammar.
---
paper_title: Real-time stable self-learning FNN controller using genetic algorithm

paper_content:

Abstract A kind of real-time stable self-learning fuzzy neural network (FNN) control system is proposed in this paper. The control system is composed of two parts: (1) A FNN controller which use genetic algorithm (GA) to search optimal fuzzy rules and membership functions for the unknown controlled plant; (2) A supervisor which can guarantee the stability of the control system during the real-time learning stage, since the GA has some random property which may cause control system unstable. The approach proposed in this paper combine a priori knowledge of designer and the learning ability of FNN to achieve optimal fuzzy control for an unknown plant in real-time. The efficiency of the approach is verified by computer simulation.
---
paper_title: Linguistic rule extraction from neural networks and genetic-algorithm-based rule selection

paper_content:

This paper proposes a hybrid approach to the design of a compact fuzzy rule-based classification system with a small number of linguistic rules. The proposed approach consists of two procedures: rule extraction from a trained neural network and rule selection by a genetic algorithm. We first describe how linguistic rules can be extracted from a multilayer feedforward neural network that has been already trained for a classification problem with many continuous attributes. In our rule extraction procedure, a linguistic input vector corresponding to the antecedent part of a linguistic rule is presented to the trained neural network, and the fuzzy output vector front the trained neural network is examined for determining the consequent part and the grade of certainty of that linguistic rule. Next we explain how a genetic algorithm can be utilized for selecting a small number of significant linguistic rules from a large number of extracted rules. Our rule selection problem has two objectives: to minimize the number of selected linguistic rules and to maximize the number of correctly classified patterns by the selected linguistic rules. A multi-objective genetic algorithm is employed for finding a set of non-dominated solutions with respect to these two objectives. Finally we illustrate our hybrid approach by computer simulations on real-world test problems.
---
paper_title: Selecting fuzzy if-then rules for classification problems using genetic algorithms

paper_content:

This paper proposes a genetic-algorithm-based method for selecting a small number of significant fuzzy if-then rules to construct a compact fuzzy classification system with high classification power. The rule selection problem is formulated as a combinatorial optimization problem with two objectives: to maximize the number of correctly classified patterns and to minimize the number of fuzzy if-then rules. Genetic algorithms are applied to this problem. A set of fuzzy if-then rules is coded into a string and treated as an individual in genetic algorithms. The fitness of each individual is specified by the two objectives in the combinatorial optimization problem. The performance of the proposed method for training data and test data is examined by computer simulations on the iris data of Fisher. >
---
paper_title: Heuristic constraints enforcement for training of and knowledge extraction from a fuzzy/neural architecture. I. Foundation

paper_content:

Using fuzzy/neural architectures to extract heuristic information from systems has received increasing attention. A number of fuzzy/neural architectures and knowledge extraction methods have been proposed. Knowledge extraction from systems where the existing knowledge limited is a difficult task. One of the reasons is that there is no ideal rulebase, which can be used to validate the extracted rules. In most of the cases, using output error measures to validate extracted rules is not sufficient as extracted knowledge may not make heuristic sense, even if the output error may meet the specified criteria. The paper proposes a novel method for enforcing heuristic constraints on membership functions for rule extraction from a fuzzy/neural architecture. The proposed method not only ensures that the final membership functions conform to a priori heuristic knowledge, but also reduces the domain of search of the training and improves convergence speed. Although the method is described on a specific fuzzy/neural architecture, it is applicable to other realizations, including adaptive or static fuzzy inference systems. The foundations of the proposed method are given in Part I. The techniques for implementation and integration into the training are given in Part II, together with applications.
---
paper_title: A neural network based fuzzy set model for organizational decision making

paper_content:

A neural network based fuzzy set model is proposed to support organizational decision making under uncertainty. This model incorporates three theories and methodologies: classical decision-making theory under conflict, as suggested by Luce and Raiffa (1957), the fuzzy set theory of Zadeh (1965, 1984), and a modified version of the backpropagation (BP) neural network algorithm originated by Rumelhart et al. (1986). An algorithm that implements the model is described, and an application of the model to a real data example is used to demonstrate its use.
---
paper_title: A genetic-based neuro-fuzzy approach for modeling and control of dynamical systems

paper_content:

Linguistic modeling of complex irregular systems constitutes the heart of many control and decision making systems, and fuzzy logic represents one of the most effective algorithms to build such linguistic models. In this paper, a linguistic (qualitative) modeling approach is proposed. The approach combines the merits of the fuzzy logic theory, neural networks, and genetic algorithms (GAs). The proposed model is presented in a fuzzy-neural network (FNN) form which can handle both quantitative (numerical) and qualitative (linguistic) knowledge. The learning algorithm of a FNN is composed of three phases. The first phase is used to find the initial membership functions of the fuzzy model. In the second phase, a new algorithm is developed and used to extract the linguistic-fuzzy rules. In the third phase, a multiresolutional dynamic genetic algorithm (MRD-GA) is proposed and used for optimized tuning of membership functions of the proposed model. Two well-known benchmarks are used to evaluate the performance of the proposed modeling approach, and compare it with other modeling approaches.
---
paper_title: Logical operation based fuzzy MLP for classification and rule generation

paper_content:

A fuzzy layered neural network for classification and rule generation is proposed using logical neurons. It can handle uncertainty and/or impreciseness in the input as well as the output. Logical operators, namely, t-norm T and t-conorm S involving And and Or neurons, are employed in place of the weighted sum and sigmoid functions. Various fuzzy implication operators are introduced to incorporate different amounts of mutual interaction during the back propagation of erros. In case of partial inputs the model is capable of querying the user for the more important feature information, if and when required. Justification for an inferred decision may be produced in rule form. The built-in And-Or structure of the network enables the generation of appropriate rules expressed as the disjunction of conjunctive clauses. The effectiveness of the model is tested on a speech recognition problem and on some artificially generated pattern sets.
---
paper_title: Decision making on creditworthiness, using a fuzzy connectionist model

paper_content:

Abstract In this paper we present an unpolished expert system development tool, based on a connectionist architecture for knowledge representation. Our work is centered around a connectionist expert system, which can be expanded, and updated through learning of sample domain specific cases [1,4]. A cell recruitment learning algorithm [2] capable of forgetting previously learned facts by learning new ones is incorporated. Using this learning mechanism, we let the system learn a knowledge base on classifying the creditworthiness of credit applicants. The knowledge base consisted of 50 credit cases and was obtained from [11]. It will be shown that the fuzznet system is capable of learning such a model, with only a small amount of the cases being presented to it for learning. In all examples learned, an acceptable model was derived (based on the average prognostic error of actual and expected output for creditworthiness). The input features and their corresponding outputs (which were learned) are all fuzzy (uncertain) at the time of learning. So far implementations of connectionist expert systems, either only allowed for crisp inputs when placed in the learning mode, or only supported uncertainty when simulated [1, 8, 9, 10]. This example will further demonstrate the versatility of the fuzznet system, when dealing with uncertainty in the inputs and outputs when being placed in the learn mode, or while being simulated.
---
paper_title: Neural network implementation of fuzzy logic

paper_content:

Abstract Fuzzy logic has gained increased attention as a methodology for managing uncertainty in a rule-based structure. In a fuzzy logic inference system, more rules can fire at any given time than in a crisp expert system. Since the propositions are modelled as possibility distributions, there is a considerable computation load on the inference engine. In this paper, a neural network structure is proposed as a means of performing fuzzy logic inference. Three variations of the network are described, but in each case, the knowledge of the rule (i.e., the antecedent and consequent clauses) are explicitly encoded in the weights of the net. The theoretical properties of this structure are developed. In fact, the network reduces to crisp modus ponens when the inputs are crisp sets. Also, under suitable conditions the degree of specificity of the consequences of the inference is a monotone function of the degree of specificity of the input. Several simulation studies are included to illustrate the performance of the fuzzy logic inference networks.
---
paper_title: A neural fuzzy system with linguistic teaching signals

paper_content:

A neural fuzzy system learning with linguistic teaching signals is proposed. This system is able to process and learn numerical information as well as linguistic information. It can be used either as an adaptive fuzzy expert system or as an adaptive fuzzy controller. First, we propose a five-layered neural network for the connectionist realization of a fuzzy inference system. The connectionist structure can house fuzzy logic rules and membership functions for fuzzy inference. We use /spl alpha/-level sets of fuzzy numbers to represent linguistic information. The inputs, outputs, and weights of the proposed network can be fuzzy numbers of any shape. Furthermore, they can be hybrid of fuzzy numbers and numerical numbers through the use of fuzzy singletons. Based on interval arithmetics, two kinds of learning schemes are developed for the proposed system: fuzzy supervised learning and fuzzy reinforcement learning. Simulation results are presented to illustrate the performance and applicability of the proposed system. >
---
paper_title: Implementation of conjunctive and disjunctive fuzzy logic rules with neural networks

paper_content:

Abstract The use of fuzzy logic to model and manage uncertainty in a rule-based system places high computational demands on an inference engine. In an earlier paper, we introduced trainable neural network structures for fuzzy logic. These networks can learn and extrapolate complex relationships between possibility distributions for the antecedents and consequents in the rules. In this paper, the power of these networks is further explored. The sensitivity of the output to noisy input distributions (which are likely if the clauses are generated from real data) is demonstrated as well as the ability of the networks to internalize multiple conjunctive clause and disjunctive clause rules. Since different rules (with the same variables) can be encoded in a single network, this approach to fuzzy logic inference provides a natural mechanism for rule conflict resolution.
---
paper_title: Implementing fuzzy logic controllers using a neural network framework

paper_content:

Abstract We describe a system for implementing fuzzy logic controllers using a neural network. A significant aspect of this system is that the linguistic values associated with the fuzzy control rules can be general concave continuous fuzzy subsets. By using structures suggested by the fuzzy logic framework, we simplify the learning requirements. On the other hand the adaptive aspect of the neural framework allows for the necessary learning.
---
paper_title: Fuzzy rule generation methods for high-level computer vision

paper_content:

Abstract In many decision making systems involving multiple sources, the decisions made may be considered as the result of a rule-based system in which the decision rules are usually enumerated by experts or generated by a learning process. In this paper, we discuss the various issues involved in the generation of fuzzy rules automatically from training data for high-level computer vision. Features are treated as linguistic variables that appear in the antecedent clauses of the rules. We present methods to generate the corresponding linguistic labels (values) and their membership functions. Rules are generated by constructing a minimal approximate fuzzy aggregation network and then training the network using gradient descent methods. Several examples are given.
---
paper_title: Evidence aggregation networks for fuzzy logic inference

paper_content:

Fuzzy logic has been applied in many engineering disciplines. The problem of fuzzy logic inference is investigated as a question of aggregation of evidence. A fixed network architecture employing general fuzzy unions and intersections is proposed as a mechanism to implement fuzzy logic inference. It is shown that these networks possess desirable theoretical properties. Networks based on parameterized families of operators (such as Yager's union and intersection) have extra predictable properties and admit a training algorithm which produces sharper inference results than were earlier obtained. Simulation studies corroborate the theoretical properties. >
---
paper_title: Review Neuro-fuzzy computing for image processing and pattern recognition

paper_content:

The relevance of integration of the merits of fuzzy set theory and neural network models for designing an efficient decision making system is explained. The feasibility of such systems and different ways of integration, so far made, in the context of image processing and pattern recognition are described. Scope for further research and development is outlined. An extensive bibliography is also provided.
---
paper_title: Modeling and formulating fuzzy knowledge bases using neural networks

paper_content:

Abstract We show how the determination of the firing level of a neuron can be viewed as a measure of possibility between two fuzzy sets, the weights of connection and the input. We then suggest a way to represent fuzzy production rules in a neural framework. Central to this representation is the notion that the linguistic variables associated with the rule, the antecedent and consequent values, are represented as weights in the resulting neural structure. The structure used to represent these fuzzy rules allows learning of the membership grades of the associated linguistic variables. A self-organization procedure for obtaining the nucleus of rules for a fuzzy knowledge base is presented.
---
paper_title: POPFNN: a pseudo outer-product based fuzzy neural network

paper_content:

Abstract A novel fuzzy neural network, called the pseudo outer-product based fuzzy neural network (POPFNN), is proposed in this paper. The functions performed by each layer in the proposed POPFNN strictly correspond to the inference steps in the truth value restriction method in fuzzy logic [ Mantaras (1990) Approximate reasoning models, Ellis Horwood]. This correspondence gives it a strong theoretical basis. Similar to most of the existing fuzzy neural networks, the proposed POPFNN uses a self-organizing algorithm ( Kohonen, 1988 , Self-organization and associative memories, Springer) to learn and initialize the membership functions of the input and output variables from a set of training data. However, instead of employing the popularly used competitive learning [ Kosko (1990) IEEE Trans. Neural Networks, 3(5), 801], this paper proposes a novel pseudo outer-product (POP) learning algorithm to identify the fuzzy rules that are supported by the training data. The proposed POP learning algorithm is fast, reliable, and highly intuitive. Extensive experimental results and comparisons are presented at the end of the paper for discussion. Copyright © 1996 Elsevier Science Ltd.
---
paper_title: Learning capacity and sample complexity on expert networks

paper_content:

A major development in knowledge-based neural networks is the integration of symbolic expert rule-based knowledge into neural networks, resulting in so-called rule-based neural (or connectionist) networks. An expert network here refers to a particular construct in which the uncertainty management model of symbolic expert systems is mapped into the activation function of the neural network. This paper addresses a yet-to-be-answered question: Why can expert networks generalize more effectively from a finite number of training instances than multilayered perceptrons? It formally shows that expert networks reduce generalization dimensionality and require relatively small sample sizes for correct generalization.
---
paper_title: Knowledge-based connectionism for revising domain theories

paper_content:

A knowledge-based connectionist model for machine learning referred to as KBCNN is presented. In the KBCNN learning model, useful domain attributes and concepts are first identified and linked in a way consistent with initial domain knowledge, and then the links are weighted properly so as to maintain the semantics. Hidden units and additional connections may be introduced into this initial connectionist structure as appropriate. Then, this primitive structure evolves to minimize empirical error. The KBCNN learning model allows the theory learned or revised to be translated into the symbolic rule-based language that describes the initial theory. Thus, a domain theory can be pushed onto the network, revised empirically over time, and decoded in symbolic form. The domain of molecular genetics is used to demonstrate the validity of the KBCNN learning model and its superiority over related learning methods. >
---
paper_title: Survey and critique of techniques for extracting rules from trained artificial neural networks

paper_content:

It is becoming increasingly apparent that, without some form of explanation capability, the full potential of trained artificial neural networks (ANNs) may not be realised. This survey gives an overview of techniques developed to redress this situation. Specifically, the survey focuses on mechanisms, procedures, and algorithms designed to insert knowledge into ANNs (knowledge initialisation), extract rules from trained ANNs (rule extraction), and utilise ANNs to refine existing rule bases (rule refinement). The survey also introduces a new taxonomy for classifying the various techniques, discusses their modus operandi, and delineates criteria for evaluating their efficacy.
---
paper_title: Connectionist expert systems

paper_content:

Connectionist networks can be used as expert system knowledge bases. Furthermore, such networks can be constructed from training examples by machine learning techniques. This gives a way to automate the generation of expert systems for classification problems.
---
paper_title: Knowledge-Based Artificial Neural Networks

paper_content:

Abstract Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN ( Knowledge-Based Artificial Neural Networks ) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific “domain theories”, represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.
---
paper_title: A connectionist incremental expert system combining production systems and associative memory

paper_content:

A connectionist expert system model is proposed in this paper. The system combines the capability of production systems and associative memory. Rules are explicitly represented within a network structure. The Perceptron algorithm is generalized to include samples with uncertain components. A gradually-augmented-node learning algorithm is used to guarantee fast memorization of all rules. The convergence of the learning algorithms is analyzed. The system has a dynamic knowledge organization and adapts in real time to acquire new knowledge, or to relearn existing knowledge, through interaction with the user. This allows the system to be built incrementally. An example system is presented.
---
paper_title: Medical diagnostic expert system based on PDP model

paper_content:

The applicability of PDP (parallel distributed processing) models to knowledge processing is clarified. The authors evaluate the diagnostic capabilities of a prototype medical diagnostic expert system based on a multilayer network. After having been trained on only 300 patients, the prototype system shows diagnostic capabilities almost equivalent to those of a symbolic expert system. Symbolic knowledge is extracted from what the multilayer network has learned. The extracted knowledge is compared with doctors' knowledge. Moreover, a method to extract rules from the network and usage of the rules in a confirmation process are proposed. >
---
paper_title: Dynamically adding symbolically meaningful nodes to knowledge-based neural networks

paper_content:

Abstract Traditional connectionist theory-refinement systems map the dependencies of a domain-specific rule base into a neural network, and then refine this network using neural learning techniques. Most of these systems, however, lack the ability to refine their network's topology and are thus unable to add new rules to the (reformulated) rule base. Therefore, with domain theories that lack rules, generalization is poor, and training can corrupt the original rules — even those that were initially correct. The paper presents TopGen, an extension to the KBANN algorithm, which heuristically searches for possible expansions to the KBANN network. TopGen does this by dynamically adding hidden nodes to the neural representation of the domain theory, in a manner that is analogous to the adding of rules and conjuncts to the symbolic rule base. Experiments indicate that the method is able to heuristically find effective places to add nodes to the knowledge bases of four real-world problems, as well as an artificial chess domain. The experiments also verify that new nodes must be added in an intelligent manner. The algorithm showed statistically significant improvements over the KBANN algorithm in all five domains.
---
paper_title: The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks

paper_content:

To date, the preponderance of techniques for eliciting the knowledge embedded in trained artificial neural networks (ANN's) has focused primarily on extracting rule-based explanations from feedforward ANN's. The ADT taxonomy for categorizing such techniques was proposed in 1995 to provide a basis for the systematic comparison of the different approaches. This paper shows that not only is this taxonomy applicable to a cross section of current techniques for extracting rules from trained feedforward ANN's but also how the taxonomy can be adapted and extended to embrace a broader range of ANN types (e,g., recurrent neural networks) and explanation structures. In addition we identify some of the key research questions in extracting the knowledge embedded within ANN's including the need for the formulation of a consistent theoretical basis for what has been, until recently, a disparate collection of empirical results.
---
paper_title: Knowledge-based connectionism for revising domain theories

paper_content:

A knowledge-based connectionist model for machine learning referred to as KBCNN is presented. In the KBCNN learning model, useful domain attributes and concepts are first identified and linked in a way consistent with initial domain knowledge, and then the links are weighted properly so as to maintain the semantics. Hidden units and additional connections may be introduced into this initial connectionist structure as appropriate. Then, this primitive structure evolves to minimize empirical error. The KBCNN learning model allows the theory learned or revised to be translated into the symbolic rule-based language that describes the initial theory. Thus, a domain theory can be pushed onto the network, revised empirically over time, and decoded in symbolic form. The domain of molecular genetics is used to demonstrate the validity of the KBCNN learning model and its superiority over related learning methods. >
---
paper_title: Knowledge-based fuzzy MLP for classification and rule generation

paper_content:

A new scheme of knowledge-based classification and rule generation using a fuzzy multilayer perceptron (MLP) is proposed. Knowledge collected from a data set is initially encoded among the connection weights in terms of class a priori probabilities. This encoding also includes incorporation of hidden nodes corresponding to both the pattern classes and their complementary regions. The network architecture, in terms of both links and nodes, is then refined during training. Node growing and link pruning are also resorted to. Rules are generated from the trained network using the input, output, and connection weights in order to justify any decision(s) reached. Negative rules corresponding to a pattern not belonging to a class can also be obtained. These are useful for inferencing in ambiguous cases. Results on real life and synthetic data demonstrate that the speed of learning and classification performance of the proposed scheme are better than that obtained with the fuzzy and conventional versions of the MLP (involving no initial knowledge encoding). Both convex and concave decision regions are considered in the process.
---
paper_title: Cascade ARTMAP: Integrating Neural Computation and Symbolic Knowledge Processing

paper_content:

This paper introduces a hybrid system termed cascade adaptive resonance theory mapping (ARTMAP) that incorporates symbolic knowledge into neural-network learning and recognition. Cascade ARTMAP, a generalization of fuzzy ARTMAP, represents intermediate attributes and rule cascades of rule-based knowledge explicitly and performs multistep inferencing. A rule insertion algorithm translates if-then symbolic rules into cascade ARTMAP architecture. Besides that initializing networks with prior knowledge can improve predictive accuracy and learning efficiency, the inserted symbolic knowledge can be refined and enhanced by the cascade ARTMAP learning algorithm. By preserving symbolic rule form during learning, the rules extracted from cascade ARTMAP can be compared directly with the originally inserted rules. Simulations on an animal identification problem indicate that a priori symbolic knowledge always improves system performance, especially with a small training set. Benchmark study on a DNA promoter recognition problem shows that with the added advantage of fast learning, cascade ARTMAP rule insertion and refinement algorithms produce performance superior to those of other machine learning systems and an alternative hybrid system known as knowledge-based artificial neural network (KBANN). Also, the rules extracted from cascade ARTMAP are more accurate and much cleaner than the NofM rules extracted from KBANN.
---
paper_title: Learning fuzzy rules and approximate reasoning in fuzzy neural networks and hybrid systems

paper_content:

The paper considers both knowledge acquisition and knowledge interpretation tasks as tightly connected and continuously interacting processes in a contemporary knowledge engineering system. Fuzzy rules are used here as a framework for knowledge representation. An algorithm REFuNN for fuzzy rules extraction from adaptive fuzzy neural networks (FuNN) is proposed. A case study of Iris classification is chosen to illustrate the algorithm. Interpretation of fuzzy rules is possible by using fuzzy neural networks or by using standard fuzzy inference methods. Both approaches are compared in the paper based on the case example. A hybrid environment FuzzyCOPE which facilitates neural network simulation, fuzzy rules extraction from fuzzy neural networks and fuzzy rules interpretation by using different methods for approximate reasoning is briefly described.
---
paper_title: Inference, inquiry, evidence censorship, and explanation in connectionist expert systems

paper_content:

The combination of the techniques of expert systems and neural networks has the potential of producing more powerful systems, for example, expert systems able to learn from experience. In this paper, we address the combinatorial neural model (CNM), a kind of fuzzy neural network able to accommodate in a simple framework the highly desirable property of incremental learning, as well as the usual capabilities of expert systems. We show how an interval-based representation for membership grades makes CNM capable of reasoning with several types of uncertainty: vagueness, ignorance, and relevance commonly found in practical applications. In addition, we show how basic functions of expert systems such as inference, inquiry, censorship of input information, and explanation may be implemented. We also report experimental results of the application of CNM to the problem of deforestation monitoring of the Amazon region using satellite images.
---
paper_title: Adaptable neuro production systems

paper_content:

Abstract Connectionist production systems are neural network realizations of production rule-based systems. The connections are adjusted to a given set of rules to allow the system to perform reasoning. Adaptable connectionist production systems are introduced in this paper. They allow adaptation of the already pre-calculated connections to new data. The production rules are used to initialize the connection weights after which training with data occurs. At any time of the neural network operation, a set of updated rules can be extracted as a current knowledge base accumulated by the network. Using a set of rules for initializing a connectionist architecture before training may result in: (1) increase in the speed of training; (2) increase in the robustness of the neural network against the ‘catastrophic forgetting’ phenomenon; (3) better explanation of the learned by the network knowledge from data. In general, the proposed method facilitates building flexible and adaptable neuro-fuzzy production systems. This is demonstrated on a case problem of chaotic time series prediction.
---
paper_title: Knowledge-Based Artificial Neural Networks

paper_content:

Abstract Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN ( Knowledge-Based Artificial Neural Networks ) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific “domain theories”, represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.
---
paper_title: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps

paper_content:

A neural network architecture is introduced for incremental supervised learning of recognition categories and multidimensional maps in response to arbitrary sequences of analog or binary input vectors, which may represent fuzzy or crisp sets of features. The architecture, called fuzzy ARTMAP, achieves a synthesis of fuzzy logic and adaptive resonance theory (ART) neural networks by exploiting a close formal similarity between the computations of fuzzy subsethood and ART category choice, resonance, and learning. Four classes of simulation illustrated fuzzy ARTMAP performance in relation to benchmark backpropagation and generic algorithm systems. These simulations include finding points inside versus outside a circle, learning to tell two spirals apart, incremental approximation of a piecewise-continuous function, and a letter recognition database. The fuzzy ARTMAP system is also compared with Salzberg's NGE systems and with Simpson's FMMC system. >
---
paper_title: Hidden Patterns in Combined and Adaptive Knowledge Networks

paper_content:

Uncertain causal knowledge is stored in fuzzy cognitive maps (FCMs). FCMs are fuzzy signed digraphs with feedback. The sign (+ or -) of FCM edges indicates causal increase or causal decrease. The fuzzy degree of causality is indicated by a number in [- 1, 1]. FCMs learn by modifying their causal connections in sign and magnitude, structurally analogous to the way in which neural networks learn. An appropriate causal learning law for inductively inferring FCMs from time-series data is the differential Hebbian law, which modifies causal connections by correlating time derivatives of FCM node outputs. The differential Hebbian law contrasts with Hebbian output-correlation learning laws of adaptive neural networks. FCM nodes represent variable phenomena or fuzzy sets. An FCM node nonlinearly transforms weighted summed inputs into numerical output, again in analogy to a model neuron. Unlike expert systems, which are feedforward search trees, FCMs are nonlinear dynamical systems. FCM resonant states are limit cycles, or time-varying patterns. An FCM limit cycle or hidden pattern is an FCM inference. Experts construct FCMs by drawing causal pictures or digraphs. The corresponding connection matrices are used for inferencing. By additively combining augmented connection matrices, any number of FCMs can be naturally combined into a single knowledge network. The credibility wi in [0, 1] of the ith expert is included in this learning process by multiplying the ith expert's augmented FCM connection matrix by w i. Combining connection matrices is a simple type of adaptive inference. In general, connection matrices are modified by an unsupervised learning law, such as the
---
paper_title: Rule Revision with Recurrent Neural Networks

paper_content:

Recurrent neural networks readily process, recognize and generate temporal sequences. By encoding grammatical strings as temporal sequences, recurrent neural networks can be trained to behave like deterministic sequential finite-state automata. Algorithms have been developed for extracting grammatical rules from trained networks. Using a simple method for inserting prior knowledge (or rules) into recurrent neural networks, we show that recurrent neural networks are able to perform rule revision. Rule revision is performed by comparing the inserted rules with the rules in the finite-state automata extracted from trained networks. The results from training a recurrent neural network to recognize a known non-trivial, randomly-generated regular grammar show that not only do the networks preserve correct rules but that they are able to correct through training inserted rules which were initially incorrect (i.e. the rules were not the ones in the randomly generated grammar).
---
paper_title: Knowledge-Based Artificial Neural Networks

paper_content:

Abstract Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN ( Knowledge-Based Artificial Neural Networks ) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific “domain theories”, represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.
---
paper_title: Connectionist theory refinement: Genetically searching the space of network topologies

paper_content:

An algorithm that learns from a set of examples should ideally be able to exploit the available resources of (a) abundant computing power and (b) domain-specific knowledge to improve its ability to generalize. Connectionist theory-refinement systems, which use background knowledge to select a neural network's topology and initial weights, have proven to be effective at exploiting domain-specific knowledge; however, most do not exploit available computing power. This weakness occurs because they lack the ability to refine the topology of the neural networks they produce, thereby limiting generalization, especially when given impoverished domain theories. We present the Regent algorithm which uses (a) domain-specific knowledge to help create an initial population of knowledge-based neural networks and (b) genetic operators of crossover and mutation (specifically designed for knowledge-based networks) to continually search for better network topologies. Experiments on three real-world domains indicate that our new algorithm is able to significantly increase generalization compared to a standard connectionist theory-refinement system, as well as our previous algorithm for growing knowledge-based networks.
---
paper_title: Rule insertion and rule extraction from evolving fuzzy neural networks: algorithms and applications for building adaptive, intelligent expert systems

paper_content:

Discusses the concept of intelligent expert systems and suggests tools for building an adaptable, in an online or in an off-line mode, rule base during the system operation in a changing environment. It applies evolving fuzzy neural networks (EFuNNs) as associative memories for the purpose of dynamic storing and modifying a rule base. Algorithms for rule extraction and rule insertion from EFuNNs are explained and applied to a case study using gas furnace data and the iris data set.
---
paper_title: Rough fuzzy MLP: knowledge encoding and classification

paper_content:

A scheme of knowledge encoding in a fuzzy multilayer perceptron (MLP) using rough set-theoretic concepts is described. Crude domain knowledge is extracted from the data set in the form of rules. The syntax of these rules automatically determines the appropriate number of hidden nodes while the dependency factors are used in the initial weight encoding. The network is then refined during training. Results on classification of speech and synthetic data demonstrate the superiority of the system over the fuzzy and conventional versions of the MLP (involving no initial knowledge).
---
paper_title: Combining rough sets learning- and neural learning-method to deal with uncertain and imprecise information

paper_content:

Abstract Any system designed to reason about the real world must be capable of dealing with uncertainty. The complexity of the real world and the finite size of most knowledge bases pose significant difficulties for the traditional concept of the learning system. Experience has shown that many learning paradigms fail to scale up to those problems. One response to these failures has been to construct systems which use multiple learning paradigms. Thus the strengths of one paradigm counterbalance some of the weaknesses of the others. As a result the effectiveness of the overall system will be enhanced. Consequently, integrated techniques have been widespread over the last years. A multistrategy which addresses those issues is presented. This approach joins two forms of learning, the technique of neural networks and rough sets. These seem at first quite different but they share the common ability to work well in a natural environment. In a closed loop fashion we will achieve more robust concept learning capabilities for a variety of difficult classification tasks. The objective of integration is twofold: (i) to improve the overall classification effectiveness of learned objects' description, (ii) to refine the dependency factors of the rules.
---
paper_title: Evolutionary Modular Design of Rough Knowledge-based Network using Fuzzy Attributes

paper_content:

This article describes a way of integrating rough set theory with a fuzzy MLP using a modular evolutionary algorithm, for classification and rule generation in soft computing paradigm. The novelty of the method lies in applying rough set theory for extracting dependency rules directly from a real-valued attribute table consisting of fuzzy membership values. This helps in preserving all the class representative points in the dependency rules by adaptively applying a threshold that automatically takes care of the shape of membership functions. An l-class classification problem is split into l two-class problems. Crude subnetwork modules are initially encoded from the dependency rules. These subnetworks are then combined and the final network is evolved using a GA with restricted mutation operator which utilizes the knowledge of the modular structure already generated, for faster convergence. The GA tunes the fuzzification parameters, and network weight and structure simultaneously, by optimising a single fitness function. This methodology helps in imposing a structure on the weights, which results in a network more suitable for rule generation. Performance of the algorithm is compared with related techniques.
---
paper_title: Staging of cervical cancer with soft computing

paper_content:

Describes a way of designing a hybrid decision support system in soft computing paradigm for detecting the different stages of cervical cancer. Hybridization includes the evolution of knowledge-based subnetwork modules with genetic algorithms (CIAs) using rough set theory and the Interactive Dichotomizer 3 (ID3) algorithm. Crude subnetworks obtained via rough set theory and the ID3 algorithm are evolved using CAs. The evolution uses a restricted mutation operator which utilizes the knowledge of the modular structure, already generated, for faster convergence. The CA tunes the network weights and structure simultaneously. The aforesaid integration enhances the performance in terms of classification score, network size and training time, as compared to the conventional multilayer perceptron. This methodology also helps in imposing a structure on the weights, which results in a network more suitable for extraction of logical rules and human interpretation of the inferencing procedure.
---
paper_title: Rough knowledge-based network, fuzziness and classification

paper_content:

A method of integrating rough sets and fuzzy multilayer perceptron (MLP) for designing a knowledge-based network for pattern recognition problems is described. Rough set theory is used to extract crude knowledge from the input domain in the form of rules. The syntax of these rules automatically determines the optimal number of hidden nodes while the dependency factors are used in the initial weight encoding. Results on classification of speech data demonstrate the superiority of the system over the fuzzy and conventional versions of the MLP.
---
paper_title: Knowledge-based fuzzy MLP for classification and rule generation

paper_content:

A new scheme of knowledge-based classification and rule generation using a fuzzy multilayer perceptron (MLP) is proposed. Knowledge collected from a data set is initially encoded among the connection weights in terms of class a priori probabilities. This encoding also includes incorporation of hidden nodes corresponding to both the pattern classes and their complementary regions. The network architecture, in terms of both links and nodes, is then refined during training. Node growing and link pruning are also resorted to. Rules are generated from the trained network using the input, output, and connection weights in order to justify any decision(s) reached. Negative rules corresponding to a pattern not belonging to a class can also be obtained. These are useful for inferencing in ambiguous cases. Results on real life and synthetic data demonstrate that the speed of learning and classification performance of the proposed scheme are better than that obtained with the fuzzy and conventional versions of the MLP (involving no initial knowledge encoding). Both convex and concave decision regions are considered in the process.
---
paper_title: Knowledge-based fuzzy MLP for classification and rule generation

paper_content:

A new scheme of knowledge-based classification and rule generation using a fuzzy multilayer perceptron (MLP) is proposed. Knowledge collected from a data set is initially encoded among the connection weights in terms of class a priori probabilities. This encoding also includes incorporation of hidden nodes corresponding to both the pattern classes and their complementary regions. The network architecture, in terms of both links and nodes, is then refined during training. Node growing and link pruning are also resorted to. Rules are generated from the trained network using the input, output, and connection weights in order to justify any decision(s) reached. Negative rules corresponding to a pattern not belonging to a class can also be obtained. These are useful for inferencing in ambiguous cases. Results on real life and synthetic data demonstrate that the speed of learning and classification performance of the proposed scheme are better than that obtained with the fuzzy and conventional versions of the MLP (involving no initial knowledge encoding). Both convex and concave decision regions are considered in the process.
---
paper_title: Multilayer perceptron, fuzzy sets, and classification

paper_content:

A fuzzy neural network model based on the multilayer perceptron, using the backpropagation algorithm, and capable of fuzzy classification of patterns is described. The input vector consists of membership values to linguistic properties while the output vector is defined in terms of fuzzy class membership values. This allows efficient modeling of fuzzy uncertain patterns with appropriate weights being assigned to the backpropagated errors depending upon the membership values at the corresponding outputs. During training, the learning rate is gradually decreased in discrete steps until the network converges to a minimum error solution. The effectiveness of the algorithm is demonstrated on a speech recognition problem. The results are compared with those of the conventional MLP, the Bayes classifier, and other related models. >
---
paper_title: Neural expert system using fuzzy teaching input and its application to medical diagnosis

paper_content:

Abstract Genetic algorithms (GAs) are inspired by Darwin's the survival of the fittest theory. This paper discusses a genetic algorithm that can automatically generate test cases to test a selected path. This algorithm takes a selected path as a target and executes sequences of operators iteratively for test cases to evolve. The evolved test case will lead the program execution to achieve the target path. To determine which test cases should survive to produce the next generation of fitter test cases, a metric named normalized extended Hamming distance (NEHD, which is used to determine whether the final test case is found) is developed. Based on NEHD, a fitness function named SIMILARITY is defined to determine which test cases should survive if the final test case has not been found. Even when there are loops in the target path, SIMILARITY can help the algorithm to lead the execution to flow along the target path.
---
paper_title: Knowledge-based fuzzy MLP for classification and rule generation

paper_content:

A new scheme of knowledge-based classification and rule generation using a fuzzy multilayer perceptron (MLP) is proposed. Knowledge collected from a data set is initially encoded among the connection weights in terms of class a priori probabilities. This encoding also includes incorporation of hidden nodes corresponding to both the pattern classes and their complementary regions. The network architecture, in terms of both links and nodes, is then refined during training. Node growing and link pruning are also resorted to. Rules are generated from the trained network using the input, output, and connection weights in order to justify any decision(s) reached. Negative rules corresponding to a pattern not belonging to a class can also be obtained. These are useful for inferencing in ambiguous cases. Results on real life and synthetic data demonstrate that the speed of learning and classification performance of the proposed scheme are better than that obtained with the fuzzy and conventional versions of the MLP (involving no initial knowledge encoding). Both convex and concave decision regions are considered in the process.
---
paper_title: Incorporating Fuzzy Membership Functions into the Perceptron Algorithm

paper_content:

The perceptron algorithm, one of the class of gradient descent techniques, has been widely used in pattern recognition to determine linear decision boundaries. While this algorithm is guaranteed to converge to a separating hyperplane if the data are linearly separable, it exhibits erratic behavior if the data are not linearly separable. Fuzzy set theory is introduced into the perceptron algorithm to produce a ``fuzzy algorithm'' which ameliorates the convergence problem in the nonseparable case. It is shown that the fuzzy perceptron, like its crisp counterpart, converges in the separable case. A method of generating membership functions is developed, and experimental results comparing the crisp to the fuzzy perceptron are presented.
---

Title: Neuro – Fuzzy Rule Generation: Survey in Soft Computing Framework

Section 1: NEURO-FUZZY AND SOFT COMPUTING
Description 1: This section focuses on different aspects of neuro-fuzzy computing, explaining the need for neuro-fuzzy integration, the different methods of neuro-fuzzy hybridization, and introducing the concept of soft computing.

Section 2: RULE GENERATION
Description 2: This section reviews the different fuzzy, neural, and neuro-fuzzy models for rule generation, inferencing, and querying, along with their salient features.

Section 3: USING KNOWLEDGE-BASED NETWORKS
Description 3: This section discusses embedding initial knowledge into the network topology of knowledge-based networks, refining the network for rule generation, and combining neuro-fuzzy concepts with genetic algorithms and rough sets.

Section 4: APPLICATION TO MEDICAL DIAGNOSIS
Description 4: This section describes the application of neuro-fuzzy models to medical diagnosis problems, including the generation of rules for medical data and handling incomplete or ambiguous information.

Section 5: CONCLUSIONS
Description 5: This section summarizes the survey, highlighting the categorization of neuro-fuzzy models under a unified soft computing framework, the inclusion of rule extraction and refinement, and observations on the convergence analysis and future research directions.