Support vector learning mechanism for fuzzy rule-based modeling: a new approach

This paper describes a fuzzy modeling framework based on support vector machine, a rule-based framework that explicitly characterizes the representation in fuzzy inference procedure. The support vector learning mechanism provides an architecture to extract support vectors for generating fuzzy IF-THEN rules from the training data set, and a method to describe the fuzzy system in terms of kernel functions. Thus, it has the inherent advantage that the model does not have to determine the number of rules in advance, and the overall fuzzy inference system can be represented as series expansion of fuzzy basis functions. The performance of the proposed approach is compared to other fuzzy rule-based modeling methods using four data sets.     

Adaptive fuzzy rule-based classification systems

This paper proposes an adaptive method to construct a fuzzy rule-based classification system with high performance for pattern classification problems. The proposed method consists of two procedures: an error correction-based learning procedure, and an additional learning procedure. The error correction-based learning procedure adjusts the grade of certainty of each fuzzy rule by its classification performance. That is, when a pattern is misclassified by a particular fuzzy rule, the grade of certainty of that rule is decreased. On the contrary, when a pattern is correctly classified, the grade of certainty is increased. Because the error correction-based learning procedure is not meaningful after all the given patterns are correctly classified, we cannot adjust a classification boundary in such a case. To acquire a more intuitively acceptable boundary, we propose an additional learning procedure. We also propose a method for selecting significant fuzzy rules by pruning unnecessary fuzzy rules, which consists of the error correction-based learning procedure and the concept of forgetting. We can construct a compact fuzzy rule-based classification system with high performance     

Missing data imputation for fuzzy rule-based classification systems

Fuzzy rule-based classification systems (FRBCSs) are known due to their ability to treat with low quality data and obtain good results in these scenarios. However, their application in problems with missing data are uncommon while in real-life data, information is frequently incomplete in data mining, caused by the presence of missing values in attributes. Several schemes have been studied to overcome the drawbacks produced by missing values in data mining tasks; one of the most well known is based on preprocessing, formerly known as imputation. In this work, we focus on FRBCSs considering 14 different approaches to missing attribute values treatment that are presented and analyzed. The analysis involves three different methods, in which we distinguish between Mamdani and TSK models. From the obtained results, the convenience of using imputation methods for FRBCSs with missing values is stated. The analysis suggests that each type behaves differently while the use of determined missing values imputation methods could improve the accuracy obtained for these methods. Thus, the use of particular imputation methods conditioned to the type of FRBCSs is required.     

An ensemble method for fuzzy rule-based classification systems

Fuzzy rule-based classification systems are very useful tools in the field of machine learning as they are able to build linguistic comprehensible models. However, these systems suffer from exponential rule explosion when the number of variables increases, degrading, therefore, the accuracy of these systems as well as their interpretability. In this article, we propose to improve the comprehensibility through a supervised learning method by automatic generation of fuzzy classification rules, designated SIFCO–PAF. Our method reduces the complexity by decreasing the number of rules and of antecedent conditions, making it thus adapted to the representation and the prediction of rather high-dimensional pattern classification problems. We perform, firstly, an ensemble methodology by combining a set of simple classification models. Subsequently, each model uses a subset of the initial attributes: In this case, we propose to regroup the attributes using linear correlation search among the training set elements. Secondly, we implement an optimal fuzzy partition thanks to supervised discretization followed by an automatic membership functions construction. The SIFCO–PAF method, analyzed experimentally on various data sets, guarantees an important reduction in the number of rules and of antecedents without deteriorating the classification rates, on the contrary accuracy is even improved.     

Supervised methods for regrouping attributes in fuzzy rule-based classification systems

This paper focuses on ensemble methods for Fuzzy Rule-Based Classification Systems (FRBCS) where the decisions of different classifiers are combined in order to form the final classification model. The proposed methods reduce the FRBCS complexity and the generated rules number. We are interested in particular in ensemble methods which cluster the attributes into subgroups of attributes and treat each subgroup separately. Our work is an extension of a previous ensemble method called SIFRA. This method uses frequent itemsets mining concept in order to deduce the groups of related attributes by analyzing their simultaneous appearances in the databases. The drawback of this method is that it forms the groups of attributes by searching for dependencies between the attributes independently from the class information. Besides, since we deal with supervised learning problems, it would be very interesting to consider the class attribute when forming the attributes subgroups. In this paper, we proposed two new supervised attributes regrouping methods which take into account not only the dependencies between the attributes but also the information about the class labels. The results obtained with various benchmark datasets show a good accuracy of the built classification model.     

GP-COACH: Genetic Programming-based learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems

In this paper we propose GP-COACH, a Genetic Programming-based method for the learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems. GP-COACH learns disjunctive normal form rules (generated by means of a context-free grammar) coded as one rule per tree. The population constitutes the rule base, so it is a genetic cooperative-competitive learning approach. GP-COACH uses a token competition mechanism to maintain the diversity of the population and this obliges the rules to compete and cooperate among themselves and allows the obtaining of a compact set of fuzzy rules. The results obtained have been validated by the use of non-parametric statistical tests, showing a good performance in terms of accuracy and interpretability.     

Effect of rule weights in fuzzy rule-based classification systems

This paper examines the effect of rule weights in fuzzy rule-based classification systems. Each fuzzy IF-THEN rule in our classification system has antecedent linguistic values and a single consequent class. We use a fuzzy reasoning method based on a single winner rule in the classification phase. The winner rule for a new pattern is the fuzzy IF-THEN rule that has the maximum compatibility grade with the new pattern. When we use fuzzy IF-THEN rules with certainty grades, the winner is determined as the rule with the maximum product of the compatibility grade and the certainty grade. In this paper, the effect of rule weights is illustrated by drawing classification boundaries using fuzzy IF-THEN rules with/without certainty grades. It is also shown that certainty grades play an important role when a fuzzy rule-based classification system is a mixture of general rules and specific rules. Through computer simulations, we show that comprehensible fuzzy rule-based systems with high classification performance can be designed without modifying the membership functions of antecedent linguistic values when we use fuzzy IF-THEN rules with certainty grades     

hGA: Hybrid genetic algorithm in fuzzy rule-based classification systems for high-dimensional problems

The aim of this work is to propose a hybrid heuristic approach (called hGA) based on genetic algorithm (GA) and integer-programming formulation (IPF) to solve high dimensional classification problems in linguistic fuzzy rule-based classification systems. In this algorithm, each chromosome represents a rule for specified class, GA is used for producing several rules for each class, and finally IPF is used for selection of rules from a pool of rules, which are obtained by GA. The proposed algorithm is experimentally evaluated by the use of non-parametric statistical tests on seventeen classification benchmark data sets. Results of the comparative study show that hGA is able to discover accurate and concise classification rules.     

Weighted fuzzy interpolative reasoning for sparse fuzzy rule-based systems

In this paper, we present a weighted fuzzy interpolative reasoning method for sparse fuzzy rule-based systems, where the antecedent variables appearing in the fuzzy rules have different weights. We also present a weights-learning algorithm to automatically learn the optimal weights of the antecedent variables of the fuzzy rules for the proposed weighted fuzzy interpolative reasoning method. We also apply the proposed weighted fuzzy interpolative reasoning method and the proposed weights-learning algorithm to handle the truck backer-upper control problem. The experimental results show that the proposed fuzzy interpolative reasoning method using the optimally learned weights by the proposed weights-learning algorithm gets better truck backer-upper control results than the ones by the traditional fuzzy inference system and the existing fuzzy interpolative reasoning methods. The proposed method provides us with a useful way for fuzzy rules interpolation in sparse fuzzy rule-based systems.     

Performance evaluation of fuzzy rule-based classification systems obtained by multi-objective genetic algorithms

In this paper, we examine the classification performance of fuzzy if-then rules selected by a GA-based multi-objective rule selection method. This rule selection method can be applied to high-dimensional pattern classification problems with many continuous attributes by restricting the number of antecedent conditions of each candidate fuzzy if-then rule. As candidate rules, we only use fuzzy if-then rules with a small number of antecedent conditions. Thus it is easy for human users to understand each rule selected by our method. Our rule selection method has two objectives: to minimize the number of selected fuzzy if-then rules and to maximize the number of correctly classified patterns. In our multi-objective fuzzy rule selection problem, there exist several solutions (i.e., several rule sets) called “non-dominated solutions” because two conflicting objectives are considered. In this paper, we examine the performance of our GA-based rule selection method by computer simulations on a real-world pattern classification problem with many continuous attributes. First we examine the classification performance of our method for training patterns by computer simulations. Next we examine the generalization ability for test patterns. We show that a fuzzy rule-based classification system with an appropriate number of rules has high generalization ability.     

A multi-objective evolutionary method for learning granularities based on fuzzy discretization to improve the accuracy-complexity trade-off of fuzzy rule-based classification systems: D-MOFARC algorithm

Multi-objective evolutionary algorithms represent an effective tool to improve the accuracy-interpretability trade-off of fuzzy rule-based classification systems. To this aim, a tuning process and a rule selection process can be combined to obtain a set of solutions with different trade-offs between the accuracy and the compactness of models. Nevertheless, an initial model needs to be defined, in particular the parameters that describe the partitions and the number of fuzzy sets of each variable (i.e. the granularities) must be determined. The simplest approach is to use a previously established single granularity and a uniform fuzzy partition for each variable. A better approach consists in automatically identifying from data the appropriate granularities and fuzzy partitions, since this usually leads to more accurate models.This contribution presents a fuzzy discretization approach, which is used to generate automatically promising granularities and their associated fuzzy partitions. This mechanism is integrated within a Multi-Objective Fuzzy Association Rule-Based Classification method, namely D-MOFARC, which concurrently performs a tuning and a rule selection process on an initial knowledge base. The aim is to obtain fuzzy rule-based classification systems with high classification performances, while preserving their complexity.     

Support vector machines with genetic fuzzy feature transformation for biomedical data classification

In this paper, we present a genetic fuzzy feature transformation method for support vector machines (SVMs) to do more accurate data classification. Given data are first transformed into a high feature space by a fuzzy system, and then SVMs are used to map data into a higher feature space and then construct the hyperplane to make a final decision. Genetic algorithms are used to optimize the fuzzy feature transformation so as to use the newly generated features to help SVMs do more accurate biomedical data classification under uncertainty. The experimental results show that the new genetic fuzzy SVMs have better generalization abilities than the traditional SVMs in terms of prediction accuracy.     

Matrix formulation of fuzzy rule-based systems

In this paper, a matrix formulation of fuzzy rule based systems is introduced. A gradient descent training algorithm for the determination of the unknown parameters can also be expressed in a matrix form for various adaptive fuzzy networks. When converting a rule-based system to the proposed matrix formulation, only three sets of linear/nonlinear equations are required instead of set of rules and an inference mechanism. There are a number of advantages which the matrix formulation has compared with the linguistic approach. Firstly, it obviates the differences among the various architectures; and secondly, it is much easier to organize data in the implementation or simulation of the fuzzy system. The formulation will be illustrated by a number of examples.     

Improving the performance of fuzzy rule-based classification systems with interval-valued fuzzy sets and genetic amplitude tuning

Among the computational intelligence techniques employed to solve classification problems, Fuzzy Rule-Based Classification Systems (FRBCSs) are a popular tool because of their interpretable models based on linguistic variables, which are easier to understand for the experts or end-users.The aim of this paper is to enhance the performance of FRBCSs by extending the Knowledge Base with the application of the concept of Interval-Valued Fuzzy Sets (IVFSs). We consider a post-processing genetic tuning step that adjusts the amplitude of the upper bound of the IVFS to contextualize the fuzzy partitions and to obtain a most accurate solution to the problem.We analyze the goodness of this approach using two basic and well-known fuzzy rule learning algorithms, the Chi et al.’s method and the fuzzy hybrid genetics-based machine learning algorithm. We show the improvement achieved by this model through an extensive empirical study with a large collection of data-sets.     

A fuzzy rule-based meta-scheduler with evolutionary learning for grid computing

Grid computing is increasingly emerging as a promising platform for large-scale problems solving in science, engineering and technology. Nevertheless, a major effort is still required to harness the high potential performance of such computational framework and in this sense, an important challenge is to develop new strategies that efficiently address scheduling on the distributed, heterogeneous and shared environment of grids. Fuzzy rule-based systems (FRBSs) models are dynamic and are currently attracting the interest of scheduling research community to obtain near-optimal solutions on grids. However, FRBSs performance is strongly related to the quality of their knowledge bases and thus, with the knowledge acquisition process. Due to the inherent dynamic nature and the typical complex search spaces of grids, automatically finding a high-quality knowledge base that accurately describes the fuzzy system is extremely relevant. In this work, we propose a scheduling system for grids considering a novel learning strategy inspired by Michigan and Pittsburgh approaches that applies genetic algorithms (GAs) to evolve the fuzzy rule bases and improves the classical learning strategies in terms of computational effort and convergence behaviour. In addition, experimental results show that the proposed schema significantly outperforms other extensively used scheduling strategies.     

Structure identification in complete rule-based fuzzy systems

The identification of a model is one of the key issues in the field of fuzzy system modeling and function approximation theory. There are numerous approaches to the issue of parameter optimization within a fixed fuzzy system structure but no reliable method to obtain the optimal topology of the fuzzy system from a set of input-output data. This paper presents a reliable method to obtain the structure of a complete rule-based fuzzy system for a specific approximation accuracy of the training data, i.e., it can decide which input variables must be taken into account in the fuzzy system and how many membership functions (MFs) are needed in every selected input variable in order to reach the approximation target with the minimum number of parameters     

Support vector machines for texture classification

This paper investigates the application of support vector machines (SVMs) in texture classification. Instead of relying on an external feature extractor, the SVM receives the gray-level values of the raw pixels, as SVMs can generalize well even in high-dimensional spaces. Furthermore, it is shown that SVMs can incorporate conventional texture feature extraction methods within their own architecture, while also providing solutions to problems inherent in these methods. One-against-others decomposition is adopted to apply binary SVMs to multitexture classification, plus a neural network is used as an arbitrator to make final classifications from several one-against-others SVM outputs. Experimental results demonstrate the effectiveness of SVMs in texture classification.     

A neuro-fuzzy scheme for simultaneous feature selection and fuzzy rule-based classification

Most methods of classification either ignore feature analysis or do it in a separate phase, offline prior to the main classification task. This paper proposes a neuro-fuzzy scheme for designing a classifier along with feature selection. It is a four-layered feed-forward network for realizing a fuzzy rule-based classifier. The network is trained by error backpropagation in three phases. In the first phase, the network learns the important features and the classification rules. In the subsequent phases, the network is pruned to an "optimal" architecture that represents an "optimal" set of rules. Pruning is found to drastically reduce the size of the network without degrading the performance. The pruned network is further tuned to improve performance. The rules learned by the network can be easily read from the network. The system is tested on both synthetic and real data sets and found to perform quite well.     

A fuzzy reasoning approach for rule-based systems based on fuzzylogics

This paper presents a weighted fuzzy reasoning algorithm for rule-based systems based on weighted fuzzy logics. The proposed algorithm allows the truth values of the conditions appearing in the antecedent portions of the rules, the certainty factors of the rules, and the weights of the conditions appearing in the antecedent portions of the rules to be represented by trapezoidal fuzzy numbers. Given the fuzzy truth values of some conditions, the algorithm can perform weighted fuzzy reasoning to evaluate the fuzzy truth values of other conditions automatically.