Fuzzy regression using least absolute deviation estimators

In fuzzy regression, that was first proposed by Tanaka et al. (Eur J Oper Res 40:389–396, 1989; Int Cong Appl Syst Cybern 4:2933–2938, 1980; IEEE Trans SystMan Cybern 12:903–907, 1982), there is a tendency that the greater the values of independent variables, the wider the width of the estimated dependent variables. This causes a decrease in the accuracy of the fuzzy regression model constructed by the least squares method. This paper suggests the least absolute deviation estimators to construct the fuzzy regression model, and investigates the performance of the fuzzy regression models with respect to a certain errormeasure. Simulation studies and examples show that the proposed model produces less error than the fuzzy regression model studied by many authors that use the least squares method when the data contains fuzzy outliers.     

Fuzzy linear regression model based on fuzzy scalar product

The new concept and method of imposing imprecise (fuzzy) input and output data upon the conventional linear regression model is proposed in this paper. We introduce the fuzzy scalar (inner) product to formulate the fuzzy linear regression model. In order to invoke the conventional approach of linear regression analysis for real-valued data, we transact the α-level linear regression models of the fuzzy linear regression model. We construct the membership functions of fuzzy least squares estimators via the form of “Resolution Identity” which is a well-known formula in fuzzy sets theory. In order to obtain the membership value of any given least squares estimate taken from the fuzzy least squares estimator, we transform the original problem into the optimization problems. We also provide two computational procedures to solve the optimization problems.     

Fuzzy clusterwise linear regression analysis with symmetrical fuzzy output variable

The traditional regression analysis is usually applied to homogeneous observations. However, there are several real situations where the observations are not homogeneous. In these cases, by utilizing the traditional regression, we have a loss of performance in fitting terms. Then, for improving the goodness of fit, it is more suitable to apply the so-called clusterwise regression analysis. The aim of clusterwise linear regression analysis is to embed the techniques of clustering into regression analysis. In this way, the clustering methods are utilized for overcoming the heterogeneity problem in regression analysis. Furthermore, by integrating cluster analysis into the regression framework, the regression parameters (regression analysis) and membership degrees (cluster analysis) can be estimated simultaneously by optimizing one single objective function. In this paper the clusterwise linear regression has been analyzed in a fuzzy framework. In particular, a fuzzy clusterwise linear regression model (FCWLR model) with symmetrical fuzzy output and crisp input variables for performing fuzzy cluster analysis within a fuzzy linear regression framework is suggested. For measuring the goodness of fit of the suggested FCWLR model with fuzzy output, a fitting index is proposed. In order to illustrate the usefulness of FCWLR model in practice, several applications to artificial and real datasets are shown.     

Fuzzy regression analysis: Systematic review and bibliography

Statistical regression analysis is a powerful and reliable method to determine the impact of one or several independent variable(s) on a dependent variable. It is the most widely used of all statistical methods and has broad applicability to numerous practical problems. However, various problems can arise, when for instance the sample size is too small, distributional assumptions are not fulfilled, the relationship between independent and dependent variables is vague or when there is an ambiguity of events. Moreover, the complexity of real-life problems often makes the underlying models inadequate, since information is frequently imprecise in many ways. To relax these rigidities, numerous researchers have modified and extended concepts of statistical regression analysis by means of concepts of fuzzy set theory. By now, there is a large number of papers on the topic of fuzzy regression analysis, especially concerning possibilistic, fuzzy least squares or machine learning approaches. Additionally, the variety of approaches includes probabilistic, logistic, type-2 and clusterwise fuzzy regression methods, among many others. Besides papers mainly devoted to advances in methodology, there are also several papers presenting case studies in various research fields. To structure this diversity of papers, proposals and applications we give in this paper a comprehensive systematic review and provide a bibliography on the topic of fuzzy regression analysis. Thus, the paper intends to consolidate the topic in order to aid new researchers in this area, focuses the field’s attention on key open questions, and highlights possible directions for future research.     

The construction of fuzzy least squares estimators in fuzzy linear regression models

A new concept and method of imposing imprecise (fuzzy) input and output data upon the conventional linear regression model is proposed. Under the considerations of fuzzy parameters and fuzzy arithmetic operations (fuzzy addition and multiplication), we propose a fuzzy linear regression model which has the similar form as that of conventional one. We conduct the h-level (conventional) linear regression models of fuzzy linear regression model for the sake of invoking the statistical techniques in (conventional) linear regression analysis for real-valued data. In order to determine the sign (nonnegativity or nonpositivity) of fuzzy parameters, we perform the statistical testing hypotheses and evaluate the confidence intervals. Using the least squares estimators obtained from the h-level linear regression models, we can construct the membership functions of fuzzy least squares estimators via the form of “Resolution Identity” which is well-known in fuzzy sets theory. In order to obtain the membership degree of any given estimate taken from the fuzzy least squares estimator, optimization problems have to be solved. We also provide two computational procedures to deal with those optimization problems.     

Linear regression analysis for fuzzy input and output data using the extension principle

The method for obtaining the fuzzy least squares estimators with the help of the extension principle in fuzzy sets theory is proposed. The membership functions of fuzzy least squares estimators will be constructed according to the usual least squares estimators. In order to obtain the membership value of any given value taken from the fuzzy least squares estimator, optimization problems have to be solved. We also provide the methodology for evaluating the predicted fuzzy output from the given fuzzy input data.     

A note on model-free regression capabilities of fuzzy systems.

Nonparametric estimation capabilities of fuzzy systems in stochastic environments are analyzed in this paper. By using ideas from sieve estimation, increasing sequences of fuzzy rule-based systems capable of consistently estimating arbitrary regression surfaces are constructed. Results include least squares learning of a mapping perturbed by additive random noise in a static-regression context. L1 (i.e., least absolute deviation) estimation is also studied, and the consistency of fuzzy rule-based sieve estimators for L1-optimal regression surfaces is shown, thus giving additional theoretical support to the robust filtering capabilities of fuzzy systems and their adequacy for modeling, prediction, and control of systems affected by impulsive noise.     

Sampling Minimal Subsets with Large Spans for Robust Estimation

When sampling minimal subsets for robust parameter estimation, it is commonly known that obtaining an all-inlier minimal subset is not sufficient; the points therein should also have a large spatial extent. This paper investigates a theoretical basis behind this principle, based on a little known result which expresses the least squares regression as a weighted linear combination of all possible minimal subset estimates. It turns out that the weight of a minimal subset estimate is directly related to the span of the associated points. We then derive an analogous result for total least squares which, unlike ordinary least squares, corrects for errors in both dependent and independent variables. We establish the relevance of our result to computer vision by relating total least squares to geometric estimation techniques. As practical contributions, we elaborate why naive distance-based sampling fails as a strategy to maximise the span of all-inlier minimal subsets produced. In addition we propose a novel method which, unlike previous methods, can consciously target all-inlier minimal subsets with large spans.     

Comparison of fuzzy logic based models for the multi-response surface problems with replicated response measures

A replicated multi-response experiment is a process that includes more than one responses with replications. One of the main objectives in these experiments is to estimate the unknown relationship between responses and input variables simultaneously. In general, classical regression analysis is used for modeling of the responses. However, in most practical problems, the assumptions for regression analysis cannot be satisfied. In this case, alternative modeling methods such as fuzzy logic based modeling approaches can be used. In this study, fuzzy least squares regression (FLSR) and fuzzy clustering based modeling methods, which are switching fuzzy C-regression (SFCR) and Takagi–Sugeno (TS) fuzzy model, are preferred. The novelty of the study is presenting the applicability of SFCR to the multi-response experiment data set with replicated response measures. Three real data set examples are given for application purposes. In order to compare the prediction performance of modeling approaches, root mean square error (RMSE) criteria is used. It is seen from the results that the SFCR gives the better prediction performance among the other fuzzy modeling approaches for the replicated multi-response experimental data sets.     

Fuzzy nonparametric regression based on local linear smoothing technique

In a great deal of literature on fuzzy regression analysis, most of research has focused on some predefined parametric forms of fuzzy regression relationships, especially on the fuzzy linear regression models. In many practical situations, it may be unrealistic to predetermine a fuzzy parametric regression relationship. In this paper, a fuzzy nonparametric model with crisp input and LR fuzzy output is considered and, based on the distance measure for fuzzy numbers suggested by Diamond [P. Diamond, Fuzzy least squares, Information Sciences 46 (1988) 141-157], the local linear smoothing technique in statistics with the cross-validation procedure for selecting the optimal value of the smoothing parameter is fuzzified to fit this model. Some simulation experiments are conducted to examine the performance of the proposed method and three real-world datasets are analyzed to illustrate the application of the proposed method. The results demonstrate that the proposed method works quite well not only in producing satisfactory estimate of the fuzzy regression function, but also in reducing the boundary effect significantly.     

Integer Matrix Approximation and Data Mining

Integer datasets frequently appear in many applications in science and engineering. To analyze these datasets, we consider an integer matrix approximation technique that can preserve the original dataset characteristics. Because integers are discrete in nature, to the best of our knowledge, no previously proposed technique developed for real numbers can be successfully applied. In this study, we first conduct a thorough review of current algorithms that can solve integer least squares problems, and then we develop an alternative least square method based on an integer least squares estimation to obtain the integer approximation of the integer matrices. We discuss numerical applications for the approximation of randomly generated integer matrices as well as studies of association rule mining, cluster analysis, and pattern extraction. Our computed results suggest that our proposed method can calculate a more accurate solution for discrete datasets than other existing methods.     

Adaptive least squares method for estimation of partial cloud coverage within a pixel

A study of the estimation of partial cloud cover within a pixel has been conducted in order to be able to use pixels partially contaminated with cloud in sea surface temperature determination.

The existing estimation methods based on the least squares method with constraints of minimizing the mixing ratio and observation vector, are theoretically compared and then an adaptive least squares method is proposed. In a comparative study the estimation accuracies for the proposed and other existing methods, including the maximum likelihood method, are compared with simulated and real satellite image data of NOAA AVHRR and MOS-1 VTIR. The results with the simulation data show that the maximum likelihood method is best followed by the adaptive least squares method, the least squares method and the observation vector, while the results with the real VTIR data show that the proposed adaptive least squares method is best followed by the least squares method, the maximum likelihood method and the observation vector but there is no significant differences between all these methods.     

Fuzzy functions based ARX model and new fuzzy basis function models for nonlinear system identification

In this study, auto regressive with exogenous input (ARX) modeling is improved with fuzzy functions concept (FF-ARX). Fuzzy function with least squares estimation (FF-LSE) method has been recently developed and widely used with a small improvement with respect to least squares estimation method (LSE). FF-LSE is structured with only inputs and their membership values. This proposed model aims to increase the capability of the FF-LSE by widening the regression matrix with lagged input–output values. In addition, by using same idea, we proposed also two new fuzzy basis function models. In the first, basis of the fuzzy system and lagged input–output values are structured together in the regression matrix and named as “L-FBF”. Secondly, instead of using basis function, the membership values of the lagged input–output values are used in the regression matrix by using Gaussian membership functions, called “M-FBF”. Therefore, the power of the fuzzy basis function is also enhanced. For the corresponding models, antecedent part parameters for the input vectors are determined with fuzzy c-means (FCM) clustering algorithm. The consequent parameters of the all models are estimated with the LSE. The proposed models are utilized and compared for the identification of nonlinear benchmark problems.     

Fuzzy rule-based support vector regression system

In this paper,we design a fuzzy rule-based support vector regression system.The proposed system utilizes the advantages of fuzzy model and support vector regression to extract support vectors to generate fuzzy if-then rules from the training data set.Based on the first-order linear Tagaki-Sugeno (TS) model,the structure of rules is identified by the support vector regression and then the consequent parameters of rules are tuned by the global least squares method.Our model is applied to the real world regression task.The simulation results gives promising performances in terms of a set of fuzzy rules,which can be easily interpreted by humans.     

Fuzzy nile-based support vector regression system

In this paper, we design a fuzzy rule-based support vector regression system. The proposed system utilizes the advantages of fuzzy model and support vector regression to extract support vectors to generate fuzzy if-then rules from the training data set. Based on the first-order hnear Tagaki-Sugeno （TS） model, the structure of rules is identified by the support vector regression and then the consequent parameters of rules are tuned by the global least squares method. Our model is applied to the real world regression task. The simulation results gives promising performances in terms of a set of fuzzy hales, which can be easily interpreted by humans.     

Fuzzy lagged variable selection in fuzzy time series with genetic algorithms

Fuzzy time series forecasting models can be divided into two subclasses which are first order and high order. In high order models, all lagged variables exist in the model according to the model order. Thus, some of these can exist in the model although these lagged variables are not significant in explaining fuzzy relationships. If such lagged variables can be removed from the model, fuzzy relationships will be defined better and it will cause more accurate forecasting results. In this study, a new fuzzy time series forecasting model has been proposed by defining a partial high order fuzzy time series forecasting model in which the selection of fuzzy lagged variables is done by using genetic algorithms. The proposed method is applied to some real life time series and obtained results are compared with those obtained from other methods available in the literature. It is shown that the proposed method has high forecasting accuracy.     

Interval analysis using least squares support vector fuzzy regression

A least squares support vector fuzzy regression model(LS-SVFR) is proposed to estimate uncertain and imprecise data by applying the fuzzy set principle to weight vectors.This model only requires a set of linear equations to obtain the weight vector and the bias term,which is different from the solution of a complicated quadratic programming problem in existing support vector fuzzy regression models.Besides,the proposed LS-SVFR is a model-free method in which the underlying model function doesn’t need to be predefined.Numerical examples and fault detection application are applied to demonstrate the effectiveness and applicability of the proposed model.     

Dynamic Fuzzy Rating Tracker (DYFRAT): a novel methodology for modeling real-time dynamic cognitive processes in rating scales

Rating scales (such as, Likert scales, Guttman scales, Feelings thermometers, etc.) represent simple tools for measuring attitudes, judgements and subjective preferences in human rating contexts. Because rating scales show some useful properties (e.g., measurement uniformity, considerable flexibility, statistically appealing), they represent popular and reliable instruments in socio-behavioral sciences. However, standard rating scales suffer also from some relevant limitations. For example, they fail in measuring vague and imprecise information and, above all, they are only able to capture the final outcome of the cognitive process of rating (i.e., the rater's response). To overcome these limitations, some fuzzy versions of these scales (e.g., fuzzy conversion scales, fuzzy rating scales) have been proposed over the years. However, also these more sophisticated scales show some important shortcomings (e.g., difficulty in fuzzy variables construction and potential lack of ecological validity). In this paper, we propose a novel methodology (DYFRAT) for modeling human rating evaluations from a fuzzy-set perspective. In particular, DYFRAT captures the fuzziness of human ratings by modeling some real-time biometric events that occur during the cognitive process of rating in an ecological measurement setting. Moreover, in order to show some important characteristics of the proposed methodology, we apply DYFRAT to some empirical rating situations concerning decision making and risk assessment scenarios.     

Formulation of linguistic regression model based on natural words

When human experts express their ideas and thoughts, human words are basically employed in these expressions. That is, the experts with much professional experiences are capable of making assessment using their intuition and experiences. The measurements and interpretation of characteristics are taken with uncertainty, because most measured characteristics, analytical result, and field data can be interpreted only intuitively by experts. In such cases, judgments may be expressed using linguistic terms by experts. The difficulty in the direct measurement of certain characteristics makes the estimation of these characteristics imprecise. Such measurements may be dealt with the use of fuzzy set theory. As Professor L. A. Zadeh has placed the stress on the importance of the computation with words, fuzzy sets can take a central role in handling words [12, 13]. In this perspective fuzzy logic approach is offten thought as the main and only useful tool to deal with human words. In this paper we intend to present another approach to handle human words instead of fuzzy reasoning. That is, fuzzy regression analysis enables us treat the computation with words. In order to process linguistic variables, we define the vocabulary translation and vocabulary matching which convert linguistic expressions into membership functions on the interval [0–1] on the basis of a linguistic dictionary, and vice versa. We employ fuzzy regression analysis in order to deal with the assessment process of experts from linguistic variables of features and characteristics of an objective into the linguistic expression of the total assessment. The presented process consists of four portions: (1) vocabulary translation, (2) estimation, (3) vocabulary matching and (4) dictionary. We employed fuzzy quantification theory type 2 for estimating the total assessment in terms of linguistic structural attributes which are obtained from an expert.This research was supported in part by Grant-in Aid for Scientific Research(C-2); Grant No.11680459 of Ministry of Education of Science, Sports and Culture.     

Simulation of fuzzy random variables

This work deals with the simulation of fuzzy random variables, which can be used to model various realistic situations, where uncertainty is not only present in form of randomness but also in form of imprecision, described by means of fuzzy sets. Utilizing the common arithmetics in the space of all fuzzy sets only induces a conical structure. As a consequence, it is difficult to directly apply the usual simulation techniques for functional data. In order to overcome this difficulty two different approaches based on the concept of support functions are presented. The first one makes use of techniques for simulating Hilbert space-valued random elements and afterwards projects on the cone of all fuzzy sets. It is shown by empirical results that the practicability of this approach is limited. The second approach imitates the representation of every element of a separable Hilbert space in terms of an orthonormal basis directly on the space of fuzzy sets. In this way, a new approximation of fuzzy sets useful to approximate and simulate fuzzy random variables is developed. This second approach is adequate to model various realistic situations.